Hi Pascal,
The enabler epic is called “Nexus file handling consolidation”. After speaking with Martyn, its become clear that we currently access nexus files in three different ways across the Mantid code base:
This spread of mechanisms presents several challenges for developers including increased maintenance, a requirement for more domain knowledge, and repeated bugs in near duplicate code across each
implementation. Furthermore, our own internal wrapper around the NeXus API causes performance bottlenecks, and it is also believed that the NeXus API is less performant than the HDF5 API.
Given that all modern files are HDF5 (with the possible exception of the HDF4 files created by Muon?), we wish to consolidate these mechanisms so that we only use the HDF5 API and a more performant
internal wrapper around the HDF5 API. This will make it easier to identify other performance bottlenecks in the loading process, and might also provide a performance improvement in and of itself.
So in short, this enabler epic is related to the “Nexus load optimization” epic. It should make it easier for developers to identify bottlenecks, and may also provide a performance improvement.
What are the available routines that we want to benchmark against?
Thanks,
Rob
From: Pascal Manuel - STFC UKRI <pascal.manuel@stfc.ac.uk>
Sent: Tuesday, December 19, 2023 5:26 PM
To: Foxley, Sarah (STFC,RAL,ISIS) <sarah.foxley@stfc.ac.uk>
Cc: King, Stephen (STFC,RAL,ISIS) <stephen.king@stfc.ac.uk>; mpb@mantidproject.org
Subject: [Mpb] Re: Nexus loading epic - stakeholders needed
Hi Sarah,
I think the project is simple enough; improve loading speed so I am not sure we should really absolutely every group represented there but I am happy to take the hit for diffraction. The test will be in
speed improvement and it should be done with loading a certain number of big files
and also loading a big number of small files to be representative of everyone’s need.
There are a few available routines that seem to do a quicker job at the moment that we should benchmark against.
What is the enabler project?
Best
P
Sent from my iPhone
On 19 Dec 2023, at 16:55, Sarah Foxley - STFC UKRI <sarah.foxley@stfc.ac.uk> wrote:
Hi Steve,
That’s great thanks. Should we have someone from Reflectometry too or is Dirk ok to represent them too?
Thanks
Sarah
From: Stephen King - STFC UKRI <stephen.king@stfc.ac.uk>
Sent: 19 December 2023 16:45
To: mpb@mantidproject.org
Subject: [Mpb] Re: Nexus loading epic - stakeholders needed
Hi Sarah,
Dirk Honecker has volunteered for SANS.
Thanks,
Steve
From: Sarah Foxley - STFC UKRI <sarah.foxley@stfc.ac.uk>
Sent: 18 December 2023 16:22
To: mpb@mantidproject.org
Subject: [Mpb] Nexus loading epic - stakeholders needed
Dear all,
We are progressing with drafting two epics to improve Nexus loading (main project and an enabler project) with the hope of putting them up for review at the next meeting in February. For the main epic we need key stakeholders from the different science groups. At the moment Rob has listed board members. However I wondered if there were any others he could approach instead? I’m more than happy if you all feel you would be appropriate stakeholders but wanted to make sure before we finish drafting this one.
For reference this is the main epic that I’m needing stakeholders for - https://isisneutronmuon.atlassian.net/jira/software/c/projects/SS/boards/5?selectedIssue=SS-51
For info – both of the epics are a bit more fully formed than when Pas looked last week. Unfortunately Rob’s initial attempts to upload the information did not work! All resolved now though.
Please could you let me know of any alternative/additional stakeholders by 12th January?
Thanks
Sarah
Sarah Foxley
Mantid Team Leader and Mantid Project Manager
Science and Technology Facilities Council
Phone – 01235 446938
She/Her/Hers
<image001.png>
_______________________________________________
Mantid Programme Board mailing list -- mpb@mantidproject.org
To unsubscribe send an email to mpb-leave@mantidproject.org