Discuss: How to handle shared files with GDA

[DominicOram]

Currently there are a number of "config" files that GDA and artemis both need:

• The lookup tables to detector distance to beam x/y (see https://github.com/DiamondLightSource/python-artemis/issues/29)
• A lot of the nexus metadata that is hardcoded in https://github.com/DiamondLightSource/python-artemis/pull/52 is in json files in /i03-config/etc
• Other stuff I've missed?

We should have a holistic discussion about how artemis should get this data, some potential options:

1. GDA passes the data to Artemis
2. GDA passes paths to Artemis of where the data is - actual files are still stored in GDA
3. GDA and Artemis both pull the data from a particular central place e.g. git repo or redis
4. Artemis also has the values in a file in this repo or hardcoded in this repo
5. We push all the data into EPICS and read it from there in both Artemis/GDA

Acceptance Criteria

• We decide which to do and document it here, along with why

[DominicOram]

My two cents:

1. This makes Artemis fiddly to run standalone
2. Slightly less fiddly to run Artemis standalone but would still be annoying
3. This is my favorite option, will involve modifications to GDA though so holds some risk. This also feeds in discussions that are ongoing within GDA on how to store configuration files. However, I'm reluctant to wait on the conclusion of those discussions for this decision to be made as it will slow things down
4. This means if we have to change any of these parameters we would have to do so twice
5. I like the idea of EPICS being the universal source of truth on this but we would have to discuss with controls and it's really just pushing some of the issues on to them

[coretl]

FYI, I was pointed to ADRs as a lightweight but easily browsable way to document decisions made on a project, although I guess GitHub issues are even more lightweight.

Anyway, here's how I store them for ophyd.v2: https://github.com/dls-controls/ophyd/tree/master/docs/explanations/decisions And here's how they look in the docs: https://dls-controls.github.io/ophyd/master/explanations/decisions.html

I went with RST to fit in with the sphinx project, but the original authors of ADRs used markdown which fits better with what you're doing.

[noemifrisina]

Adding my two cents.

I am not a fan of the first two options, if you want to run Artemis separated from GDA. And considering we're mostly talking of hardcoded values, it just seems like you'd be adding an avoidable additional layer to the code. So far I've been saving the values I need for Nexus writing in a local file, but I recognise it might not be the best option. It works well enough and it's convenient to have everything in the same place, although as @DominicOram mentioned, that means having to make changes twice if there's something to be fixed. It's not a huge issue, just an annoying one. In theory, I do like the idea of pulling these config files from a central location for both GDA and Artemis. However, my understanding of GDA is probably too limited to have a clear idea of if and how this could work.

[DominicOram]

This is getting more important now that beamline parameters are being required for setting up snapshots, see https://github.com/DiamondLightSource/python-artemis/pull/121/

[DominicOram]

@callumforrester is there any discussion of how this will be handled in the new architecture in the future?

[callumforrester]

In general we want to be less reliant on files in the future where possible as they don't make for a great API, so I'm not a fan of 2 or 4. This is especially important in a multi-application land (e.g. Artemis + GDA). Agree with @noemifrisina on 1 and 2 so I would generally favour 3 and 5, particularly 3. Having a single source of truth should lead to a simple codebase and commissioning procedure.