Closed harryjmoss closed 1 year ago
@JBorrow the time allocated to the team at UCL to work on this has now ended so they're handing what they have (which is now fairly mature) over to us. I think that what we should do from here is:
{"flamingos_NxxxxLxxxx_physicslabel_snapnum": "/cosma8/.../snap.hdf5"}
, and then a user should be able to set up a RemoteSWIFTDataset
with a label plus their credentials.If you agree with step 1, at least, then please go ahead and approve the merge to a branch and we can work from there :)
@MatthieuSchaller you likely want to be aware of this PR, maybe you have input on the proposed workflow from here.
Actually there's no reason not to merge this into a development branch, and I need to make a few commits to fix bugs due to the new SWIFTUnits._handle
attribute - and don't have push permission on the fork. @JBorrow detailed review can wait for the eventual PR to master, but perhaps worth having a browse of the branch in the meantime.
Thanks @JBorrow. It's on my list to make a draft PR for this branch with a to-do list before it can be merged. Getting testing set up alone is going to be a big job. Ideally a lot of existing tests should be refactored to use fixtures so that we can test both the stand-alone and "client" SWIFTDatasets. Always hesitant to start making significant changes to tests, though, so will want to brainstorm that a bit before investing any effort. Figuring out how to host a server for the tests is also a bit of a tricky one - probably possible but not similar to anything I've tried before. Lots of other little stuff to do, too.
As part of work between UCL and Durham, we have produced a prototype FastAPI app for sending numpy arrays derived from HDF5 files, as well as other SWIFT objects.
Here is the API