radiasoft / sirepo

Sirepo is a framework for scientific cloud computing. Try it out!
https://sirepo.com
Apache License 2.0
64 stars 31 forks source link

raydata: understand chx scan status files #7155

Open e-carlin opened 3 months ago

e-carlin commented 3 months ago

Where do they come from and how are they updated? Where are they written to? We need to better understand these files so we can create a common library that csx will use to create files with the same sorts of details.

e-carlin commented 2 months ago

I met with Lutz to discuss. There are two components:

  1. The "papermill loop". This is the loop that watches for new uid's and then calls papermill to start the analysis process. Part of the json file is created in this loop.
  2. json file creation in the analysis notebooks themselves.

He is going to share the papermill loop part. For us that will live inside of the scan monitor because that is our papermill loop. He is also going to share the latest notebooks so we can have notebooks that create the scans.