slaclab / pysmurf

Other
2 stars 9 forks source link

Data_path_id that allows us add a uique id to pysmurf output and plot paths. #664

Closed jlashner closed 3 years ago

jlashner commented 3 years ago

Describe the problem

We've encountered a problem where two instances of pysmurf (controlling different slots) that are instantiated at the same time will have colliding output and plot directories. For instance, if we start two pysmurf instances for slots 2 and 3, at the same time, they could possibly both be writing data to the directory: /data/smurf_data/20210714/1626302915/outputs. This means that if both pysmurf instances try to write an IV-file within the same second, they will have the same path name and overwrite each other (this has actually happened to princeton).

To avoid this, we would like to be able to add a unique identifier to the data path, that we can set for each slot when instantiating. That way instead of colliding, we would have two separate output directories:

slot 2: /data/smurf_data/20210714/crate1slot2/1626302915/outputs
slot 3: /data/smurf_data/20210714/crate1slot3/1626302915/outputs

The issue experienced at pton is documented here: https://github.com/simonsobs/sodetlib/issues/136