search

AllenNeuralDynamics / aind-data-transfer-service

FastAPI service to run data compression and transfer jobs on the hpc
MIT License
1 stars 0 forks source link

Metadata_dir_force should specify subject vs procedures vs both #178

Closed saskiad closed 6 days ago

saskiad commented 3 weeks ago

Since the metadata service doesn't yet pull most of our procedures, it would be best to have an option where the metadata service creates the subject file but doesn't overwrite a procedures file provided by the scientist.

jtyoung84 commented 2 weeks ago

Update docs and double check. I think this is the current behavior.

jtyoung84 commented 1 week ago

@saskiad I double-checked this.

The current behavior is if a user puts a procedures.json file in the metadata_dir location and sets the metadata_dir_force flag to True, then that file will be considered the source of truth. The subject.json file will still be constructed automatically if it isn't found inside the metadata_dir location. Likewise, a user can create a subject.json and the procedures.json will be constructed automatically. A user can also put both the subject.json and procedures.json file in the metadata_dir location. The default for metadata_dir_force is set to False. So if a user doesn't change that, then the metadata files generated automatically will be considered the source of truth. I think when we added that, there were too many instances of incorrect files being copied and pasted.

Let me know if that's sufficient to close this ticket.