search

AllenNeuralDynamics / aind-watchdog-service

Data staging service that prepares acquistion data for cloud upload to Amazon S3 and Code Ocean processing.
https://allenneuraldynamics.github.io/aind-watchdog-service/
MIT License
2 stars 1 forks source link

Allow using aind-data-schema-models classes natively #59

Closed bruno-f-cruz closed 2 months ago

bruno-f-cruz commented 2 months ago

Closes #58

Unfortunately, because these classes are implemented upstream it's hard to provide proper type hints and still ensure proper schema compilation. This is the best I could think of for now.

bruno-f-cruz commented 2 months ago

Not sure what is going on with the CD/CI but tests are passing on my local machine with a clean environment. Can someone else test it?

jtyoung84 commented 2 months ago

It looks like the acq datetime is off a little:

ManifestConfig(name='test', processor_full_name='na', subject_id=7, acquisition_datetime=datetime.datetime(2024, 9, 3, 9, 18, 29, 186489), schedule_time=None, force_cloud_sync=False, transfer_endpoint='http://aind-data-transfer-service/api/v1/submit_jobs', platform='behavior', capsule_id=None, mount=None, s3_bucket=<BucketType.PRIVATE: 'private'>, project_name='no project', destination='path', modalities={'behavior': ['path']}, schemas=['path'], script={}) != ManifestConfig(name='test', processor_full_name='na', subject_id=7, acquisition_datetime=datetime.datetime(2024, 9, 3, 9, 18, 29, 186436), schedule_time=None, force_cloud_sync=False, transfer_endpoint='http://aind-data-transfer-service/api/v1/submit_jobs', platform='behavior', capsule_id=None, mount=None, s3_bucket=<BucketType.PRIVATE: 'private'>, project_name='no project', destination='path', modalities={'behavior': ['path']}, schemas=['path'], script={})

Expected :ManifestConfig(name='test', processor_full_name='na', subject_id=7, acquisition_datetime=datetime.datetime(2024, 9, 3, 9, 18, 29, 186436), schedule_time=None, force_cloud_sync=False, transfer_endpoint='http://aind-data-transfer-service/api/v1/submit_ ...

Actual   :ManifestConfig(name='test', processor_full_name='na', subject_id=7, acquisition_datetime=datetime.datetime(2024, 9, 3, 9, 18, 29, 186489), schedule_time=None, force_cloud_sync=False, transfer_endpoint='http://aind-data-transfer-service/api/v1/submit_ ...
bruno-f-cruz commented 2 months ago

Yep that's what I said on the PR. I just can't see a wait to fix it without fixing it upstream like I have been asking for. The tests I added with this PR showcases the use case. You can basically enter it as an AIND-data-schema-model object.

arielleleon commented 2 months ago

Got it, I misunderstood that.