New repairkit sweeper for updating metadata loaded with older versions of Harvest to use lists instead of single-value strings

al-niessner commented 11 months ago

🗒️ Summary

Script that looks for current problems from historical harvest runs.

⚙️ Test Data and/or Report

None

♻️ Related Issues

Closes NASA-PDS/registry-api#349

sonatype-lift[bot] commented 11 months ago

Sonatype Lift is retiring

Sonatype Lift will be retiring on Sep 12, 2023, with its analysis stopping on Aug 12, 2023. We understand that this news may come as a disappointment, and Sonatype is committed to helping you transition off it seamlessly. If you’d like to retain your data, please export your issues from the web console. We are extremely grateful and thank you for your support over the years.

📖 Read about the impacts and timeline

alexdunnjpl commented 11 months ago

@al-niessner I'll do a code-review this afternoon, but in the meantime could you please lint and push that commit?

EDIT Functional tests too, please - no way it works as-is. The testing mocks aren't necessarily intuitive, so ping/email me if you want it broken down

al-niessner commented 11 months ago

@alexdunnjpl

I do not know what CR is and I am letting sonatype give me pylint complaints.

al-niessner commented 11 months ago

Mandatory: Branch tests must pass

Strong request: Implement functional testing for new functionality

Sure, what functional testing?

al-niessner commented 11 months ago

@alexdunnjpl @nutjob4life

~~Sorry, but I do not understand the last error. Annotate a dictionary??~~

Never mind. I condensed the code a little and mypy seems happy.

al-niessner commented 11 months ago

Wow. I spent more time doing whitespace than developing working code, at least with the test db. Congratulations on making more tedious hurdles than I do. There it is in all of glorious ugliness. I would recommend commenting out the write updates bit until you have run it once against a larger DB just to see what the repairs look like. My db does not have any problems, so hard to check that. I did add a functional test for allarrays.repair() and it does create a dictionary that looks like it will work given all documentation, provenance etc. Of course, never really know until opensearch does something with it.

Oh yeah, sonatype is having problems but python, tox, etc are all happy with the import so do not know what you want to do with sonatype except maybe ignore it.

alexdunnjpl commented 11 months ago

@al-niessner re linting, that should never/rarely be a manual process. Run the linter with tox the same way the branch cicd does and it will autofix everything it can (i.e. whitespace and styling), and provide descriptive errors for everything it can't (i.e. bad parens, type annotations, etc). Sonatype-lift pass is not a requirement, it isn't 1:1 with the rules that get fed into tox, iirc, and the tool is disappearing soon.

Re functional tests, the expectation is that an end-to-end test exists (for ends "mocked json doc input" and "update output json sent to OpenSearch", respectively) that covers at least the happy path, and preferably any obvious-ish edge cases.

In this case, that would be

given mocked docs having legacy-style properties, run the sweeper and ensure correct updates are produced (happy path)
given mocked docs having non-legacy-style properties (including a/some single-element properties), run the sweeper and ensure no updates are produced (edge-ish case)

At that point we can feel safe that if everything outside the sweeper is skookum, so shall the sweeper behaviour be.

alexdunnjpl commented 11 months ago

@al-niessner reviewing the codebase and test-supporting code, the primary purpose of the Db mock and functional tests was to confirm correctness for ancestry, because of its relatively complex behavioural specification.

Since that is not at all the case here, your unit tests are close-enough to good-enough.

There are some necessary changes to make before merge - currently repairkit is making a db call for every document update rather than leveraging the _bulk API, but that's related to some other efficiency work I have planned. Do you want me to push to my own branch and PR into here, or are you happy for me to add it right on to this PR?

al-niessner commented 11 months ago

@alexdunnjpl

It is git and github. Make the changes you need/want right here.

I like bulk too, but with scripts like these I would go more inefficient rather than an miles of runtime lost due to tripping inches before the finish line. If you do each document and something breaks before the finish line, then you rerun the script it has to do less work on the next iteration. Experience tells me that live data, like what in our dbs, will surprise you constantly; meaning you will crash for the unknown unknowns more often than not.

Also, running it now will alleviate the broken documents pressure. You can then add efficiency while it is running making the whole process even more efficient.

alexdunnjpl commented 11 months ago

Experience tells me that live data, like what in our dbs, will surprise you constantly; meaning you will crash for the unknown unknowns more often than not.

This is amenable to broad try-catch-log-continue, but I agree with the rest of the reasoning..

Approving now.

alexdunnjpl commented 11 months ago

@al-niessner just realised I forgot

Is the intent to run this in the near-term in ECS @jordanpadams @tloubrieu-jpl ? If so, the new sweeper will need to be added to ./docker/sweepers_driver.py
@al-niessner Could you please test it manually against a local opensearch (just write a document with some non-array-wrapped fields) and confirm that it corrects them as expected, and doesn't touch anything else.

jordanpadams commented 10 months ago

@alexdunnjpl

Is the intent to run this in the near-term in ECS @jordanpadams @tloubrieu-jpl ? If so, the new sweeper will need to be added to ./docker/sweepers_driver.py

Yes. At least in the interim until we can verify everyone has upgraded to the new tools. It may only need to run once...

@al-niessner Could you please test it manually against a local opensearch (just write a document with some non-array-wrapped fields) and confirm that it corrects them as expected, and doesn't touch anything else.

+1 @al-niessner can you run it and show the output? I would also like to see the log outputs.

al-niessner commented 10 months ago

@alexdunnjpl @jordanpadams

Requested run against test DB:

$ PYTHONPATH=/home/niessner/Projects/PDS/registry-sweepers/src python3 -m pds.registrysweepers.repairkit -b https://localhost:9200 -p admin -u admin --insecure -L DEBUG
2023-08-09 10:02:07,344::pds.registrysweepers.repairkit::INFO::starting CLI processing
2023-08-09 10:02:07,344::pds.registrysweepers.utils::INFO::Initiating query: {'query': {'match_all': {}}, '_source': {}, 'size': 10000}
2023-08-09 10:02:07,344::pds.registrysweepers.utils::INFO::Query progress: 0%
2023-08-09 10:02:07,345::urllib3.connectionpool::DEBUG::Starting new HTTPS connection (1): localhost:9200
2023-08-09 10:02:07,461::urllib3.connectionpool::DEBUG::https://localhost:9200 "GET /registry/_search?scroll=10m HTTP/1.1" 200 179249
2023-08-09 10:02:07,464::pds.registrysweepers.utils::DEBUG::   paging query (0 to 21 of 21)
2023-08-09 10:02:07,464::pds.registrysweepers.repairkit::DEBUG::working on document: urn:nasa:pds:mars2020.spice::1.0
2023-08-09 10:02:07,464::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:creation_date_time
2023-08-09 10:02:07,464::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:file_name
2023-08-09 10:02:07,464::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:file_ref
2023-08-09 10:02:07,464::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:file_size
2023-08-09 10:02:07,464::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:md5_checksum
2023-08-09 10:02:07,465::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:mime_type
2023-08-09 10:02:07,465::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:blob
2023-08-09 10:02:07,465::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:creation_date_time
2023-08-09 10:02:07,465::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:file_name
2023-08-09 10:02:07,465::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:file_ref
2023-08-09 10:02:07,465::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:file_size
2023-08-09 10:02:07,465::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:json_blob
2023-08-09 10:02:07,465::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:md5_checksum
2023-08-09 10:02:07,465::pds.registrysweepers.utils::INFO::Query progress: 9%
2023-08-09 10:02:07,465::pds.registrysweepers.repairkit::DEBUG::working on document: urn:nasa:pds:mars2020.spice:spice_kernels::1.0
2023-08-09 10:02:07,465::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:creation_date_time
2023-08-09 10:02:07,465::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:file_name
2023-08-09 10:02:07,465::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:file_ref
2023-08-09 10:02:07,465::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:file_size
2023-08-09 10:02:07,465::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:md5_checksum
2023-08-09 10:02:07,465::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:mime_type
2023-08-09 10:02:07,465::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:blob
2023-08-09 10:02:07,465::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:creation_date_time
2023-08-09 10:02:07,465::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:file_name
2023-08-09 10:02:07,465::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:file_ref
2023-08-09 10:02:07,466::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:file_size
2023-08-09 10:02:07,466::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:json_blob
2023-08-09 10:02:07,466::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:md5_checksum
2023-08-09 10:02:07,466::pds.registrysweepers.utils::INFO::Query progress: 14%
2023-08-09 10:02:07,466::pds.registrysweepers.repairkit::DEBUG::working on document: urn:nasa:pds:mars2020.spice:spice_kernels:sclk_m2020_168_sclkscet_refit_v01.tsc::1.0
2023-08-09 10:02:07,466::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:creation_date_time
2023-08-09 10:02:07,466::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:file_name
2023-08-09 10:02:07,466::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:file_ref
2023-08-09 10:02:07,466::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:file_size
2023-08-09 10:02:07,466::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:md5_checksum
2023-08-09 10:02:07,466::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:mime_type
2023-08-09 10:02:07,466::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:blob
2023-08-09 10:02:07,466::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:creation_date_time
2023-08-09 10:02:07,466::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:file_name
2023-08-09 10:02:07,466::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:file_ref
2023-08-09 10:02:07,466::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:file_size
2023-08-09 10:02:07,466::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:json_blob
2023-08-09 10:02:07,466::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:md5_checksum
2023-08-09 10:02:07,466::pds.registrysweepers.utils::INFO::Query progress: 19%
2023-08-09 10:02:07,467::pds.registrysweepers.repairkit::DEBUG::working on document: urn:nasa:pds:mars2020.spice:spice_kernels:ck_m2020_surf_ra_tlmres_0000_0089_v1.bc::1.0
2023-08-09 10:02:07,467::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:creation_date_time
2023-08-09 10:02:07,467::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:file_name
2023-08-09 10:02:07,467::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:file_ref
2023-08-09 10:02:07,467::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:file_size
2023-08-09 10:02:07,467::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:md5_checksum
2023-08-09 10:02:07,467::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:mime_type
2023-08-09 10:02:07,467::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:blob
2023-08-09 10:02:07,467::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:creation_date_time
2023-08-09 10:02:07,467::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:file_name
2023-08-09 10:02:07,467::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:file_ref
2023-08-09 10:02:07,467::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:file_size
2023-08-09 10:02:07,467::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:json_blob
2023-08-09 10:02:07,467::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:md5_checksum
2023-08-09 10:02:07,467::pds.registrysweepers.repairkit::DEBUG::working on document: urn:nasa:pds:mars2020.spice:spice_kernels:ck_m2020_surf_rover_tlm_0000_0089_v1.bc::1.0
2023-08-09 10:02:07,467::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:creation_date_time
2023-08-09 10:02:07,467::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:file_name
2023-08-09 10:02:07,467::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:file_ref
2023-08-09 10:02:07,467::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:file_size
2023-08-09 10:02:07,467::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:md5_checksum
2023-08-09 10:02:07,467::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:mime_type
2023-08-09 10:02:07,468::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:blob
2023-08-09 10:02:07,468::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:creation_date_time
2023-08-09 10:02:07,468::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:file_name
2023-08-09 10:02:07,468::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:file_ref
2023-08-09 10:02:07,468::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:file_size
2023-08-09 10:02:07,468::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:json_blob
2023-08-09 10:02:07,468::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:md5_checksum
2023-08-09 10:02:07,468::pds.registrysweepers.utils::INFO::Query progress: 28%
2023-08-09 10:02:07,468::pds.registrysweepers.repairkit::DEBUG::working on document: urn:nasa:pds:mars2020.spice:spice_kernels:mk_m2020::1.0
2023-08-09 10:02:07,468::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:creation_date_time
2023-08-09 10:02:07,468::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:file_name
2023-08-09 10:02:07,468::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:file_ref
2023-08-09 10:02:07,468::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:file_size
2023-08-09 10:02:07,468::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:md5_checksum
2023-08-09 10:02:07,468::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:mime_type
2023-08-09 10:02:07,468::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:blob
2023-08-09 10:02:07,468::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:creation_date_time
2023-08-09 10:02:07,468::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:file_name
2023-08-09 10:02:07,468::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:file_ref
2023-08-09 10:02:07,468::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:file_size
2023-08-09 10:02:07,469::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:json_blob
2023-08-09 10:02:07,469::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:md5_checksum
2023-08-09 10:02:07,469::pds.registrysweepers.utils::INFO::Query progress: 33%
2023-08-09 10:02:07,469::pds.registrysweepers.repairkit::DEBUG::working on document: urn:nasa:pds:mars2020.spice::2.0
2023-08-09 10:02:07,469::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:creation_date_time
2023-08-09 10:02:07,469::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:file_name
2023-08-09 10:02:07,469::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:file_ref
2023-08-09 10:02:07,469::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:file_size
2023-08-09 10:02:07,469::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:md5_checksum
2023-08-09 10:02:07,469::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:mime_type
2023-08-09 10:02:07,469::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:blob
2023-08-09 10:02:07,469::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:creation_date_time
2023-08-09 10:02:07,469::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:file_name
2023-08-09 10:02:07,469::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:file_ref
2023-08-09 10:02:07,469::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:file_size
2023-08-09 10:02:07,469::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:json_blob
2023-08-09 10:02:07,469::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:md5_checksum
2023-08-09 10:02:07,469::pds.registrysweepers.utils::INFO::Query progress: 38%
2023-08-09 10:02:07,469::pds.registrysweepers.repairkit::DEBUG::working on document: urn:nasa:pds:mars2020.spice:spice_kernels::2.0
2023-08-09 10:02:07,469::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:creation_date_time
2023-08-09 10:02:07,470::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:file_name
2023-08-09 10:02:07,470::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:file_ref
2023-08-09 10:02:07,470::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:file_size
2023-08-09 10:02:07,470::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:md5_checksum
2023-08-09 10:02:07,470::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:mime_type
2023-08-09 10:02:07,470::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:blob
2023-08-09 10:02:07,470::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:creation_date_time
2023-08-09 10:02:07,470::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:file_name
2023-08-09 10:02:07,470::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:file_ref
2023-08-09 10:02:07,470::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:file_size
2023-08-09 10:02:07,470::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:json_blob
2023-08-09 10:02:07,470::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:md5_checksum
2023-08-09 10:02:07,470::pds.registrysweepers.repairkit::DEBUG::working on document: urn:nasa:pds:mars2020.spice:spice_kernels:sclk_m2020_168_sclkscet_00007.tsc::1.0
2023-08-09 10:02:07,470::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:creation_date_time
2023-08-09 10:02:07,470::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:file_name
2023-08-09 10:02:07,470::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:file_ref
2023-08-09 10:02:07,470::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:file_size
2023-08-09 10:02:07,470::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:md5_checksum
2023-08-09 10:02:07,470::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:mime_type
2023-08-09 10:02:07,470::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:blob
2023-08-09 10:02:07,471::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:creation_date_time
2023-08-09 10:02:07,471::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:file_name
2023-08-09 10:02:07,471::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:file_ref
2023-08-09 10:02:07,471::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:file_size
2023-08-09 10:02:07,471::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:json_blob
2023-08-09 10:02:07,471::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:md5_checksum
2023-08-09 10:02:07,471::pds.registrysweepers.utils::INFO::Query progress: 47%
2023-08-09 10:02:07,471::pds.registrysweepers.repairkit::DEBUG::working on document: urn:nasa:pds:mars2020.spice:spice_kernels:sclk_m2020_168_sclkscet_refit_v02.tsc::1.0
2023-08-09 10:02:07,471::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:creation_date_time
2023-08-09 10:02:07,471::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:file_name
2023-08-09 10:02:07,471::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:file_ref
2023-08-09 10:02:07,471::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:file_size
2023-08-09 10:02:07,471::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:md5_checksum
2023-08-09 10:02:07,471::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:mime_type
2023-08-09 10:02:07,471::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:blob
2023-08-09 10:02:07,471::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:creation_date_time
2023-08-09 10:02:07,471::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:file_name
2023-08-09 10:02:07,471::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:file_ref
2023-08-09 10:02:07,471::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:file_size
2023-08-09 10:02:07,471::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:json_blob
2023-08-09 10:02:07,471::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:md5_checksum
2023-08-09 10:02:07,472::pds.registrysweepers.utils::INFO::Query progress: 52%
2023-08-09 10:02:07,472::pds.registrysweepers.repairkit::DEBUG::working on document: urn:nasa:pds:mars2020.spice:spice_kernels:ck_m2020_surf_ra_tlmres_0089_0179_v1.bc::1.0
2023-08-09 10:02:07,472::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:creation_date_time
2023-08-09 10:02:07,472::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:file_name
2023-08-09 10:02:07,472::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:file_ref
2023-08-09 10:02:07,472::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:file_size
2023-08-09 10:02:07,472::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:md5_checksum
2023-08-09 10:02:07,472::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:mime_type
2023-08-09 10:02:07,472::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:blob
2023-08-09 10:02:07,472::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:creation_date_time
2023-08-09 10:02:07,472::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:file_name
2023-08-09 10:02:07,472::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:file_ref
2023-08-09 10:02:07,472::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:file_size
2023-08-09 10:02:07,472::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:json_blob
2023-08-09 10:02:07,472::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:md5_checksum
2023-08-09 10:02:07,472::pds.registrysweepers.utils::INFO::Query progress: 57%
2023-08-09 10:02:07,472::pds.registrysweepers.repairkit::DEBUG::working on document: urn:nasa:pds:mars2020.spice:spice_kernels:ck_m2020_surf_rover_tlm_0089_0179_v1.bc::1.0
2023-08-09 10:02:07,472::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:creation_date_time
2023-08-09 10:02:07,472::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:file_name
2023-08-09 10:02:07,472::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:file_ref
2023-08-09 10:02:07,472::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:file_size
2023-08-09 10:02:07,473::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:md5_checksum
2023-08-09 10:02:07,473::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:mime_type
2023-08-09 10:02:07,473::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:blob
2023-08-09 10:02:07,473::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:creation_date_time
2023-08-09 10:02:07,473::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:file_name
2023-08-09 10:02:07,473::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:file_ref
2023-08-09 10:02:07,473::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:file_size
2023-08-09 10:02:07,473::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:json_blob
2023-08-09 10:02:07,473::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:md5_checksum
2023-08-09 10:02:07,473::pds.registrysweepers.repairkit::DEBUG::working on document: urn:nasa:pds:mars2020.spice:spice_kernels:mk_m2020::2.0
2023-08-09 10:02:07,473::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:creation_date_time
2023-08-09 10:02:07,473::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:file_name
2023-08-09 10:02:07,473::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:file_ref
2023-08-09 10:02:07,473::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:file_size
2023-08-09 10:02:07,473::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:md5_checksum
2023-08-09 10:02:07,473::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:mime_type
2023-08-09 10:02:07,473::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:blob
2023-08-09 10:02:07,473::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:creation_date_time
2023-08-09 10:02:07,473::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:file_name
2023-08-09 10:02:07,473::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:file_ref
2023-08-09 10:02:07,473::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:file_size
2023-08-09 10:02:07,474::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:json_blob
2023-08-09 10:02:07,474::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:md5_checksum
2023-08-09 10:02:07,474::pds.registrysweepers.utils::INFO::Query progress: 66%
2023-08-09 10:02:07,474::pds.registrysweepers.repairkit::DEBUG::working on document: urn:nasa:pds:mars2020.spice::3.0
2023-08-09 10:02:07,474::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:creation_date_time
2023-08-09 10:02:07,474::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:file_name
2023-08-09 10:02:07,474::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:file_ref
2023-08-09 10:02:07,474::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:file_size
2023-08-09 10:02:07,474::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:md5_checksum
2023-08-09 10:02:07,474::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:mime_type
2023-08-09 10:02:07,474::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:blob
2023-08-09 10:02:07,474::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:creation_date_time
2023-08-09 10:02:07,474::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:file_name
2023-08-09 10:02:07,474::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:file_ref
2023-08-09 10:02:07,474::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:file_size
2023-08-09 10:02:07,474::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:json_blob
2023-08-09 10:02:07,474::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:md5_checksum
2023-08-09 10:02:07,474::pds.registrysweepers.utils::INFO::Query progress: 71%
2023-08-09 10:02:07,474::pds.registrysweepers.repairkit::DEBUG::working on document: urn:nasa:pds:mars2020.spice:spice_kernels::3.0
2023-08-09 10:02:07,474::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:creation_date_time
2023-08-09 10:02:07,474::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:file_name
2023-08-09 10:02:07,475::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:file_ref
2023-08-09 10:02:07,475::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:file_size
2023-08-09 10:02:07,475::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:md5_checksum
2023-08-09 10:02:07,475::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:mime_type
2023-08-09 10:02:07,475::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:blob
2023-08-09 10:02:07,475::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:creation_date_time
2023-08-09 10:02:07,475::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:file_name
2023-08-09 10:02:07,475::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:file_ref
2023-08-09 10:02:07,475::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:file_size
2023-08-09 10:02:07,475::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:json_blob
2023-08-09 10:02:07,475::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:md5_checksum
2023-08-09 10:02:07,475::pds.registrysweepers.utils::INFO::Query progress: 76%
2023-08-09 10:02:07,475::pds.registrysweepers.repairkit::DEBUG::working on document: urn:nasa:pds:mars2020.spice:spice_kernels:sclk_m2020_168_sclkscet_refit_v03.tsc::1.0
2023-08-09 10:02:07,475::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:creation_date_time
2023-08-09 10:02:07,475::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:file_name
2023-08-09 10:02:07,475::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:file_ref
2023-08-09 10:02:07,475::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:file_size
2023-08-09 10:02:07,475::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:md5_checksum
2023-08-09 10:02:07,475::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:mime_type
2023-08-09 10:02:07,475::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:blob
2023-08-09 10:02:07,475::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:creation_date_time
2023-08-09 10:02:07,475::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:file_name
2023-08-09 10:02:07,476::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:file_ref
2023-08-09 10:02:07,476::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:file_size
2023-08-09 10:02:07,476::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:json_blob
2023-08-09 10:02:07,476::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:md5_checksum
2023-08-09 10:02:07,476::pds.registrysweepers.repairkit::DEBUG::working on document: urn:nasa:pds:mars2020.spice:spice_kernels:ck_m2020_surf_ra_tlmres_0179_0299_v1.bc::1.0
2023-08-09 10:02:07,476::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:creation_date_time
2023-08-09 10:02:07,476::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:file_name
2023-08-09 10:02:07,476::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:file_ref
2023-08-09 10:02:07,476::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:file_size
2023-08-09 10:02:07,476::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:md5_checksum
2023-08-09 10:02:07,476::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:mime_type
2023-08-09 10:02:07,476::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:blob
2023-08-09 10:02:07,476::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:creation_date_time
2023-08-09 10:02:07,476::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:file_name
2023-08-09 10:02:07,476::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:file_ref
2023-08-09 10:02:07,476::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:file_size
2023-08-09 10:02:07,476::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:json_blob
2023-08-09 10:02:07,476::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:md5_checksum
2023-08-09 10:02:07,477::pds.registrysweepers.utils::INFO::Query progress: 85%
2023-08-09 10:02:07,477::pds.registrysweepers.repairkit::DEBUG::working on document: urn:nasa:pds:mars2020.spice:spice_kernels:ck_m2020_surf_rover_tlm_0179_0299_v1.bc::1.0
2023-08-09 10:02:07,477::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:creation_date_time
2023-08-09 10:02:07,477::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:file_name
2023-08-09 10:02:07,477::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:file_ref
2023-08-09 10:02:07,477::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:file_size
2023-08-09 10:02:07,477::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:md5_checksum
2023-08-09 10:02:07,477::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:mime_type
2023-08-09 10:02:07,477::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:blob
2023-08-09 10:02:07,477::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:creation_date_time
2023-08-09 10:02:07,477::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:file_name
2023-08-09 10:02:07,477::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:file_ref
2023-08-09 10:02:07,477::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:file_size
2023-08-09 10:02:07,477::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:json_blob
2023-08-09 10:02:07,477::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:md5_checksum
2023-08-09 10:02:07,477::pds.registrysweepers.utils::INFO::Query progress: 90%
2023-08-09 10:02:07,477::pds.registrysweepers.repairkit::DEBUG::working on document: urn:nasa:pds:mars2020.spice:spice_kernels:mk_m2020::3.0
2023-08-09 10:02:07,477::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:creation_date_time
2023-08-09 10:02:07,477::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:file_name
2023-08-09 10:02:07,477::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:file_ref
2023-08-09 10:02:07,477::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:file_size
2023-08-09 10:02:07,477::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:md5_checksum
2023-08-09 10:02:07,477::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:mime_type
2023-08-09 10:02:07,477::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:blob
2023-08-09 10:02:07,477::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:creation_date_time
2023-08-09 10:02:07,477::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:file_name
2023-08-09 10:02:07,478::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:file_ref
2023-08-09 10:02:07,478::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:file_size
2023-08-09 10:02:07,478::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:json_blob
2023-08-09 10:02:07,478::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:md5_checksum
2023-08-09 10:02:07,478::pds.registrysweepers.utils::INFO::Query progress: 95%
2023-08-09 10:02:07,478::pds.registrysweepers.repairkit::DEBUG::working on document: urn:nasa:pds:mars2020.spice:document::1.0
2023-08-09 10:02:07,478::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:creation_date_time
2023-08-09 10:02:07,478::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:file_name
2023-08-09 10:02:07,478::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:file_ref
2023-08-09 10:02:07,478::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:file_size
2023-08-09 10:02:07,478::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:md5_checksum
2023-08-09 10:02:07,478::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:mime_type
2023-08-09 10:02:07,478::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:blob
2023-08-09 10:02:07,478::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:creation_date_time
2023-08-09 10:02:07,478::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:file_name
2023-08-09 10:02:07,478::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:file_ref
2023-08-09 10:02:07,478::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:file_size
2023-08-09 10:02:07,478::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:json_blob
2023-08-09 10:02:07,478::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:md5_checksum
2023-08-09 10:02:07,478::pds.registrysweepers.utils::INFO::Query progress: 100%
2023-08-09 10:02:07,478::pds.registrysweepers.repairkit::DEBUG::working on document: urn:nasa:pds:mars2020.spice:document:spiceds::1.0
2023-08-09 10:02:07,478::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:creation_date_time
2023-08-09 10:02:07,478::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:file_name
2023-08-09 10:02:07,478::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:file_ref
2023-08-09 10:02:07,478::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:file_size
2023-08-09 10:02:07,478::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:md5_checksum
2023-08-09 10:02:07,478::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:mime_type
2023-08-09 10:02:07,478::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:blob
2023-08-09 10:02:07,478::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:creation_date_time
2023-08-09 10:02:07,478::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:file_name
2023-08-09 10:02:07,478::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:file_ref
2023-08-09 10:02:07,478::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:file_size
2023-08-09 10:02:07,478::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:json_blob
2023-08-09 10:02:07,478::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:md5_checksum
2023-08-09 10:02:07,479::urllib3.connectionpool::DEBUG::Starting new HTTPS connection (1): localhost:9200
2023-08-09 10:02:07,510::urllib3.connectionpool::DEBUG::https://localhost:9200 "DELETE /_search/scroll/FGluY2x1ZGVfY29udGV4dF91dWlkDXF1ZXJ5QW5kRmV0Y2gBFlBRSVBuc2QxUkFldG5jaGswaDVvYVEAAAAAAAAAARZrQ1lBZmF6ZlNwT0xCeGlwOU5PVzRR HTTP/1.1" 200 32
2023-08-09 10:02:07,511::pds.registrysweepers.utils::INFO::Query complete!

alexdunnjpl commented 10 months ago

@al-niessner did you check the db state after to confirm that the expected changes were made?

EDIT I'd expect to see logs from both the repairkit and the write calls themselves, neither of which are present in those logs.

al-niessner commented 10 months ago

Building off of NASA-PDS/registry-api#368

$ curl 'http://localhost:8080/products/urn:nasa:pds:mars2020.spice:spice_kernels:sclk_m2020_168_sclkscet_refit_v01.tsc::1.0' -H 'accept: application/vnd.nasa.pds.pds4+json'
{"timestamp":1691694618445,"status":500,"error":"Internal Server Error","path":"/products/urn:nasa:pds:mars2020.spice:spice_kernels:sclk_m2020_168_sclkscet_refit_v01.tsc::1.0"}

Then after repairkit:

$ curl 'http://localhost:8080/products/urn:nasa:pds:mars2020.spice:spice_kernels:sclk_m2020_168_sclkscet_refit_v01.tsc::1.0' -H 'accept: application/vnd.nasa.pds.pds4+json'
{
"id": "urn:nasa:pds:mars2020.spice:spice_kernels:sclk_m2020_168_sclkscet_refit_v01.tsc::1.0",
"meta": {"node_name":"PDS_ENG","ops:Label_File_Info":{"ops:file_name":"m2020_168_sclkscet_refit_v01.xml","ops:file_ref":"http://localhost:81/archive/custom-datasets/naif1/spice_kernels/m2020_168_sclkscet_refit_v01.xml","ops:creation_date":"2022-04-27T20:21:38Z","ops:file_size":"3552","ops:md5_checksum":"ccf4d21cbddc39c8249c656bc6a37229"},"ops:Data_Files":[{"ops:file_name":"2022-04-27T20:21:36Z","ops:file_ref":"http://localhost:81/archive/custom-datasets/naif1/spice_kernels/m2020_168_sclkscet_refit_v01.tsc","ops:creation_date":"2022-04-27T20:21:36Z","ops:file_size":"39866","ops:md5_checksum":"4c82d1c7e11f9d2e00417676790fa4dc","ops:mime_type":"text/plain"}],"ops:Tracking_Meta":{"ops:archive_status":"archived"}},
"pds4": {"Product_SPICE_Kernel":{"Identification_Area":{"product_class":"Product_SPICE_Kernel","information_model_version":"1.5.0.0","logical_identifier":"urn:nasa:pds:mars2020.spice:spice_kernels:sclk_m2020_168_sclkscet_refit_v01.tsc","version_id":"1.0","Citation_Information":{"publication_year":"2021","description":"SPICE SCLK file providing improved correlation data for the primary M2020 on-board clock, created by NAIF, JPL.","keyword":"Observation Geometry"},"title":"m2020_168_sclkscet_refit_v01.tsc"},"xmlns":"http://pds.nasa.gov/pds4/pds/v1","File_Area_SPICE_Kernel":{"SPICE_Kernel":{"kernel_type":"SCLK","offset":{"unit":"byte","content":"0"},"parsing_standard_id":"SPICE","description":"SPICE SCLK file providing improved correlation data for the primary M2020 on-board clock, created by NAIF, JPL.","object_length":{"unit":"byte","content":"39866"},"encoding_type":"Character"},"File":{"file_name":"m2020_168_sclkscet_refit_v01.tsc","md5_checksum":"4c82d1c7e11f9d2e00417676790fa4dc","creation_date_time":"2021-08-20T11:52:01","file_size":{"unit":"byte","content":"39866"}}},"Reference_List":{"Internal_Reference":{"lid_reference":"urn:nasa:pds:mars2020.spice:document:spiceds","reference_type":"data_to_document"}},"xsi:schemaLocation":"http://pds.nasa.gov/pds4/pds/v1 http://pds.nasa.gov/pds4/pds/v1/PDS4_PDS_1500.xsd","xmlns:xsi":"http://www.w3.org/2001/XMLSchema-instance","Context_Area":{"Investigation_Area":{"Internal_Reference":{"lid_reference":"urn:nasa:pds:context:investigation:mission.mars2020","reference_type":"data_to_investigation"},"name":"Mars 2020 Perseverance Rover Mission","type":"Mission"},"Observing_System":{"Observing_System_Component":{"Internal_Reference":{"lid_reference":"urn:nasa:pds:context:instrument_host:spacecraft.mars2020","reference_type":"is_instrument_host"},"name":"Mars 2020 Perseverance Rover","type":"Spacecraft"}},"Primary_Result_Summary":{"purpose":"Observation Geometry","processing_level":"Derived"},"Time_Coordinates":{"stop_date_time":"2050-01-01T00:00:00Z","start_date_time":"2020-07-30T12:51:34Z"},"Target_Identification":{"Internal_Reference":{"lid_reference":"urn:nasa:pds:context:target:planet.mars","reference_type":"data_to_target"},"name":"MARS","type":"Planet"}}}}}

Output from repairkit:

$ PYTHONPATH=/home/niessner/Projects/PDS/registry-sweepers/src python3 -m pds.registrysweepers.repairkit -b https://localhost:9200 -p admin -u admin --insecure -L DEBUG
2023-08-10 12:11:31,060::pds.registrysweepers.repairkit::INFO::starting CLI processing
2023-08-10 12:11:31,060::pds.registrysweepers.utils::INFO::Initiating query: {'query': {'match_all': {}}, '_source': {}, 'size': 10000}
2023-08-10 12:11:31,060::pds.registrysweepers.utils::INFO::Query progress: 0%
2023-08-10 12:11:31,061::urllib3.connectionpool::DEBUG::Starting new HTTPS connection (1): localhost:9200
2023-08-10 12:11:31,095::urllib3.connectionpool::DEBUG::https://localhost:9200 "GET /registry/_search?scroll=10m HTTP/1.1" 200 179345
2023-08-10 12:11:31,097::pds.registrysweepers.utils::DEBUG::   paging query (0 to 21 of 21)
2023-08-10 12:11:31,097::pds.registrysweepers.repairkit::DEBUG::working on document: urn:nasa:pds:mars2020.spice::1.0
2023-08-10 12:11:31,097::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:creation_date_time
2023-08-10 12:11:31,097::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:file_name
2023-08-10 12:11:31,097::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:file_ref
2023-08-10 12:11:31,097::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:file_size
2023-08-10 12:11:31,097::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:md5_checksum
2023-08-10 12:11:31,097::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:mime_type
2023-08-10 12:11:31,097::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:blob
2023-08-10 12:11:31,097::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:creation_date_time
2023-08-10 12:11:31,097::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:file_name
2023-08-10 12:11:31,097::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:file_ref
2023-08-10 12:11:31,097::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:file_size
2023-08-10 12:11:31,097::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:json_blob
2023-08-10 12:11:31,097::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:md5_checksum
2023-08-10 12:11:31,097::pds.registrysweepers.utils::INFO::Query progress: 9%
2023-08-10 12:11:31,097::pds.registrysweepers.repairkit::DEBUG::working on document: urn:nasa:pds:mars2020.spice:spice_kernels::1.0
2023-08-10 12:11:31,097::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:creation_date_time
2023-08-10 12:11:31,098::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:file_name
2023-08-10 12:11:31,098::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:file_ref
2023-08-10 12:11:31,098::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:file_size
2023-08-10 12:11:31,098::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:md5_checksum
2023-08-10 12:11:31,098::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:mime_type
2023-08-10 12:11:31,098::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:blob
2023-08-10 12:11:31,098::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:creation_date_time
2023-08-10 12:11:31,098::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:file_name
2023-08-10 12:11:31,098::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:file_ref
2023-08-10 12:11:31,098::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:file_size
2023-08-10 12:11:31,098::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:json_blob
2023-08-10 12:11:31,098::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:md5_checksum
2023-08-10 12:11:31,098::pds.registrysweepers.utils::INFO::Query progress: 14%
2023-08-10 12:11:31,098::pds.registrysweepers.repairkit::DEBUG::working on document: urn:nasa:pds:mars2020.spice:spice_kernels:ck_m2020_surf_ra_tlmres_0000_0089_v1.bc::1.0
2023-08-10 12:11:31,098::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:creation_date_time
2023-08-10 12:11:31,098::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:file_name
2023-08-10 12:11:31,098::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:file_ref
2023-08-10 12:11:31,098::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:file_size
2023-08-10 12:11:31,098::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:md5_checksum
2023-08-10 12:11:31,098::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:mime_type
2023-08-10 12:11:31,098::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:blob
2023-08-10 12:11:31,098::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:creation_date_time
2023-08-10 12:11:31,099::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:file_name
2023-08-10 12:11:31,099::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:file_ref
2023-08-10 12:11:31,099::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:file_size
2023-08-10 12:11:31,099::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:json_blob
2023-08-10 12:11:31,099::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:md5_checksum
2023-08-10 12:11:31,099::pds.registrysweepers.utils::INFO::Query progress: 19%
2023-08-10 12:11:31,099::pds.registrysweepers.repairkit::DEBUG::working on document: urn:nasa:pds:mars2020.spice:spice_kernels:ck_m2020_surf_rover_tlm_0000_0089_v1.bc::1.0
2023-08-10 12:11:31,099::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:creation_date_time
2023-08-10 12:11:31,099::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:file_name
2023-08-10 12:11:31,099::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:file_ref
2023-08-10 12:11:31,099::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:file_size
2023-08-10 12:11:31,099::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:md5_checksum
2023-08-10 12:11:31,099::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:mime_type
2023-08-10 12:11:31,099::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:blob
2023-08-10 12:11:31,099::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:creation_date_time
2023-08-10 12:11:31,099::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:file_name
2023-08-10 12:11:31,099::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:file_ref
2023-08-10 12:11:31,099::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:file_size
2023-08-10 12:11:31,099::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:json_blob
2023-08-10 12:11:31,099::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:md5_checksum
2023-08-10 12:11:31,099::pds.registrysweepers.repairkit::DEBUG::working on document: urn:nasa:pds:mars2020.spice:spice_kernels:mk_m2020::1.0
2023-08-10 12:11:31,099::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:creation_date_time
2023-08-10 12:11:31,100::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:file_name
2023-08-10 12:11:31,100::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:file_ref
2023-08-10 12:11:31,100::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:file_size
2023-08-10 12:11:31,100::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:md5_checksum
2023-08-10 12:11:31,100::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:mime_type
2023-08-10 12:11:31,100::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:blob
2023-08-10 12:11:31,100::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:creation_date_time
2023-08-10 12:11:31,100::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:file_name
2023-08-10 12:11:31,100::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:file_ref
2023-08-10 12:11:31,100::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:file_size
2023-08-10 12:11:31,100::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:json_blob
2023-08-10 12:11:31,100::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:md5_checksum
2023-08-10 12:11:31,100::pds.registrysweepers.utils::INFO::Query progress: 28%
2023-08-10 12:11:31,100::pds.registrysweepers.repairkit::DEBUG::working on document: urn:nasa:pds:mars2020.spice::2.0
2023-08-10 12:11:31,100::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:creation_date_time
2023-08-10 12:11:31,100::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:file_name
2023-08-10 12:11:31,100::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:file_ref
2023-08-10 12:11:31,100::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:file_size
2023-08-10 12:11:31,100::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:md5_checksum
2023-08-10 12:11:31,100::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:mime_type
2023-08-10 12:11:31,100::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:blob
2023-08-10 12:11:31,100::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:creation_date_time
2023-08-10 12:11:31,100::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:file_name
2023-08-10 12:11:31,101::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:file_ref
2023-08-10 12:11:31,101::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:file_size
2023-08-10 12:11:31,101::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:json_blob
2023-08-10 12:11:31,101::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:md5_checksum
2023-08-10 12:11:31,101::pds.registrysweepers.utils::INFO::Query progress: 33%
2023-08-10 12:11:31,101::pds.registrysweepers.repairkit::DEBUG::working on document: urn:nasa:pds:mars2020.spice:spice_kernels::2.0
2023-08-10 12:11:31,101::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:creation_date_time
2023-08-10 12:11:31,101::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:file_name
2023-08-10 12:11:31,101::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:file_ref
2023-08-10 12:11:31,101::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:file_size
2023-08-10 12:11:31,101::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:md5_checksum
2023-08-10 12:11:31,101::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:mime_type
2023-08-10 12:11:31,101::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:blob
2023-08-10 12:11:31,101::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:creation_date_time
2023-08-10 12:11:31,101::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:file_name
2023-08-10 12:11:31,101::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:file_ref
2023-08-10 12:11:31,101::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:file_size
2023-08-10 12:11:31,101::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:json_blob
2023-08-10 12:11:31,101::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:md5_checksum
2023-08-10 12:11:31,101::pds.registrysweepers.utils::INFO::Query progress: 38%
2023-08-10 12:11:31,102::pds.registrysweepers.repairkit::DEBUG::working on document: urn:nasa:pds:mars2020.spice:spice_kernels:sclk_m2020_168_sclkscet_00007.tsc::1.0
2023-08-10 12:11:31,102::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:creation_date_time
2023-08-10 12:11:31,102::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:file_name
2023-08-10 12:11:31,102::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:file_ref
2023-08-10 12:11:31,102::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:file_size
2023-08-10 12:11:31,102::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:md5_checksum
2023-08-10 12:11:31,102::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:mime_type
2023-08-10 12:11:31,102::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:blob
2023-08-10 12:11:31,102::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:creation_date_time
2023-08-10 12:11:31,102::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:file_name
2023-08-10 12:11:31,102::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:file_ref
2023-08-10 12:11:31,102::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:file_size
2023-08-10 12:11:31,102::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:json_blob
2023-08-10 12:11:31,102::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:md5_checksum
2023-08-10 12:11:31,102::pds.registrysweepers.repairkit::DEBUG::working on document: urn:nasa:pds:mars2020.spice:spice_kernels:sclk_m2020_168_sclkscet_refit_v02.tsc::1.0
2023-08-10 12:11:31,102::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:creation_date_time
2023-08-10 12:11:31,102::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:file_name
2023-08-10 12:11:31,102::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:file_ref
2023-08-10 12:11:31,102::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:file_size
2023-08-10 12:11:31,102::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:md5_checksum
2023-08-10 12:11:31,102::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:mime_type
2023-08-10 12:11:31,102::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:blob
2023-08-10 12:11:31,103::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:creation_date_time
2023-08-10 12:11:31,103::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:file_name
2023-08-10 12:11:31,103::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:file_ref
2023-08-10 12:11:31,103::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:file_size
2023-08-10 12:11:31,103::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:json_blob
2023-08-10 12:11:31,103::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:md5_checksum
2023-08-10 12:11:31,103::pds.registrysweepers.utils::INFO::Query progress: 47%
2023-08-10 12:11:31,103::pds.registrysweepers.repairkit::DEBUG::working on document: urn:nasa:pds:mars2020.spice:spice_kernels:ck_m2020_surf_ra_tlmres_0089_0179_v1.bc::1.0
2023-08-10 12:11:31,103::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:creation_date_time
2023-08-10 12:11:31,103::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:file_name
2023-08-10 12:11:31,103::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:file_ref
2023-08-10 12:11:31,103::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:file_size
2023-08-10 12:11:31,103::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:md5_checksum
2023-08-10 12:11:31,103::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:mime_type
2023-08-10 12:11:31,103::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:blob
2023-08-10 12:11:31,103::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:creation_date_time
2023-08-10 12:11:31,103::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:file_name
2023-08-10 12:11:31,103::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:file_ref
2023-08-10 12:11:31,103::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:file_size
2023-08-10 12:11:31,103::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:json_blob
2023-08-10 12:11:31,103::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:md5_checksum
2023-08-10 12:11:31,103::pds.registrysweepers.utils::INFO::Query progress: 52%
2023-08-10 12:11:31,103::pds.registrysweepers.repairkit::DEBUG::working on document: urn:nasa:pds:mars2020.spice:spice_kernels:ck_m2020_surf_rover_tlm_0089_0179_v1.bc::1.0
2023-08-10 12:11:31,104::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:creation_date_time
2023-08-10 12:11:31,104::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:file_name
2023-08-10 12:11:31,104::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:file_ref
2023-08-10 12:11:31,104::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:file_size
2023-08-10 12:11:31,104::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:md5_checksum
2023-08-10 12:11:31,104::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:mime_type
2023-08-10 12:11:31,104::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:blob
2023-08-10 12:11:31,104::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:creation_date_time
2023-08-10 12:11:31,104::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:file_name
2023-08-10 12:11:31,104::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:file_ref
2023-08-10 12:11:31,104::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:file_size
2023-08-10 12:11:31,104::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:json_blob
2023-08-10 12:11:31,104::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:md5_checksum
2023-08-10 12:11:31,104::pds.registrysweepers.utils::INFO::Query progress: 57%
2023-08-10 12:11:31,104::pds.registrysweepers.repairkit::DEBUG::working on document: urn:nasa:pds:mars2020.spice:spice_kernels:mk_m2020::2.0
2023-08-10 12:11:31,104::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:creation_date_time
2023-08-10 12:11:31,104::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:file_name
2023-08-10 12:11:31,104::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:file_ref
2023-08-10 12:11:31,104::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:file_size
2023-08-10 12:11:31,104::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:md5_checksum
2023-08-10 12:11:31,104::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:mime_type
2023-08-10 12:11:31,104::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:blob
2023-08-10 12:11:31,104::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:creation_date_time
2023-08-10 12:11:31,104::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:file_name
2023-08-10 12:11:31,105::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:file_ref
2023-08-10 12:11:31,105::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:file_size
2023-08-10 12:11:31,105::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:json_blob
2023-08-10 12:11:31,105::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:md5_checksum
2023-08-10 12:11:31,105::pds.registrysweepers.repairkit::DEBUG::working on document: urn:nasa:pds:mars2020.spice::3.0
2023-08-10 12:11:31,105::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:creation_date_time
2023-08-10 12:11:31,105::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:file_name
2023-08-10 12:11:31,105::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:file_ref
2023-08-10 12:11:31,105::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:file_size
2023-08-10 12:11:31,105::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:md5_checksum
2023-08-10 12:11:31,105::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:mime_type
2023-08-10 12:11:31,105::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:blob
2023-08-10 12:11:31,105::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:creation_date_time
2023-08-10 12:11:31,105::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:file_name
2023-08-10 12:11:31,105::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:file_ref
2023-08-10 12:11:31,105::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:file_size
2023-08-10 12:11:31,105::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:json_blob
2023-08-10 12:11:31,105::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:md5_checksum
2023-08-10 12:11:31,105::pds.registrysweepers.utils::INFO::Query progress: 66%
2023-08-10 12:11:31,105::pds.registrysweepers.repairkit::DEBUG::working on document: urn:nasa:pds:mars2020.spice:spice_kernels::3.0
2023-08-10 12:11:31,105::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:creation_date_time
2023-08-10 12:11:31,106::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:file_name
2023-08-10 12:11:31,106::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:file_ref
2023-08-10 12:11:31,106::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:file_size
2023-08-10 12:11:31,106::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:md5_checksum
2023-08-10 12:11:31,106::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:mime_type
2023-08-10 12:11:31,106::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:blob
2023-08-10 12:11:31,106::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:creation_date_time
2023-08-10 12:11:31,106::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:file_name
2023-08-10 12:11:31,106::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:file_ref
2023-08-10 12:11:31,106::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:file_size
2023-08-10 12:11:31,106::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:json_blob
2023-08-10 12:11:31,106::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:md5_checksum
2023-08-10 12:11:31,106::pds.registrysweepers.utils::INFO::Query progress: 71%
2023-08-10 12:11:31,106::pds.registrysweepers.repairkit::DEBUG::working on document: urn:nasa:pds:mars2020.spice:spice_kernels:sclk_m2020_168_sclkscet_refit_v03.tsc::1.0
2023-08-10 12:11:31,106::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:creation_date_time
2023-08-10 12:11:31,106::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:file_name
2023-08-10 12:11:31,106::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:file_ref
2023-08-10 12:11:31,106::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:file_size
2023-08-10 12:11:31,106::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:md5_checksum
2023-08-10 12:11:31,106::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:mime_type
2023-08-10 12:11:31,106::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:blob
2023-08-10 12:11:31,106::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:creation_date_time
2023-08-10 12:11:31,106::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:file_name
2023-08-10 12:11:31,106::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:file_ref
2023-08-10 12:11:31,107::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:file_size
2023-08-10 12:11:31,107::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:json_blob
2023-08-10 12:11:31,107::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:md5_checksum
2023-08-10 12:11:31,107::pds.registrysweepers.utils::INFO::Query progress: 76%
2023-08-10 12:11:31,107::pds.registrysweepers.repairkit::DEBUG::working on document: urn:nasa:pds:mars2020.spice:spice_kernels:ck_m2020_surf_ra_tlmres_0179_0299_v1.bc::1.0
2023-08-10 12:11:31,107::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:creation_date_time
2023-08-10 12:11:31,107::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:file_name
2023-08-10 12:11:31,107::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:file_ref
2023-08-10 12:11:31,107::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:file_size
2023-08-10 12:11:31,107::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:md5_checksum
2023-08-10 12:11:31,107::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:mime_type
2023-08-10 12:11:31,107::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:blob
2023-08-10 12:11:31,107::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:creation_date_time
2023-08-10 12:11:31,107::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:file_name
2023-08-10 12:11:31,107::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:file_ref
2023-08-10 12:11:31,107::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:file_size
2023-08-10 12:11:31,107::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:json_blob
2023-08-10 12:11:31,107::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:md5_checksum
2023-08-10 12:11:31,107::pds.registrysweepers.repairkit::DEBUG::working on document: urn:nasa:pds:mars2020.spice:spice_kernels:ck_m2020_surf_rover_tlm_0179_0299_v1.bc::1.0
2023-08-10 12:11:31,107::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:creation_date_time
2023-08-10 12:11:31,107::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:file_name
2023-08-10 12:11:31,108::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:file_ref
2023-08-10 12:11:31,108::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:file_size
2023-08-10 12:11:31,108::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:md5_checksum
2023-08-10 12:11:31,108::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:mime_type
2023-08-10 12:11:31,108::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:blob
2023-08-10 12:11:31,108::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:creation_date_time
2023-08-10 12:11:31,108::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:file_name
2023-08-10 12:11:31,108::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:file_ref
2023-08-10 12:11:31,108::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:file_size
2023-08-10 12:11:31,108::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:json_blob
2023-08-10 12:11:31,108::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:md5_checksum
2023-08-10 12:11:31,108::pds.registrysweepers.utils::INFO::Query progress: 85%
2023-08-10 12:11:31,108::pds.registrysweepers.repairkit::DEBUG::working on document: urn:nasa:pds:mars2020.spice:spice_kernels:mk_m2020::3.0
2023-08-10 12:11:31,108::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:creation_date_time
2023-08-10 12:11:31,108::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:file_name
2023-08-10 12:11:31,108::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:file_ref
2023-08-10 12:11:31,108::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:file_size
2023-08-10 12:11:31,108::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:md5_checksum
2023-08-10 12:11:31,108::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:mime_type
2023-08-10 12:11:31,108::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:blob
2023-08-10 12:11:31,108::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:creation_date_time
2023-08-10 12:11:31,108::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:file_name
2023-08-10 12:11:31,108::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:file_ref
2023-08-10 12:11:31,108::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:file_size
2023-08-10 12:11:31,109::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:json_blob
2023-08-10 12:11:31,109::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:md5_checksum
2023-08-10 12:11:31,109::pds.registrysweepers.utils::INFO::Query progress: 90%
2023-08-10 12:11:31,109::pds.registrysweepers.repairkit::DEBUG::working on document: urn:nasa:pds:mars2020.spice:document::1.0
2023-08-10 12:11:31,109::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:creation_date_time
2023-08-10 12:11:31,109::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:file_name
2023-08-10 12:11:31,109::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:file_ref
2023-08-10 12:11:31,109::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:file_size
2023-08-10 12:11:31,109::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:md5_checksum
2023-08-10 12:11:31,109::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:mime_type
2023-08-10 12:11:31,109::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:blob
2023-08-10 12:11:31,109::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:creation_date_time
2023-08-10 12:11:31,109::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:file_name
2023-08-10 12:11:31,109::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:file_ref
2023-08-10 12:11:31,109::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:file_size
2023-08-10 12:11:31,109::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:json_blob
2023-08-10 12:11:31,109::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:md5_checksum
2023-08-10 12:11:31,109::pds.registrysweepers.utils::INFO::Query progress: 95%
2023-08-10 12:11:31,109::pds.registrysweepers.repairkit::DEBUG::working on document: urn:nasa:pds:mars2020.spice:document:spiceds::1.0
2023-08-10 12:11:31,109::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:creation_date_time
2023-08-10 12:11:31,109::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:file_name
2023-08-10 12:11:31,109::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:file_ref
2023-08-10 12:11:31,110::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:file_size
2023-08-10 12:11:31,110::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:md5_checksum
2023-08-10 12:11:31,110::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:mime_type
2023-08-10 12:11:31,110::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:blob
2023-08-10 12:11:31,110::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:creation_date_time
2023-08-10 12:11:31,110::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:file_name
2023-08-10 12:11:31,110::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:file_ref
2023-08-10 12:11:31,110::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:file_size
2023-08-10 12:11:31,110::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:json_blob
2023-08-10 12:11:31,110::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:md5_checksum
2023-08-10 12:11:31,110::pds.registrysweepers.utils::INFO::Query progress: 100%
2023-08-10 12:11:31,110::pds.registrysweepers.repairkit::DEBUG::working on document: urn:nasa:pds:mars2020.spice:spice_kernels:sclk_m2020_168_sclkscet_refit_v01.tsc::1.0
2023-08-10 12:11:31,110::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:creation_date_time
2023-08-10 12:11:31,110::pds.registrysweepers.repairkit.allarrays::INFO::found string for ops:Data_File_Info/ops:creation_date_time where it should be an array
2023-08-10 12:11:31,110::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:file_name
2023-08-10 12:11:31,110::pds.registrysweepers.repairkit.allarrays::INFO::found string for ops:Data_File_Info/ops:file_name where it should be an array
2023-08-10 12:11:31,110::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:file_ref
2023-08-10 12:11:31,110::pds.registrysweepers.repairkit.allarrays::INFO::found string for ops:Data_File_Info/ops:file_ref where it should be an array
2023-08-10 12:11:31,110::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:file_size
2023-08-10 12:11:31,110::pds.registrysweepers.repairkit.allarrays::INFO::found string for ops:Data_File_Info/ops:file_size where it should be an array
2023-08-10 12:11:31,110::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:md5_checksum
2023-08-10 12:11:31,110::pds.registrysweepers.repairkit.allarrays::INFO::found string for ops:Data_File_Info/ops:md5_checksum where it should be an array
2023-08-10 12:11:31,110::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Data_File_Info/ops:mime_type
2023-08-10 12:11:31,110::pds.registrysweepers.repairkit.allarrays::INFO::found string for ops:Data_File_Info/ops:mime_type where it should be an array
2023-08-10 12:11:31,110::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:blob
2023-08-10 12:11:31,111::pds.registrysweepers.repairkit.allarrays::INFO::found string for ops:Label_File_Info/ops:blob where it should be an array
2023-08-10 12:11:31,111::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:creation_date_time
2023-08-10 12:11:31,111::pds.registrysweepers.repairkit.allarrays::INFO::found string for ops:Label_File_Info/ops:creation_date_time where it should be an array
2023-08-10 12:11:31,111::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:file_name
2023-08-10 12:11:31,111::pds.registrysweepers.repairkit.allarrays::INFO::found string for ops:Label_File_Info/ops:file_name where it should be an array
2023-08-10 12:11:31,111::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:file_ref
2023-08-10 12:11:31,111::pds.registrysweepers.repairkit.allarrays::INFO::found string for ops:Label_File_Info/ops:file_ref where it should be an array
2023-08-10 12:11:31,111::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:file_size
2023-08-10 12:11:31,111::pds.registrysweepers.repairkit.allarrays::INFO::found string for ops:Label_File_Info/ops:file_size where it should be an array
2023-08-10 12:11:31,111::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:json_blob
2023-08-10 12:11:31,111::pds.registrysweepers.repairkit.allarrays::INFO::found string for ops:Label_File_Info/ops:json_blob where it should be an array
2023-08-10 12:11:31,111::pds.registrysweepers.repairkit.allarrays::DEBUG::checking ops:Label_File_Info/ops:md5_checksum
2023-08-10 12:11:31,111::pds.registrysweepers.repairkit.allarrays::INFO::found string for ops:Label_File_Info/ops:md5_checksum where it should be an array
2023-08-10 12:11:31,111::pds.registrysweepers.repairkit::INFO::Writing repairs to document: urn:nasa:pds:mars2020.spice:spice_kernels:sclk_m2020_168_sclkscet_refit_v01.tsc::1.0
2023-08-10 12:11:31,111::pds.registrysweepers.utils::INFO::Updating documents for 1 products...
2023-08-10 12:11:31,111::pds.registrysweepers.utils::INFO::Writing documents updates for 1 remaining products to db...
2023-08-10 12:11:31,112::urllib3.connectionpool::DEBUG::Starting new HTTPS connection (1): localhost:9200
2023-08-10 12:11:31,165::urllib3.connectionpool::DEBUG::https://localhost:9200 "DELETE /_search/scroll/FGluY2x1ZGVfY29udGV4dF91dWlkDXF1ZXJ5QW5kRmV0Y2gBFl9QN2hHRVo3VHZ1eDYyaERHR3FjYlEAAAAAAAAANBZrQ1lBZmF6ZlNwT0xCeGlwOU5PVzRR HTTP/1.1" 200 32
2023-08-10 12:11:31,165::pds.registrysweepers.utils::INFO::Query complete!

alexdunnjpl commented 10 months ago

Quality noice 👍 👍

EDIT @al-niessner don't forget to add it to sweepers_driver.py prior to merge. @jordanpadams do you want @sjoshi-jpl and I to deploy this version to delta->prod once merged?

tloubrieu-jpl commented 10 months ago

Hi @al-niessner ,

I think you still need to add the repair kit to the sweepers_driver there https://github.com/NASA-PDS/registry-sweepers/blob/1a92b530c9e7a29d0b79e7afbfbc559dae4f3d0c/docker/sweepers_driver.py#L111

Thanks,

Thomas

NASA-PDS / registry-sweepers