ebi-ait / dcp-ingest-central

Central point of access for the Ingestion Service of the HCA DCP
Apache License 2.0
0 stars 0 forks source link

Testing the direct archiver for Phase 1 #580

Open ke4 opened 2 years ago

ke4 commented 2 years ago

Choose a few projects from the archives that have been submitted with DSP. Archive the projects using the direct archiver to the test environment of the archives. Compare the projects from each archive. The results should be identical (with the exception of different accession codes, dates, etc.) and no errors should be present.

Acceptance Test Scenarios

Test tracking for Direct Archiving

MightyAx commented 2 years ago

Some issues found with the data set sent by @ESapenaVentura, this ticket is for listing these.

ami-day commented 2 years ago

needs discussion

amnonkhen commented 2 years ago

@prabh-t to setup a meeting to discuss mitigation of recurring BioSamples issue.

ipediez commented 2 years ago

working on asynchronous implementation of the archiving

idazucchi commented 2 years ago

testing with medium and large datasets today!

ESapenaVentura commented 2 years ago

Tested big dataset - 2 separate tests:

gabsie commented 2 years ago

They will continue testing here with @ipediez , but the big dataset has gone through.

idazucchi commented 2 years ago

Irene has tested the medium and small dataset. @ke4 and @prabh-t will investigate an issue with run accessions

ESapenaVentura commented 2 years ago

Did some testing with @gabsie and @Wkt8 . There were some issues with file archiving.

Did another post request but didn't have capacity to investigate.

@Wkt8 did you manage to archive successfully?

@ke4 to investigate later with @prabh-t . Messages take time to process? may be the underlying issue.

Wkt8 commented 2 years ago

This did not archive successfully - going to try again.

Wkt8 commented 2 years ago

Still not archiving successfully. Using the GET command results in a number of readings with

fileArchiveResult":null,

gabsie commented 2 years ago

@ke4 did some tests yesterday. There are currently problems with: Alkaslasi - file upload With metadata archiving as well.

Investigation is under way, together between @MightyAx and @ke4