fix trailing slash problem in ingest api (managed access)

[amnonkhen]

related to managed access implementation: ebi-ait/dcp-ingest-central#967

tasks

• [x] revert to pre managed access version
  • [x] ingest-core gitlab pipeline
  • [x] ingest-state-tracking gitlab pipeline
  • [x] ingest-validator gitlab pipeline
  • [x] ingest-broker gitlab pipeline
• [x] run integration tests successfully using the reverted code
  • [x] update spreadsheet used in integration tests on staging branch
  • [ ] update spreadsheet used in integration tests on master branch
• [x] re-implement "trailing slash" protection in ingest-core
  • [x] cherry pick the changes related to blocking unauthorized access to urls ending with slash.
  • [x] update SecurityTest unit test to latest code
  • [x] update SecurityConfig
  • [x] unit & integration tests pass
  • [x] test the integration tests are passing
• [x] revert other ingest services:
  • [x] ingest-state-tracking
  • [x] revert validator
  • [x] revert graph validator
  • [x] revert broker
  • [ ] revert integration test
• [ ] run integration tests
  • [ ] update any integration test changes to staging/master branches
• [x] test with ingest-ui that includes DUOS ID
  • [ ] modify ingest-ui - use auth header in the protected api calls. See ebi-ait/dcp-ingest-central#1020

reverting a branch

1. find last good commit in the git log
2. tag it with dc-967-managed-access-base
3. reset the dev branch to it

   git checkout dev
   git reset --hard dc-967-managed-access-base
   git push -f origin dev

4. run build in gitlab
5. check integration tests
6. manual tests if needed

Cleanup

1. re-enable branch protection on:
   • [ ] core
   • [ ] validator
   • [ ] state tracker
   • [ ] broker

[amnonkhen]

On Fri 12/7/2024 @amnonkhen @KociOrges and Dipayan tried the following:

1. rollback dev - integration tests problems of failing integration tests persisted
2. noticed a long startup times for state-tracker - we cleared the db and restarted state tracker and redis. The problem of failing integration tests when running in parallel persisted
3. tried to run ingest_upload_to_terra test in prod but got different unrelated errors

Next steps for Monday:

1. investigate and fix the prod errors
2. try a manual submission on prod
3. go back to investigating the error on dev using log messages