Multi-archive serve - Githubissues

stschiff commented 1 year ago

OK, so I've implemented a new feature that now makes the server host multiple independent archives. The command line works like this:

trident serve -d myArchive1=/path/to/archive1_basedir1 \
              -d myArchive2=/path/to/archive2_basedir1
              -d myArchive1=/path/to/archive1_basedir2
              -z /path/to/zipBaseDirectory

So internally, all base directories are associated with an archive name, and multiple base directories can be associated to multiple archive-names. They are then of course internally combined.

The APIs work like before, but now use a new parameter in the query-string named archive. So for example http://server/zip_file/myPac?archive=myArchive2 will tell the server that it should look for the package under myArchive2.

If no archive parameter is given, the server will automatically select the first of the archives given on the command line.

If an archive is selected that does not exist, the server will raise an HTTP 500 error, with a meaningful error message. Our client current does not process this, I think, so this would be something to take care of.

Here are a few test command lines that may be useful for testing:

stack run trident -- serve -d myArch=test/testDat/testPackages/ancient/Lamnidis_2018 \
              -d myArch2=test/testDat/testPackages/ancient/Schiffels_2016 \
              -d myArch=test/testDat/testPackages/ancient/Wang_2020 -z /tmp/test \
              --logMode VerboseLog

This would then serve two archives, one with two packages, one with one.

Here are some example queries:

curl 'http://localhost:3000/packages' -> This will yield the packages of archive myArch (the default)
curl 'http://localhost:3000/packages?archive=asda' -> This will yield an error listing the possible archives
curl 'http://localhost:3000/packages?archive=myArch2' -> This will yield packages from archive myArch2
curl 'http://localhost:3000/zip_file/Wang_2020?archive=myArch2' -> This will complain that there is no package with that name in that archive
curl 'http://localhost:3000/zip_file/Wang_2020?archive=myArch' --output Wang_2020.zip -> This will download the package.

I think this fulfils the minimal requirements server-side. The next step is to implement this feature client-wise, which should be fairly straight forward.

codecov[bot] commented 1 year ago

Codecov Report

Patch coverage: 78.94% and project coverage change: +0.25% :tada:

Comparison is base (2025835) 69.55% compared to head (818e209) 69.81%.

Additional details and impacted files

```diff @@ Coverage Diff @@ ## master #258 +/- ## ========================================== + Coverage 69.55% 69.81% +0.25% ========================================== Files 23 23 Lines 3078 3111 +33 Branches 323 324 +1 ========================================== + Hits 2141 2172 +31 - Misses 614 615 +1 - Partials 323 324 +1 ``` | [Files Changed](https://app.codecov.io/gh/poseidon-framework/poseidon-hs/pull/258?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=poseidon-framework) | Coverage Δ | | |---|---|---| | [src/Poseidon/CLI/Fetch.hs](https://app.codecov.io/gh/poseidon-framework/poseidon-hs/pull/258?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=poseidon-framework#diff-c3JjL1Bvc2VpZG9uL0NMSS9GZXRjaC5ocw==) | `49.24% <76.92%> (+4.54%)` | :arrow_up: | | [src/Poseidon/CLI/Serve.hs](https://app.codecov.io/gh/poseidon-framework/poseidon-hs/pull/258?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=poseidon-framework#diff-c3JjL1Bvc2VpZG9uL0NMSS9TZXJ2ZS5ocw==) | `56.56% <78.72%> (+3.38%)` | :arrow_up: | | [src/Poseidon/SecondaryTypes.hs](https://app.codecov.io/gh/poseidon-framework/poseidon-hs/pull/258?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=poseidon-framework#diff-c3JjL1Bvc2VpZG9uL1NlY29uZGFyeVR5cGVzLmhz) | `69.35% <80.00%> (+0.25%)` | :arrow_up: | | [src/Poseidon/CLI/List.hs](https://app.codecov.io/gh/poseidon-framework/poseidon-hs/pull/258?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=poseidon-framework#diff-c3JjL1Bvc2VpZG9uL0NMSS9MaXN0Lmhz) | `73.84% <83.33%> (ø)` | | ... and [1 file with indirect coverage changes](https://app.codecov.io/gh/poseidon-framework/poseidon-hs/pull/258/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=poseidon-framework)

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

nevrome commented 1 year ago

I added an --archive option to list and fetch -- as discussed. I also included the archive functionality to the unit tests. I'll merge this now.

poseidon-framework / poseidon-hs

Multi-archive serve #258

Codecov Report