Enable user-specified path to sidecar JSON files with `fetch`

[sidneymbell]

Just documenting offline discussion in case someone else pops up with a similar need.

Issue

Currently, fetch can only find sidecar files such as the frequencies.json required for the frequencies panel, by looking at the main JSON -> navigating to its parent directory -> looking for the sidecar files alongside with the same build name. This means that sidecar files can't be provided if your file URLs aren't set up like a publicly-accessible directory.

Example

We use presigned URLs from AWS for our main JSON files, but don't have a way to provide the sidecar file (which has its own non-derivative presigned URL).

Solutions discussed

1. Provide the sidecar URL via the : signifier currently used for tanglegrams. Downside: overloads the use of :
2. Provide the sidecar URL via the main JSON file Downsides: have to parse main JSON first before loading, and having static URLs sitting there is undesirable from a security standpoint.

From @tsibley --

3. /fetch could support two additional parameters one way or another (e.g. in the query part of the URL): root-sequence=… and tip-frequencies=…. These would provide alternate URLs for the sidecars, and you'd populate them with additional pre-signed URLs. Would require careful thought and redesigning some significant bits about how nextstrain.org and auspice work.

4. With no changes from Nextstrain, you could put a small redirecting layer between nextstrain.org and your S3 endpoints. This redirecting layer would have the sole job of redirecting sidecar-compatible URLs used for /fetch to the underlying S3 objects with pre-signed URLs. Low-hanging fruit that doesn't require a change from nextstrain

Current plans

We're scoping a potential implementation of option (4) on our end. Will update here with progress / to point interested folks to whatever we end up doing. cc @jgadling

[schneid44]

Hi!

My name is Justin and I'm a computer science student at the University of Michigan. For a final project another student and I are tasked with working on an open source project. I was wondering if this issue is still relevant and ready to be worked on or if you had any other suggestions about what could be worked on. We are both seniors, familiar with Javascript, and expected to do about 15-20 hours of work. Thanks in advance!

[tsibley]

@schneid44 Hi! Thanks for your interest in contributing. I don't think this is the best issue to work on as it's still uncertain whether we want to make any changes to Nextstrain here (nextstrain.org + Auspice) or what those changes would even be. I think a task that's already more well-scoped would be better. I'll respond separately to your other comment on #432.

And since I'm here revisiting this issue, @sidneymbell, any update on pursuing option (4) above on your end?

[sidneymbell]

I don't think we ended up pursuing this; @danrlu might know more.

[danrlu]

That's correct. We haven't had any further development on our end.