Feature request - download lists

esm-dev / esm.sh

A fast, smart, & global CDN for modern(es2015+) web development.

https://esm.sh

MIT License

3.11k stars 150 forks source link

Feature request - download lists #793

Open guybedford opened 9 months ago

guybedford commented 9 months ago

For JSPM, we recently added a feature to the API (https://jspm.org/cdn/api#download) to support getting file lists of packages across different CDNs as generated for JSPM import maps. This allows taking a JSPM generated import map against the CDN and rewriting that import map against local files.

Currently esm.sh is unsupported because I don't know a good way to get a file listing for our package requests. If there were some ability at the package boundary to get a deterministic listing of the sub URLs of that package that can be requested for files of that package that would be really useful to include esm.sh in this feature.

ije commented 9 months ago

that's so cool! i will add it later

renhiyama commented 9 months ago

+1, this can create much faster downloads, at the cost of downloading additional not needed files like readme (maybe we can Blacklist popular not needed files?)

guybedford commented 9 months ago

Thanks for being open to it! And good point @renhiyama - I wonder if it might make sense to also provide the tracing graph of the package for downloaders to selectively download?

Some thoughts:

Would it help to have ?module-graphs-only as a parameter to only include the module graph?
Alternatively, perhaps the file listing just provides all the graph data in a way that the tooling can then choose how to consume, perhaps a structure that both lists all the files, and also provides the module trace metadata for each file when it's a module?

I might experiment with some graph metadata additions to this this weekend, although would only be supported for JSPM packages and not other providers as we have this trace data prepopulated.

Having some esm.sh feedback / alignment would be a huge help in figuring this out too.

renhiyama commented 9 months ago

Idk wdym by module graphs, tracing etc. so I can't give any views on it. As long as it makes sense for me, I give it a +1. Since JSPM is trying to solve a similar problem to what reejs is trying to too, I believe reejs (and me) would benefit from the same 🙃

guybedford commented 9 months ago

I've added a new exclude feature to the download API for JSPM in https://jspm.org/cdn/api#download.

Having a file list for esm.sh would be enough to implement it actually - since it's a filter applied against the existing files list once that list is identified.