Fetching ZIP files directly via API

[lloesche]

A user of my Valheim Docker container created the following issue

https://github.com/lloesche/valheim-server-docker/discussions/279

asking if I would add BepInExPack Valheim support to my container like I did for ValheimPlus.

I looked into your API and saw that there are no direct links to the ZIP archive available as part of the API response. I figured this is likely on purpose.

But I though I'd just ask how okay or not okay you would be if my container checked for updates against your API and when it finds one downloads and installs it directly. Figured in the end this just updates the server side and all clients still have to open your website or download manager to download the mod for their desktop client.

[Woovie]

I looked into your API and saw that there are no direct links to the ZIP archive available as part of the API response

You can look at the headers returned in the description of a mod for the download URL.

curl -I https://valheim.thunderstore.io/package/download/Woovie/BroheimModpack/1.3.1/
HTTP/2 302
date: Sun, 28 Mar 2021 21:58:58 GMT
content-type: text/html; charset=utf-8
content-length: 0
location: https://cdn.thunderstore.io/live/repository/packages/Woovie-BroheimModpack-1.3.1.zip
x-frame-options: DENY
x-content-type-options: nosniff
referrer-policy: same-origin
strict-transport-security: max-age=15724800; includeSubDomains

Location returns the download that is returned, including filename.

[MythicManiac]

@lloesche looks like you used the experimental API, which as the name would suggest is experimental and not yet fully ready.

I'd recommend you use the v1 API endpoint e.g. https://valheim.thunderstore.io/api/v1/package/ and use the download URLs listed here. They're links to the website, but the website will respond with a 301 redirect to the actual file location. In other words, you can rely on them taking you to the file download, and using wget for example works just fine, it just doesn't know the filename unless you specify it manually or use the flag --content-disposition. For example:

$ wget --content-disposition https://valheim.thunderstore.io/package/download/denikson/BepInExPack_Valheim/5.4.601/

--2021-03-29 01:49:57--  https://valheim.thunderstore.io/package/download/denikson/BepInExPack_Valheim/5.4.601/
Resolving valheim.thunderstore.io (valheim.thunderstore.io)... 46.101.70.132
Connecting to valheim.thunderstore.io (valheim.thunderstore.io)|46.101.70.132|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://cdn.thunderstore.io/live/repository/packages/denikson-BepInExPack_Valheim-5.4.601.zip [following]
--2021-03-29 01:49:57--  https://cdn.thunderstore.io/live/repository/packages/denikson-BepInExPack_Valheim-5.4.601.zip
Resolving cdn.thunderstore.io (cdn.thunderstore.io)... 2606:4700:3035::ac43:8920, 2606:4700:3034::6815:26b5, 172.67.137.32, ...
Connecting to cdn.thunderstore.io (cdn.thunderstore.io)|2606:4700:3035::ac43:8920|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 6059910 (5.8M) [application/x-zip-compressed]
Saving to: 'denikson-BepInExPack_Valheim-5.4.601.zip'

denikson-BepInExPack_Valheim-5.4.6 100%[================================================================>]   5.78M  3.84MB/s    in 1.5s

2021-03-29 01:49:59 (3.84 MB/s) - 'denikson-BepInExPack_Valheim-5.4.601.zip' saved [6059910/6059910]

curl has similar flags:

$ curl -JLO https://valheim.thunderstore.io/package/download/denikson/BepInExPack_Valheim/5.4.601/

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100 5917k  100 5917k    0     0  5917k      0  0:00:01  0:00:01 --:--:-- 4608k
curl: Saved to filename 'denikson-BepInExPack_Valheim-5.4.601.zip'

Feel free to re-open this if the proposed solutions are not adequate for what you're looking for, but I'm closing the issue on the assumption that they'll work

[MythicManiac]

I also want to mention that the experimental package API does work, but it's not correctly subdomain-aware, because we're phasing out subdomains entirely in the next API version. The bepinex package you used as a test for example is only accessible via the valheim.thunderstore.io subdomain, which is why you probably hit issues.

If you want to use the experimental API, it will work as expected if you use the correct subdomain, e.g. https://valheim.thunderstore.io/api/experimental/package/denikson/BepInExPack_Valheim/5.4.901/

I would still recommend using the v1 API as it's guaranteed to be stable, whereas we might change things on the experimental API without prior notice.

[lloesche]

Ah that's fantastic. Thanks for the quick reply!

[lloesche]

@MythicManiac the issue I had with the v1 API was that it asked for a uuid as arg instead of namespace/name. So when checking whether there is an update available I didn't know how to query for the correct package.

[MythicManiac]

Yeah, with v1 the detail view is not actually implemented at all, you'll just want to fetch the whole index and parse from that. Might feel a bit excessive but it should be reasonably fast since it's fully cached.