thunderstore-io / Thunderstore

Thunderstore is a mod database and API for downloading mods. Thunderstore Discord: https://discord.thunderstore.io/
https://thunderstore.io/
GNU Affero General Public License v3.0
134 stars 27 forks source link

Support mirror CDN for package and shared profile downloads #960

Closed anttimaki closed 10 months ago

anttimaki commented 10 months ago

API calls can indicate that mirror CDN is preferred by including a query paramater flag in the request. Currently this is supported by the API endpoints that provide download URLs for packages and shared mod manager profiles.

If the backend receives the query parameter flag and is configured to use MirroredS3Storage, it will try to return URLs that point to the first mirror CDN.

This is intended to solve the small but growing number of incidents where users for some reason can't access Thunderstore's main CDN.

Refs TS-2003

codecov[bot] commented 10 months ago

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Comparison is base (7ba4403) 92.93% compared to head (6e1659b) 92.94%.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## master #960 +/- ## ========================================== + Coverage 92.93% 92.94% +0.01% ========================================== Files 286 286 Lines 8488 8502 +14 Branches 788 790 +2 ========================================== + Hits 7888 7902 +14 Misses 496 496 Partials 104 104 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

MythicManiac commented 10 months ago

These shouldn't be tied to the mirror storage definitions at all, as those might have nothing to do with public-facing CDN routing.

If we want to validate the domains (which is not a bad idea), add a new setting for it and go through that list instead.

anttimaki commented 10 months ago

These shouldn't be tied to the mirror storage definitions at all, as those might have nothing to do with public-facing CDN routing.

If we want to validate the domains (which is not a bad idea), add a new setting for it and go through that list instead.

Reimplemented as:

  1. preferred CDN domain is received as query parameter
  2. Config includes allowlist for legit CDNs
  3. If preferred CDN is allowed, replace the domain and pray the CDN actually has the file at the same path