search

PrivateBin / Directory

Rust based directory application to collect list of federated instances
https://privatebin.info/directory/
Other
25 stars 8 forks source link

Use new version of Mozilla Observatory #106

Closed vlnst closed 2 weeks ago

vlnst commented 2 months ago

The previous version of Mozilla Observatory is is deprecated:

The previous version of HTTP Observatory is still available, however it is now deprecated and will soon be sunsetted.

The deprecated version will stop working in September 2024.

We should use the new version instead of previous or at least be prepared that it can stop working anytime now and the PrivateBin Instance Directory will not be able to supply users with the most recent data about the instance security rating

elrido commented 1 month ago

We will, once the API which we consume has been updated. As per their FAQ it hasn't yet: https://developer.mozilla.org/en-US/observatory/docs/faq#has_the_http_observatory_api_been_updated_to_use_the_new_tests

I checked their code (it's OSS) and the API seems to be identical and all we retrieve from it are the ratings, so the only change we should notice are that one day lots of ratings may change at once.

vlnst commented 2 weeks ago

The new v2 API is available at an updated URL The v1 endpoint will be shut down on October 31, 2024.

Update

elrido commented 2 weeks ago

Was really as simple us updating the endpoint URL. Change got deployed in 0.12.0, result of a full cron job run was no change in ratings, no errors reported. Let's observe.

elrido commented 4 days ago

grml Sorry, turned out the change was subtly a bit different after all.

I got suspicious since there were no rating changes for a week and all new instances showed up with "-" rating. The request is now POST, not GET and the field with the status is now an integer instead of a string and named differently, so the JSON parser no longer mapped it (type-safety FTW). I could also remove the attempted rescan logic, as that call is no longer supported.

Mozilla have since added a section to their Readme on how to migrate that explains some of these subtleties better. Also, the reason I didn't see any updates or API related changes in their git repo was that the new API service lives in a different repo and is a new development. Clearly this could have been communicated/announced better. They have since added a deprecation message and links to the new git repo into the old repos Readme file.

Anyhow, re-running a manually triggered full scan after the upgrade to 0.13.0 yielded the following updated scans:

[...]
all instance update queries concluded (31.609385ms)
Instance https://paste.canine.tools/ rating updated to: A- (4.021µs)
Instance https://privatebin.lol/ rating updated to: C (4.053µs)
Instance https://bin.0xfc.de/ rating updated to: B+ (13.669µs)
Instance https://paste.nolog.cz/ rating updated to: C (10.141µs)
Instance https://paste.getcobalt.org/ rating updated to: B (5.005µs)
Instance https://paste.helkor.eu/ rating updated to: B- (3.703µs)
Instance https://paste.devsite.pl/ rating updated to: B (3.733µs)
Instance https://paste.psr.tf/ rating updated to: C (13.064µs)
Instance https://bin.68degrees.no/ rating updated to: B (4.578µs)
Instance https://privatebin.devol.it/ rating updated to: A+ (4.177µs)
Instance https://paste.tecff.de/ rating updated to: C (4.443µs)
Instance https://paste.fitgirl-repacks.site/ rating updated to: C (4.203µs)
Instance https://paste.nomagic.uk/ rating updated to: B- (5.75µs)
Instance https://www.nervengas.com/privacy/ rating updated to: D (7.037µs)
all scan update queries concluded (107.64792ms)
[...]