How do I update data/recommendation/localData.json?

[pdehaan]

It looks like ./data/recommendation/localData.json may be coming from an API or something, but I'm not sure how I can download a newer version, or if it's just something that you manually update every X days/weeks or when new exploits are released.

If it comes from a sensible API, not sure if it'd be easier to auto generate this on some npm "postinstall" hook or something, or add some npm script to grab the latest available version.

[pdehaan]

Actually, it looks like this may just be a prettified version of the https://haveibeenpwned.com/api/v2/breaches endpoint. So I added this to a "./bin/fetch-breaches.js" file locally (and added an alias to the package.json scripts), so now I can just do:

$ npm run fetch-breaches

and it will update the file.

const fs = require('fs');
const path = require('path');

const fetch = require('node-fetch');

const BREACHES_URL = 'https://haveibeenpwned.com/api/v2/breaches';
const BREACH_PATH = 'data/recommendation/localData.json';

fetch(BREACHES_URL)
  .then((res) => res.json())
  .then((breaches) => {
    fs.writeFileSync(path.resolve(__dirname, '..', BREACH_PATH), JSON.stringify(breaches, null, 2));
    console.info(`Wrote breaches to ${BREACH_PATH}`);
  })
  .catch((err) => {
    console.error(err.message);
    process.exit(1);
  });

[pdehaan]

In fact, it felt absurd to leave that in a comment, so I just threw it in a PR instead for your amusement or deletion. Enjoy.

Ref: #50

[Osmose]

localData.json is a backup; the add-on code attempts to fetch from the API itself, and falls back to the local data if it can't hit the API: https://github.com/mozilla/security-advisor-shield-study/blob/master/lib/Advisor.js#L189

On one hand, keeping the backup up-to-date helps allow users who can't hit the API to still have some level of current protection. On the other hand, actually getting an up-to-date backup involves having network access that would allow you to get data from the API anyway, and API data always supercedes the local data, since all of this gets stored in simple storage anyway.

44 mentions wanting a script to keep the domains.txt file up-to-date for legal purposes, and the script from #50 could be re-purposed for that. But for updating localData.json itself, I don't think we need to. So this issue at least is not something we want to add. Thanks though!