What are the options for dealing with a failed API request in a data file?

[smth]

If fetching from an API in a JavaScript data file, is there a way to avoid a build after a bad API response?

Say for example a site is being built on Netlify at a time when the API is down. Rather than have the site rebuild with missing data, can we make the build fail, or retry the API before building perhaps?

I currently have errors caught and console logged (so wouldn't be such an issue when working locally).

.catch(function (err) {
  console.error(':(', err);
});

[chrissy-dev]

This is an excellent use case I actually hadn't thought about. I currently delete my dist directory on build first which means even if I did catch the error during build I'd have nothing to fall back on 😰.

One solution could be to write a backup data file to disk which stores the last succesful response, then within catch return that if the API fails?

This isn't tested, but something like this? Here's an example grabbing an Insta feed using node-fetch.

npm i -D node-fetch

touch src/_data/instagram_backup.js

const fetch = require('node-fetch')
const fs = require('fs');
const instagram_backup = require('./instagram_backup');

module.exports = async function () {
  console.log('Fetching Instagram profile as JSON...')

  return fetch('https://www.instagram.com/scottishstoater/')
    .then(result => result.text())
    .then(res => {
      var jsonData = res.match(/<script type="text\/javascript">window\._sharedData = (.*)<\/script>/)[1].slice(0, -1)
      var json = JSON.parse(jsonData);

      fs.writeFile(`${__dirname}/instagram_backup.js`, `module.exports=${JSON.stringify(json.entry_data.ProfilePage[0])};`, function (err) {
        if (err) {
          return console.log(`There was an error writing the backup file:`);
        }
      });

      return json.entry_data.ProfilePage[0]
    })
    .catch(err => {
      console.error(`${err}: using backup`)
      return instagram_backup;
    });
}

Not sure if the _data dir is the correct place to store the backup file as it'll be picked up as this is just a concept I'll leave it there. Could introduce a _databackup folder or something similar and write the backup files to there.

Also interested in a more elegant solution if one exists?

[jeromecoupe]

An example of caching API data to files using flat-cache is available in the docs. https://www.11ty.dev/docs/quicktips/eliminate-js/

[smth]

I'm using caching as described in the docs. To be clear, it doesn't in itself avoid the problem, right? You would need the additional logic that says if there's an error, continue to use the expired cache, as suggested by @chrisssycollins

Probably also worth reiterating from the docs: "if you’re using this on a Netlify build, it will not maintain updates to the cache (as it resets the cache to the files that are checked into git) and will likely re-run every time". So for we'd need to commit the cache I guess.

[zachleat]

Howdy here I am three years later but we do have a very specific example of this on the Fetch plugin docs: https://www.11ty.dev/docs/plugins/fetch/#what-happens-when-a-request-fails

Thanks y’all!

This is an automated message to let you know that a helpful response was posted to your issue and for the health of the repository issue tracker the issue will be closed. This is to help alleviate issues hanging open waiting for a response from the original poster.

If the response works to solve your problem—great! But if you’re still having problems, do not let the issue’s closing deter you if you have additional questions! Post another comment and we will reopen the issue. Thanks!

[zachleat]

Oh, as an addendum: if you want to fail the build, don’t catch the error!