styxlab / gatsby-theme-try-ghost

A Gatsby theme to build flaring fast blogs from headless Ghost CMS
MIT License
136 stars 56 forks source link

support for ignoring http errors like 403 when try to download image #265

Closed ghost closed 3 years ago

ghost commented 3 years ago

Hi, This happened very rarely for me, But few times image could not get download because of 403 error.

Right now plugin uses Promise.all to download all images but if even one fails whole batch will fail . and right now code throws error if there is a http error in any image

try {
    fileNode = await createRemoteFileNode({
      url,
      store,
      cache,
      createNode,
      createNodeId
    });

    if (fileNode) {
      await cache.set(cacheKey, {
        fileNodeId: fileNode.id
      });
    }
  } catch (e) {
    throw Error(e);
  }

I think there should be an optional config to ignore http errors in this plugin, that would be helpful in certain cases.

Thanks

styxlab commented 3 years ago

@rahulbreezo: Welcome to this project! The possibility you describe is worth thinking about. However, before trying to work on the symptoms, I'd rather like to understand the root cause. From a conceptual point, a failing build is better than a successful build with missing images.

The 403 comes from your Ghost instance (or the service hosting your Ghost), so please contact your provider or debug your self-hosted setup in order to find out why image fetching does not work reliably.

ghost commented 3 years ago

@styxlab I am getting webflow images https://uploads-ssl.webflow.com/... in few blogs. And among them only few having this 403 error problem ,That' s I wanted an optional param to handled these erros Actually getting access denied on those webflow images


<Code>AccessDenied</Code>
<Message>Access Denied</Message>
<RequestId>3AYS11V7NW7JTR3Q</RequestId>
<HostId>2MesEWL59TWr3uTKYhza+k5o7snkk6kMK1/qXkDEQB0gASXIpUiYt+Wanig7YcOOCzINe8nYFZ8=</HostId>
</Error>```
ghost commented 3 years ago

yeah I made sure, due to failing of any img download , causing failing of htmlAst creation for that whole blog.

ghost commented 3 years ago

@styxlab Actually It will be fine if i could just filter it based on domain, is providing that feature is acceptable?

styxlab commented 3 years ago

I don't want to add code that only masks errors. The right way to deal with this issue is to

  1. Identify the images you don't have access to in your Ghost CMS.
  2. Replace those image with image you do have access to.
  3. Discuss with your provider how to prevent such problems or switch to another, more reliable provider.

If I would add such features, I would get a lot of issues where people don't understand why their images don't show up in their blog site and it would be much harder to debug and find the root cause for it.

However, if you really feel adding those features are necessary for you use case, I recommend you to fork this repo and change it to your own needs. Hope this is okay for you!?

ghost commented 3 years ago

ohk, we decided to get images from another cdn. No download while building