feat: download base64 images and temporary s3 URL's instead of them being as they are.

[that-ambuj]

This Pull request looks to fix the issue #55, by fixing the behaviour of images in base64 format and images stored in notion's s3 storage by converting them to actual images instead of 3 hour limited s3 signed url or base64url strings. This PR is looking to solve the issue when someone wants to statically host their notion files using Hugo or any other static-site generator.

[souvikinator]

What do you think about the image not being rendered by many of the markdown parsers? It doesn't work on GitHub or other major online markdown editor services.

[that-ambuj]

The main reason behind this PR is for people to easily build static sites using Hugo or Jekyll. There is a huge hurdle when people use this tool to build and deploy static sites from their notion database. The main ones being:

• Markdown parsers and builders not processing base64urls correctly
• No correct caption for images taken from upstream
• Image URLs(s3) expiring 3 hours after downloading markdown files using this tool

In respect to ephemeral URLs, people will have to rebuild their sites every 3 hours for no reason even when there is no actual change to. This wastes time, energy and power to setup and deploy a CD pipeline and building it every 3 hours.

As for the question of not being rendered by many of the markdown parsers, I have a solution in mind: Basically we give the users of this library an option to either encode all images in base64url in markdown files(which is bad for markdown parsers, and this PR should fix it) or download all images to a given directory as they please. This creates uniformity in the working this library and reduces a lot of the hassles when using this library.

[souvikinator]

Sounds good to me and will serve both use cases. Not sure about downloading the image, maybe it can directly use the url for now.

One can pass this as an option like so:

const n2m = new NotionToMarkdown({ notionClient: notion, image_to_base64: true });

or if you have something else in mind feel free to share.

[that-ambuj]

Yeah, thanks for the suggestion. I'll starting working on the base64 option in a seperate PR and will let you know. We'll iron out rest of the things after that.

[souvikinator]

@that-ambuj what's the status?

[that-ambuj]

Hi @souvikinator, I have moved the download feature to another branch as it is WIP and I have worked on the Base64 Image feature and It is available at https://github.com/that-ambuj/notion-to-md/tree/feature-image-to-base64

The features has been built but is kind of incomplete, because I need your help with two other issues:

• This feature needs to be tested before being pushed to the main branch. So, I'll need your help on how to do it correctly as there are many variables that affect this feature.
• This feature converts images to url-safe base64 string but does not prefix data:image/png before the base64 string for the mime type(which it can't detect and may be manually needed to be added based on file extension). Also, we'll need to test if the data prefix is even needed by the markdown parsers that work with base64 data(github and vscode are known to NOT work with base64).

[souvikinator]

Alright! Sure, so you want to know how to test it locally? Also, what are the factors that can affect this feature? About data:image/png we can test it out with popular static site generators and online editors that support base64 image.

[that-ambuj]

I want to write automated tests such that the feature can be tested automatically. Just like in the .spec.ts file. For static site generators, we can use Hugo but I don't think it handles base64 that well, I haven't tried 11ty so I'll test that too.

[souvikinator]

Well, we have to figure that out. We can use https://picsum.photos/200 and convert that to base64 however I'm not sure if that's the best practice. Maybe we can do something with the image object or canvas to mock an image.