NotionX / react-notion-x

Fast and accurate React renderer for Notion. TS batteries included. ⚡️
https://react-notion-x-demo.transitivebullsh.it
MIT License
4.79k stars 559 forks source link

Notion image security issues #395

Closed Pipe-Runner closed 1 year ago

Pipe-Runner commented 1 year ago

Description

I hope it is a one-time issue or a random event since I have a presentation in a few days for which I plan on using my website built on top of notion-x. I noticed that I had a static build running yesterday with all the bells and whistles, but today when I was showing the blogs, the images didn't appear at all. I tried to open the image on a new tab, and I got a security error saying, "Access denied". I checked on a local build, and they were working fine. So I rebuilt the static build and then hosted it again, and now it's okay. Has anyone noticed images breaking on static build? Has notion introduced some sort of a new security mechanism for images?

Notion Test Page ID

This is the image I opened on a new tab:

https://s3.us-west-2.amazonaws.com/secure.notion-static.com/df539596-4970-4750-87f2-d8d2cbb941c9/SmartSelect_20210609-163909_Samsung_Notes.jpg?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Credential=AKIAT73L2G45EIPT3X45%2F20221020%2Fus-west-2%2Fs3%2Faws4_request&X-Amz-Date=20221020T102349Z&X-Amz-Expires=86400&X-Amz-Signature=35da7bc031f2fd87261431147b388ac354be615e394f4f5c665c1e86bd656ec3&X-Amz-SignedHeaders=host&x-id=GetObject

It is from this page:

Pipe-Runner commented 1 year ago

I am sorry if this is a false alarm, but my website breaking on the day of the presentation is the last thing I need.

Pipe-Runner commented 1 year ago

Yup, it seems like the image links expire within a day. Can someone please help me out with this?

Pipe-Runner commented 1 year ago

This is what I get when I open the image in a new tab: image

For this image link: https://s3.us-west-2.amazonaws.com/secure.notion-static.com/a93b0b82-7783-4340-a60b-e072a885986b/Notes_221020_053031.jpg?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Credential=AKIAT73L2G45EIPT3X45%2F20221022%2Fus-west-2%2Fs3%2Faws4_request&X-Amz-Date=20221022T174259Z&X-Amz-Expires=86400&X-Amz-Signature=fc23d68ebef2a90c32fb79974f07a4611117fab66ec68e0937e92004aa6077ec&X-Amz-SignedHeaders=host&x-id=GetObject

Pipe-Runner commented 1 year ago

It seems related to this: https://twitter.com/emilioaray/status/1463122263037857795 But how comes this never surfaced until now.

Pipe-Runner commented 1 year ago

@normdoow @transitive-bullshit do you guys know anything about this? I there something I can do to work around this problem?

Pipe-Runner commented 1 year ago

After a bit of looking around, the same image on the blog generated and rendered using notion-x vs the same page as a public page via notion has very different-looking URLs, respectively:

https://s3.us-west-2.amazonaws.com/secure.notion-static.com/a93b0b82-7783-4340-a60b-e072a885986b/Notes_221020_053031.jpg?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Credential=AKIAT73L2G45EIPT3X45%2F20221022%2Fus-west-2%2Fs3%2Faws4_request&X-Amz-Date=20221022T174259Z&X-Amz-Expires=86400&X-Amz-Signature=fc23d68ebef2a90c32fb79974f07a4611117fab66ec68e0937e92004aa6077ec&X-Amz-SignedHeaders=host&x-id=GetObject

https://piperunner.notion.site/image/https%3A%2F%2Fs3-us-west-2.amazonaws.com%2Fsecure.notion-static.com%2Fa93b0b82-7783-4340-a60b-e072a885986b%2FNotes_221020_053031.jpg?table=block&id=852b9329-6142-44a8-a01c-4d4580f6ba02&spaceId=a03366e1-eeb8-4da4-b12e-4cfa2a4bbfcf&width=2000&userId=&cache=v2

One has a 24hr expiration date, while the other does not.

Pipe-Runner commented 1 year ago

Dumping whatever I could find here:

transitive-bullshit commented 1 year ago

Yes, this is a well-understood albeit unfortunate aspect of Notion's API.

All files uploaded to Notion including images are hosted by their AWS S3 account as private files and must be accessed via a "signing" endpoint that returns URLs with a 24-hour expiry time. It'll be the same regardless of which Notion wrapper you're using.

In the example you gave, the X-Amz-Expires=86400 means that this signed url will expire after 24 hour.

The recommended way to solve this is to either:

  1. Have your pages cached for no more than 24 hours so both they and their embedded images will be re-generated daily
  2. Add a persistent cache / CDN in front of all your file / image assets. This can be tweaked via mapImageUrl https://github.com/NotionX/react-notion-x/blob/master/packages/react-notion-x/src/map-image-url.ts

See my accompanying starter kit for more inspiration https://github.com/transitive-bullshit/nextjs-notion-starter-kit

Pipe-Runner commented 1 year ago

@transitive-bullshit yeah, after digging for 3hrs, I understood what was going on and what I was taking for granted. I noticed that once you have a public page, they host images for the public page differently. They end up having fixed URLs with notion.so/image/${emb_url}. Is it safe to assume that this is something static since it does not come with the AMZ query params or the 24hr expiry?

Sorry if I am asking dumb questions.

Pipe-Runner commented 1 year ago

What I don't understand is, how has it surfaced in my blog all of a sudden. Even in Notion-X, the logic hasn't changed for the default mapImageUrl. I am so confused. 😓

transitive-bullshit commented 1 year ago

Is it safe to assume that this is something static since it does not come with the AMZ query params or the 24hr expiry?

It's unlikely that it's completely static. Notion has to maintain very granular file permissions under the hood given their access control design (e.g., since you can make individual blocks / pages private at any time). It's likely pointing to a redirection under the hood which is using the same signing mechanics.

What I don't understand is, how has it surfaced in my blog all of a sudden. Even in Notion-X, the logic hasn't changed for the default mapImageUrl. I am so confused. 😓

Notion's unofficial API sometimes changes as their team develops new releases. We try to maintain compatibility as best we can with both their behavior and especially breaking changes, but it's not officially supported by Notion, and this is a free / OSS project, so it's not always perfect. Not sure what would've changed or why you wouldn't have noticed it until now, but if you figure it out, let me know and I'll look into it.

Pipe-Runner commented 1 year ago

@transitive-bullshit Thanks a lot for your time. If I manage to find something, I'll let to community know. Have a good day.

Pipe-Runner commented 1 year ago

Writing a custom image mapping function and removing the Amz part from it was the way to go. As of now, it has been more than 24hrs, and my images are still intact. I might move to a CDN later if I face this issue again. Closing the ticket for now.

byseop commented 1 year ago

Writing a custom image mapping function and removing the Amz part from it was the way to go. As of now, it has been more than 24hrs, and my images are still intact. I might move to a CDN later if I face this issue again. Closing the ticket for now.

@Pipe-Runner I have the same issue. Can you share the custom function?

Pipe-Runner commented 1 year ago

@byseop

import { Block } from 'notion-types'; 

 export const customMapImageUrl = (url: string, block: Block): string => { 
   if (!url) { 
     throw new Error("URL can't be empty"); 
   } 

   if (url.startsWith('data:')) { 
     return url; 
   } 

   // more recent versions of notion don't proxy unsplash images 
   if (url.startsWith('https://images.unsplash.com')) { 
     return url; 
   } 

   try { 
     const u = new URL(url); 

     if ( 
       u.pathname.startsWith('/secure.notion-static.com') && 
       u.hostname.endsWith('.amazonaws.com') 
     ) { 
       if ( 
         u.searchParams.has('X-Amz-Credential') && 
         u.searchParams.has('X-Amz-Signature') && 
         u.searchParams.has('X-Amz-Algorithm') 
       ) { 
         // if the URL is already signed, then use it as-is 
         url = u.origin + u.pathname; 
       } 
     } 
   } catch { 
     // ignore invalid urls 
   } 

   if (url.startsWith('/images')) { 
     url = `https://www.notion.so${url}`; 
   } 

   url = `https://www.notion.so${ 
     url.startsWith('/image') ? url : `/image/${encodeURIComponent(url)}` 
   }`; 

   const notionImageUrlV2 = new URL(url); 
   let table = block.parent_table === 'space' ? 'block' : block.parent_table; 
   if (table === 'collection' || table === 'team') { 
     table = 'block'; 
   } 
   notionImageUrlV2.searchParams.set('table', table); 
   notionImageUrlV2.searchParams.set('id', block.id); 
   notionImageUrlV2.searchParams.set('cache', 'v2'); 

   url = notionImageUrlV2.toString(); 

   return url; 
 };
Pipe-Runner commented 1 year ago

Slap the function into your blog renderer and it should work.

byseop commented 1 year ago

customMapImageUrl

@Pipe-Runner It works. Thank you so much. This only works on public posts (if the post is private, you do not have permission), but this is enough. You save my time. Thanks again!

alexblack commented 1 year ago

@byseop thanks, I tried your code, it works well, I like that it means I now I have image urls that don't expire.

However, I hit a problem, for some images I get a 2000px wide version, when I expected like 184px. eg this original notion image link: https://s3.us-west-2.amazonaws.com/secure.notion-static.com/d70d5bf5-52c6-44fc-8dc9-d1e65e3a5d80/Untitled.png?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Credential=AKIAT73L2G45EIPT3X45%2F20230301%2Fus-west-2%2Fs3%2Faws4_request&X-Amz-Date=20230301T205340Z&X-Amz-Expires=86400&X-Amz-Signature=3b7aa0641c771394862b88ef59e8c4ddca9fb78e8b2d49a0a3ee0104ea107eec&X-Amz-SignedHeaders=host&x-id=GetObject

(which expires)

is 184px wide, but when I convert it with your code, I get a 2000px wide image

https://www.notion.so/image/https%3A%2F%2Fs3.us-west-2.amazonaws.com%2Fsecure.notion-static.com%2Fd70d5bf5-52c6-44fc-8dc9-d1e65e3a5d80%2FUntitled.png?table=block&id=15ee0b3d-92f5-4c48-a7d1-3b5601300ed8&cache=v2

refresh
byseop commented 1 year ago

@byseop thanks, I tried your code, it works well, I like that it means I now I have image urls that don't expire.

However, I hit a problem, for some images I get a 2000px wide version, when I expected like 184px. eg this original notion image link: https://s3.us-west-2.amazonaws.com/secure.notion-static.com/d70d5bf5-52c6-44fc-8dc9-d1e65e3a5d80/Untitled.png?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Credential=AKIAT73L2G45EIPT3X45%2F20230301%2Fus-west-2%2Fs3%2Faws4_request&X-Amz-Date=20230301T205340Z&X-Amz-Expires=86400&X-Amz-Signature=3b7aa0641c771394862b88ef59e8c4ddca9fb78e8b2d49a0a3ee0104ea107eec&X-Amz-SignedHeaders=host&x-id=GetObject

(which expires)

is 184px wide, but when I convert it with your code, I get a 2000px wide image

https://www.notion.so/image/https%3A%2F%2Fs3.us-west-2.amazonaws.com%2Fsecure.notion-static.com%2Fd70d5bf5-52c6-44fc-8dc9-d1e65e3a5d80%2FUntitled.png?table=block&id=15ee0b3d-92f5-4c48-a7d1-3b5601300ed8&cache=v2

refresh

@alexblack Try adding an attribute like width=200 to the end of the URL.

https://www.notion.so/image/https%3A%2F%2Fs3.us-west-2.amazonaws.com%2Fsecure.notion-static.com%2Fd70d5bf5-52c6-44fc-8dc9-d1e65e3a5d80%2FUntitled.png?table=block&id=15ee0b3d-92f5-4c48-a7d1-3b5601300ed8&cache=v2&width=200

It's not a complete solution, but it seems to work.

alexblack commented 1 year ago

thanks @byseop hmm, how would I know what width to put? Some image are larger, some are smaller. I'm not understanding why the width of the image changes from 184px to 2000px :(