jamalex / notion-py

Unofficial Python API client for Notion.so
MIT License
4.29k stars 474 forks source link

Notion Webclipper functionality #16

Open 5hay opened 5 years ago

5hay commented 5 years ago

The webclipper is convenient because it also saves the content of the page.

I'd love to be able to programmatically request Notion to parse and save the contents of links into a Notion table.

Background: I want any link I store into my Pocket account to also be available in my Notion list (especially the content).

jamalex commented 5 years ago

This would be a great candidate for a community contribution, if anyone wants to take a stab! I haven't played with the webclipper functionality much yet myself, or looked at the backend data structures. Happy to answer questions, though, if anyone wants to take it on.

Penagwin commented 5 years ago

I'll try my hand at it later today.

Looks like the endpoints are getWebClipperData, searchWebClipperPages, and addWebClipperURLs

Blucknote commented 5 years ago

In simplest way looks like requests.post can be used

curl -X POST -H "Content-Type: application/json" --cookie "token_v2="YOURTOKEN" -d '{"type":"block","blockId":"PAGE_WHERE_WEBCLIPPEDPAGE_APPEAR","items":[{"url":"https://gist.github.com/subfuzion/08c5d85437d5d4f00e58","title":"curl examples"}],"from":"curl"}' https://www.notion.so/api/v3/addWebClipperURLs

Will get you response: {"parentBlock":{"record":{"table":"block","type":"page","id":"PAGEID","role":"editor","name":"NAME","iconEmoji":"📲"},"ancestors":[]},"createdBlockIds":["CREATEDBLOCKID"]}

Data pass formatted:

{
    "type": "block",
    "blockId": "BLOCK_WHERE_PAGE_APPEAR",
    "items": [
        {
            "url": "URL",
            "title": "TITLE"
        }
    ],
    "from": "ANYTHING?"
}

Url https://www.notion.so/api/v3/addWebClipperURLs

Whole page structure renders in Notion correctly. I guess backend is responsible to that. Not sure if block id can be nested page

5hay commented 5 years ago

Thank you for providing the raw curl command. Unfortunately it doesn't seem to work, I'm getting the following response:


{ 
   "errorId":"RANDOM ID",
   "name":"ValidationError",
   "message":"Invalid record request"
}
Blucknote commented 5 years ago

in "blockId" I passed top-level page.id gotten from:

page = client.get_block("https://www.notion.so/PAGE")
page.id
'99999999-9999-9999-9999-999999999999'
{
    "type": "block",
    "blockId": "99999999-9999-9999-9999-999999999999",
    "items": [
        {
            "url": "https://www.postgresql.org/docs/current/functions-json.html",
            "title": "result page"
        }
    ],
    "from": "ANYTHING?"
}

https://www.notion.so/result-page-85559943128b4c14803f7d1de15017a3

5hay commented 5 years ago

Oh my bad, somehow I thought the blockId is just the url part (in your example PAGE). Thank you for clarifying!

jamalex commented 4 years ago

Did anybody make any progress on adding this to the library? Let me know if you have any questions!

realCrush commented 4 years ago

in "blockId" I passed top-level page.id gotten from:

page = client.get_block("https://www.notion.so/PAGE")
page.id
'99999999-9999-9999-9999-999999999999'
{
    "type": "block",
    "blockId": "99999999-9999-9999-9999-999999999999",
    "items": [
        {
            "url": "https://www.postgresql.org/docs/current/functions-json.html",
            "title": "result page"
        }
    ],
    "from": "ANYTHING?"
}

https://www.notion.so/result-page-85559943128b4c14803f7d1de15017a3

hello, I have tried what you say, but still get response like: { "errorId":"RANDOM ID", "name":"ValidationError", "message":"Invalid record request" }

May I ask if your code still work? And can you tell me if https://www.notion.so/PAGE in " page = client.get_block("https://www.notion.so/PAGE") " must be a database page or normal page? I was confused

Thanks!

elblogbruno commented 4 years ago

Take a look on how I have implemented it myself https://github.com/elblogbruno/NotionAI-MyMind

amariusz commented 4 years ago

I've been trying to make this curl command work but got responses like :

"name":"ValidationError","message":"Invalid input.
"name":"ValidationError","message":"User cannot edit block."

Turned out

Thanks @elblogbruno for code sample.

TitusRobyK commented 3 years ago

curl -X POST -H "Content-Type: application/json" --cookie "token_v2="YOURTOKEN" -d '{"type":"block","blockId":"PAGE_WHERE_WEBCLIPPEDPAGE_APPEAR","items":[{"url":"https://gist.github.com/subfuzion/08c5d85437d5d4f00e58","title":"curl examples"}],"from":"curl"}' https://www.notion.so/api/v3/addWebClipperURLs

Hey, Just for learning purposes , I tried my hand in obtaining the addWebClipperURLs , i was using Google Chrome , when i trigger the Webclipper Extension , i couldn't register any network calls , What method did you try ?

Blucknote commented 3 years ago

Hey, Just for learning purposes , I tried my hand in obtaining the addWebClipperURLs , i was using Google Chrome , when i trigger the Webclipper Extension , i couldn't register any network calls , What method did you try ?

You can debug extension and there you can view network calls. BTW with official API I'm not sure how this will work in future

TitusRobyK commented 3 years ago

Hey, Just for learning purposes , I tried my hand in obtaining the addWebClipperURLs , i was using Google Chrome , when i trigger the Webclipper Extension , i couldn't register any network calls , What method did you try ?

You can debug extension and there you can view network calls. BTW with official API I'm not sure how this will work in future

Hey Thanks btw, But for me Inspect views : background.html is not coming , when i open manage extensions

image

May be they might have done something to hide it ... not sure though

Blucknote commented 3 years ago

Hey Thanks btw, But for me Inspect views : background.html is not coming , when i open manage extensions

May be they might have done something to hide it ... not sure though

No problem. I left Chrome a while, found on SO this answer which might help. May be it's under details?

Blucknote commented 3 years ago

@TitusRobyK can you try this method? chrome-extension://<extension-id>/background.html Found from this article

TitusRobyK commented 3 years ago

Hey, Just for learning purposes , I tried my hand in obtaining the addWebClipperURLs , i was using Google Chrome , when i trigger the Webclipper Extension , i couldn't register any network calls , What method did you try ?

You can debug extension and there you can view network calls. BTW with official API I'm not sure how this will work in future

Thank a lot @Blucknote

TitusRobyK commented 3 years ago

chrome-extension:///background.html

Nope this did not work , Even checked the Network call as well , upon adding an article via Notion Web Clipper , I guess its some sort of new implementation i think ( i am not sure though )

image

wns3645 commented 3 years ago

Hi, it seems that the Notion web clipper does not use background html, but it uses popup html(and the file name is index.html). So you can check network calls via inspection of popup html.

image

TitusRobyK commented 3 years ago

Hi, it seems that the Notion web clipper does not use background html, but it uses popup html(and the file name is index.html). So you can check network calls via inspection of popup html.

image

thank you !!!!! Never noticed it