Closed kodjima33 closed 3 weeks ago
I have worked with Notion API before, can you lock this task to me.
I'm interested in working on this also.
I can work on this, I made an extensive personal python api wrapper for notion (although it is personal). this sounds pretty easy
Hey I've already done this before! I've got setup the project locally and got the notion oauth setup. In the next day or two I will be able to complete this! Here's is the link to the PR. Link to the PR
It would be great of you could assign this issue to me.
The project locally setup.
@Jainakin @oussamajabnouni @proudguy69 @dhanush17-tech
Added more details to the task, but have some questions for you guys.
Questions
Pls DM me on discord https://discord.gg/9puRjM2g with answers to those questions if you are still interested.
@josancamon19 is my nickname.
Hey 👋!! I have a few questions, listed them below:
There's no way to detect changes on a notion page but we can have a cron job to reindex every other day or something of that sort.
For chunking data we could use Semantic Chunking using NLP but the processing time is going to be a lot or we could just stick to Fixed-Length Chunks like 200-300 words.
Why we do need to create a firestore collection for all the notion pages?
Hi @dhanush17-tech , I hope this comment will help you, if any:
Don't forget to take a look at mem.ai, make sure you have an account and try their import from Notion feature.
So, when you figure out these things, you will have clearly picture to offer the best solution in both technical and product.
After that, le~e~t code.
@Jainakin @oussamajabnouni @proudguy69 @dhanush17-tech any updates on the code?
Unfortunately we don't assign bounties without previous PRs - you can submit a PR and if it makes sense, will lock it on you!
So here is how I would go about it. I might be wrong so correct me.
Step 1 (Setup OAuth) Create implement Notion OAuth. for this we go to Link to create the public integration key (the redirect url should be com.friend://notion_callback) this will get you a Client ID to perform the OAuth. The OAuth could be performed in the flutter app with an inapp browser after which we will get the access token to perform the rest of the page fetch functions.
Step 2 (Fetch all the notion pages and vectorize it) We could get all the notion pages with https://api.notion.com/v1/search, once we get it we could pass it through a parser and convert it to a markdown/html and then chunk it with a fixed length.
Step 3 (Check for Changes in the pages) The Notion API allows you to search and filter pages based on the last_edited_time. You can use this to request only the pages that have been edited or created since the last sync. Make an API call to fetch pages where the last_edited_time is greater than the last sync time. We could call this function periodically, perhaps twice everyday or once and the vectorization would only happen if they are any changes at all, so we make this efficient.
I still don't find the need for a Firestore database here, even on scale with 1000 users, this should work fine. Again, correct me if I'm wrong.
Created Oauth. If anyone needs it, send me a message in telegram @kodjima33
Latest update, not planned for now.
Is your feature request related to a problem? Please describe. Users want to be able to chat with their notion data. Notion AI is not smart and doesn't do it well. We need to let people to upload their Notion into Friend app and that data should be used same way we currently use memories.
Describe the solution you'd like You need to make a "Connect Notion" plugin for Friend app, in which the user will be able to authorize their Notion account through oauth and import all their notion content into Friend.
Check how Mem AI does it - you need to do the same
We already have a notion plugin but it is for exporting memories once created to a notion db.
This plugin should index all your notion content into Friend vector db, and allow you to chat with this data through Friend app chat.
Consider that one person might have 100k words in notion, and possibly 100s or 1000s of pages, we can't put that directly into memories, so possibly a new collection in firestore should be created, and the notion indexed should probably not appear in the memories tab page but somewhere else in the app.
Questions
Learn how to create plugins here
Additional context This is a paid task. Reward is $2000 in cash. Simply link your PR with this task and we will check it asap. Check how Contributions and bounties work here
If no one is assigned to this task, then no one is working on it. We don't lock bounty tasks unless you've created PRs before.