search

monicahq / monica

Personal CRM. Remember everything about your friends, family and business relationships.
https://beta.monicahq.com
GNU Affero General Public License v3.0
21.05k stars 2.1k forks source link

Allow importing of conversations from WhatsApp #1827

Open TomGranot opened 5 years ago

TomGranot commented 5 years ago

Thanks for filing an issue and for your interest in the project.

Is your feature request related to a problem? Please describe. Not a problem per se, but what seems like a logical use case for the Conversations feature recently added.

Describe the solution you'd like I talk on WhatsApp A LOT. I think most of my communication with people is done over the platform - including keeping up with people Monica reminds me to keep up with! I would like a way to import my WhatsApp chat log into the app, so that it filters out only the contacts I have on monica, and updates the conversations accordingly.

Describe alternatives you've considered (optional) Another way to go about it is to do a "Share To Chandler" type thing, where I select a few messages from WhatsApp, then do a Share to the Chandler app, and it parses what I shared, asks me which contact to attach it to , and then creates the conversations appropriately. This might be better than the other option, since it lets me filter what gets added in a certain conversation and what doesn't.

Wouter0100 commented 5 years ago

Would be a great feature, unfortunately - WhatsApp does not have an API of some sort, so it complicates this process a lot

petritz commented 5 years ago

With WhatsaApp there is the funcationality to export the chat, this results in a plain .txt file (1 message per line) and if you select, it also exports the media (images, videos, etc) with it.

So there is definitly the way of just importing those .txt file and representing them as chat messages (grouped by day?) in monica. The bad part however is the updating process. One would need to export each chat and transmit it somehow to monica (via email, dedicated app, etc)

TomGranot commented 5 years ago

@petritz That could be done with a wrapper app for Android. So I'd imagine a reminder popping up once a week with instructions on how to do that, and (if you're hosting monica yourself) an option to upload that file to Monica. This would require some script on the server that will do the actual import, but eventually it's just a parser for that txt file (which probably exists somewhere already) with some SQL to throw it in the correct place.

It could also be a great way of batch-adding contacts to Monica - I have 70-80% of my correspondence over WhatsApp, so If I could have all those people (AND NOT all the people in my address book - some of which I haven't spoken to in years) in Monica that would save me a lot of manual work.

@asbiin @djaiss I think this is something definitely worth doing. Is there some roadmap I can add that to? I'll assign it to myself and code it sometime in the summer.

petritz commented 5 years ago

I don't know about Android... with iOS that would not be possible.

Anyway there is also one possibility I haven't considered: Fetching the data directly from the Whatsapp Sqlite Database. This would be even easier once schema is understood. You just have to find a way to easily export this database.

TomGranot commented 5 years ago

Can you do that in iOS?

petritz commented 5 years ago

I have to investigate further. But afaik it is only possible via a PC connection (probably only Mac(?))Also we should check whether the ChatStorage.sqlite file hast the same schema in iOS and Android, so we don’t have to implement two separate import mechanism.

TomGranot commented 5 years ago

So, basically, more investigating:) Which is why I suggested we add it to the roadmap as a feature to implement, and then the whole process will be much smoother.

redlukas commented 4 years ago

might it be able to pull the conversations from the daily backups?

TomGranot commented 4 years ago

That was my thinking, but that's a Google Drive authentication issue. There needs to be a smart way somehow to architect this with minor permissions and easy operation. I'll have some time in a few months to work on it so I'll test some theories then.

Clausinho commented 4 years ago

just to chime in on this. there is a whatsapp business API which could be of use. Furthermore the Google drive authentication should not be a huge issue tbh. it just needs a Google API key for the self hosted variant and could also allow for Monica to access other stuff and notes from Google drive or even mirror data to there.

@tomgs which theories turned out to be solid?

TomGranot commented 4 years ago

@Clausinho I haven't touched it since the discussion, tbh. I also figured out that I'm literally only using the reminders feature of Monica, since manual data entry is not really a thing I enjoy;( I was thinking of forking Monica and removing all the cruft, leaving me with just the feature I wanted, then dockerizing that and throwing it on a server somewhere.

Clausinho commented 4 years ago

@tomgs totally can relate to not wanting to manually enter data. my dream would be a connection to the apis whatsapp, telegram, maybe discord, Facebook, LinkedIn and Google (contacts) allowing me to just combine the contacts I have and all their social media profiles, get all that data into Monica and then get reminded about birthdays and if I have not talked with them for a long time. basically like the App Cloze but with a bit more integrations. I might just have to also do a fork and maybe a PR because it seems my wishes are pretty specific to my use case only. if you fork I'd be interested in seeing how it evolves

Wouter0100 commented 4 years ago

I could look into implementing WhatsApp Web. This would require a different Go application for now, but I've recently made some WhatsApp bots with it. This could setup a Web Whatsapp connection at like 4AM and then read all the day's messages into Monica.

TomGranot commented 4 years ago

Hmmmp. How is auth being dine there? Or do you use a headless version of the user's current browser?

Wouter0100 commented 4 years ago

@tomgs auth between? The additional service will be authenticated using the QR code that will be provided by WhatsApp, and you should scan that QR code with your phone.

It's a implementation of the WhatsApp web API. It's indeed headless. See here.

TomGranot commented 4 years ago

@Wouter0100 oh, sorry, should have been clearer: i meant auth between the phone and the browser. I'll look at the link - but this sounds promising!

Wouter0100 commented 4 years ago

Authentication is just the same as web whatsapp works normally, although in a headless environment. If I'm correctly, it's persisted by a session token - which could be stored inside Monica.