search

mlomb / chat-analytics

Generate interactive, beautiful and insightful chat analysis reports
https://chatanalytics.app
GNU Affero General Public License v3.0
708 stars 51 forks source link

DST in Telegram exports #79

Closed vignakke closed 1 year ago

vignakke commented 1 year ago

Hello, I get this error when I wait for the report to be generated, a friend told me that the cause might be the summer/winter time change if that helps.

_Error parsing file "result.json":

MessagesInterval can only be extended to the future_

mlomb commented 1 year ago

Good catch! Yes problably the cause is summer/winter time. Do you have a small report so that we can reproduce the problem? Also, which language is your Telegram client? And which timezone is your computer? (so I can try to replicate too)

vignakke commented 1 year ago

hi, no I don't have a small report, the report I wanted to upload is 180mb i have telegram in english, and timezone is utc+01:00

hopperelec commented 1 year ago

Could you produce a minimal reproducible example? That is, remove as much from result.json as possible in a way that still results in that error. I'd imagine you could just remove all but one messages from the file and it would still result in the same error

vignakke commented 1 year ago

i tried to remove like 480k and "only" 400k message (still 40k+ in the file but like this it's working), I have just deleted all the messages of the time change (2022-2023) and there indeed it works

hopperelec commented 1 year ago

Could you try removing all the messages except forsome going across the time change? Of course, you may want to obfuscate names and message content

Laiteux commented 1 year ago

Hello, encountering the exact same issue! Indeed seems related to summer/winter time change.

Can provide an export of around 1M Telegram messages from a private Telegram group I have with friends. Give me a few minutes, time to obfuscate it.

hopperelec commented 1 year ago

You shouldn't need to include all the messages to re-create the error, and so many messages will take some time to download. However, for future it might be a good idea to write a script which automatically obfuscates an exportso it's easier for samples to be provided for debugging and testing. I'll have a go at doing this later!

Laiteux commented 1 year ago

Sounds good!

Still, I'll include the whole 1M messages, because I remember some graphs not being generated (console error) even when I succeeded with the import. Would love to look into this with you as well.

hopperelec commented 1 year ago

Ah alright. I would make new issues for those then. On mobile right now so won't be able to have a good look until later, but once I can I'll be sure to!

Laiteux commented 1 year ago

Sure, will be happy to open a new issue after this one then! (as fixing this is basically a beforehand requirement)

Laiteux commented 1 year ago

Who should I send the file to? Don't feel like uploading it there. Any Telegram username I can contact you at? Mine is @Matteo

hopperelec commented 1 year ago

Oh, I had already downloaded the original file, sorry. I don't have Telegram but if you would prefer I use the updated version you can contact me at any of these places https://www.hopperelec.co.uk/contact/

hopperelec commented 1 year ago

https://github.com/mlomb/chat-analytics/issues/79#issuecomment-1537138675

You shouldn't need to include all the messages to re-create the error, and so many messages will take some time to download. However, for future it might be a good idea to write a script which automatically obfuscates an exportso it's easier for samples to be provided for debugging and testing. I'll have a go at doing this later!

I'ma work on this. Is this something that should be integrated into the repository in some way? If so, would it be a stand-alone file, would it be a part of the app or should it be a Jupyter Notebook (Python) so that it can be used straight from GitHub?

mlomb commented 1 year ago

Probably something like https://chatanalytics.app/sample. Making everything in plain JS in assets/public/sample.html I think it would be the most simple and practical option

Laiteux commented 1 year ago

https://chatanalytics.app/sample

/demo*, yep

mlomb commented 1 year ago

Maybe /obfuscate or /obfuscate_sample, or something else. sample may be confused with a demo 😅

Laiteux commented 1 year ago

Edited my comment, sorry if it wasn't clear

hopperelec commented 1 year ago

So the file should be stand-alone but accessible via the app?

mlomb commented 1 year ago

So the file should be stand-alone but accessible via the app?

Yup

Kaputchino commented 1 year ago

Is it normal that I still have this issue?

hopperelec commented 1 year ago

Yes, there has not yet been a good fix for this https://github.com/mlomb/chat-analytics/pull/80

mlomb commented 1 year ago

I have made some changes that I think will fix the issue in #80!

@Laiteux @Kaputchino @vignakke can you try your exports in https://pr-80.chat-analytics.pages.dev ?

Kaputchino commented 1 year ago

It's working just fine, thx.