mholt / timeliner

All your digital life on a single timeline, stored locally -- DEPRECATED, SEE TIMELINIZE (link below)
https://timelinize.com
GNU Affero General Public License v3.0
3.57k stars 116 forks source link

adding telegram integration #26

Closed hstolte closed 5 years ago

hstolte commented 5 years ago

Hi, here is a draft for integrating the telegram data export into timeliner (issue #17).

There are a couple of things that would need to be sorted out before merging the pull request.

  1. The telegram archive contains a lot of files that potentially have the same name (e.g. "giphy.mp4" or "sticker.webp"). I tried to circumvent the issue of overwritten files by prepending a hash of the timestamp and conversation id to the filename, but still, I can see some errors during the import and some files seem to be missing. Do you have any ideas? Here is the error message: [ERROR] Processing item graph: processing collection: adding item from collection to storage: replacing data file with identical existing file: removing duplicate data file: remove timeliner_repo\data\2018\04\telegram\2505156913-video9.mp4: The process cannot access the file because it is being used by another process.

  2. The datetime strings in this data source do not have any timezone information. Therefore I would suggest to add a custom parameter for the default timezone. See line 61 onwards in telegram.go

mholt commented 5 years ago

Awesome! Will take a look at it soon. This reminds me, I need to set up CI.

mholt commented 5 years ago

but still, I can see some errors during the import and some files seem to be missing. Do you have any ideas?

Are you using Windows perchance?

The datetime strings in this data source do not have any timezone information. Therefore I would suggest to add a custom parameter for the default timezone.

We could either default to UTC or make the time zone configurable as a parameter, yes.

hstolte commented 5 years ago

Are you using Windows perchance?

Yes, I do. However, no other application is accessing or locking the files in parallel. What could be the problem here?

mholt commented 5 years ago

I'm not sure. Please let us know if you find out! Would love to fix it.

mholt commented 5 years ago

Closing due to inactivity. Feel free to continue working on this and then open it back up if you feel like it!