Freika / dawarich

Self-hosted alternative to Google Location History (Google Maps Timeline)
https://dawarich.app
GNU Affero General Public License v3.0
766 stars 16 forks source link

Can't import from new Google timeline. #113

Closed ve3 closed 2 weeks ago

ve3 commented 3 weeks ago

Describe the bug Due to the new Google timeline is not available on PC* web browser any more, it is available on app only. I can export data use Google takeout but I don't see Records.json. When I try to import (any available files), I stuck at this command -> bundle exec rake import:big_file. with error message

Don't know how to build task 'import:big_file[Timelineedits.json,'

Version 0.8.6

To Reproduce Steps to reproduce the behavior:

  1. Go to Google Takeout
  2. Follow the export process the same on your wiki until step 9. There is no file named Records.json.
  3. Try to upload, copy, import any available json files (try one by one).
  4. None of them (json files) be able to run bundle exec rake import:big_file command.

Expected behavior I hope that there is at least one of them be able to import but it's not.

Screenshots No

Logs

Don't know how to build task 'import:big_file[Timelineedits.json,' (See the list of available tasks with rake --tasks) /var/app/vendor/bundle/ruby/3.2.0/gems/rake-13.2.1/lib/rake/task_manager.rb:59:in []' /var/app/vendor/bundle/ruby/3.2.0/gems/rake-13.2.1/lib/rake/application.rb:187:ininvoke_task'/var/app/vendor/bundle/ruby/3.2.0/gems/rake-13.2.1/lib/rake/application.rb:138:in block (2 levels) in top_level' /var/app/vendor/bundle/ruby/3.2.0/gems/rake-13.2.1/lib/rake/application.rb:138:ineach' /var/app/vendor/bundle/ruby/3.2.0/gems/rake-13.2.1/lib/rake/application.rb:138:in block in top_level' /var/app/vendor/bundle/ruby/3.2.0/gems/rake-13.2.1/lib/rake/application.rb:147:inrun_with_threads' /var/app/vendor/bundle/ruby/3.2.0/gems/rake-13.2.1/lib/rake/application.rb:132:in top_level' /var/app/vendor/bundle/ruby/3.2.0/gems/rake-13.2.1/lib/rake/application.rb:83:inblock in run'/var/app/vendor/bundle/ruby/3.2.0/gems/rake-13.2.1/lib/rake/application.rb:214:in standard_exception_handling' /var/app/vendor/bundle/ruby/3.2.0/gems/rake-13.2.1/lib/rake/application.rb:80:inrun' /var/app/vendor/bundle/ruby/3.2.0/gems/rake-13.2.1/exe/rake:27:in <top (required)>' /var/app/vendor/bundle/ruby/3.2.0/bin/rake:25:inload' /var/app/vendor/bundle/ruby/3.2.0/bin/rake:25:in <top (required)>' /usr/local/bundle/gems/bundler-2.5.9/lib/bundler/cli/exec.rb:58:inload' /usr/local/bundle/gems/bundler-2.5.9/lib/bundler/cli/exec.rb:58:in kernel_load' /usr/local/bundle/gems/bundler-2.5.9/lib/bundler/cli/exec.rb:23:inrun' /usr/local/bundle/gems/bundler-2.5.9/lib/bundler/cli.rb:451:in exec' /usr/local/bundle/gems/bundler-2.5.9/lib/bundler/vendor/thor/lib/thor/command.rb:28:inrun' /usr/local/bundle/gems/bundler-2.5.9/lib/bundler/vendor/thor/lib/thor/invocation.rb:127:in invoke_command' /usr/local/bundle/gems/bundler-2.5.9/lib/bundler/vendor/thor/lib/thor.rb:527:indispatch' /usr/local/bundle/gems/bundler-2.5.9/lib/bundler/cli.rb:34:in dispatch' /usr/local/bundle/gems/bundler-2.5.9/lib/bundler/vendor/thor/lib/thor/base.rb:584:instart' /usr/local/bundle/gems/bundler-2.5.9/lib/bundler/cli.rb:28:in start' /usr/local/bundle/gems/bundler-2.5.9/exe/bundle:28:inblock in <top (required)>' /usr/local/bundle/gems/bundler-2.5.9/lib/bundler/friendly_errors.rb:117:in with_friendly_errors' /usr/local/bundle/gems/bundler-2.5.9/exe/bundle:20:in<top (required)>' /usr/local/bundle/bin/bundle:25:in load' /usr/local/bundle/bin/bundle:25:in

'

Additional context No

*Reference 1 2 3

Freika commented 3 weeks ago

Hi @ve3! Can you show files and directory structure of your Takeout? Thanks

ve3 commented 3 weeks ago

Takeout |--ประวัติตำแหน่ง (ไทม์ไลน์) .....|--Encrypted Backups.txt .....|--Settings.json .....|--Timeline Edits.json .....|--Tombstones.csv |--archive_browser.html

When I copy file to docker, I renamed Timeline Edits.json to be Timelineedits.json.

ve3 commented 3 weeks ago

New Google time line use new way to export and here is instruction.

However, I'm currently stuck with their export failed 1, 2.

Freika commented 3 weeks ago

Takeout |--ประวัติตำแหน่ง (ไทม์ไลน์) .....|--Encrypted Backups.txt .....|--Settings.json .....|--Timeline Edits.json .....|--Tombstones.csv |--archive_browser.html

When I copy file to docker, I renamed Timeline Edits.json to be Timelineedits.json.

I'm not sure what can be imported from this Takeout, it seems basically empty to me. Are there any files bigger than several kilobytes?

ve3 commented 3 weeks ago

The new timeline is can't export from takeout anymore. It has been moved to user's phone (and in Google backup which can't import from that). To export from phone, please follow instruction here. But it may stuck at export failed.

tylwright commented 2 weeks ago

I can take this as an action item to create a wiki page on this. As of a few minor releases ago, the new "phone app dump" is supported as an import method. I should have the doc in an hour or so and will post a link here when it gets merged up!

tylwright commented 2 weeks ago

Looks like forking GitHub Wikis isn't really a thing. I have my proposed change. @Freika - Would you be okay granting me granting me wiki edit privs, if possible? Otherwise, I have my wiki addition here: https://github.com/tylwright/dawarich/wiki/How-to-import-your-Google-Maps-Timeline-data.

ve3 commented 2 weeks ago

I can import using web UI (imports page) now. Based on @tylwright URL and it is easier (different) than using command in wiki page.

It is currently running imports process (very long time now).

Please update wiki on official page. 🙂

However, I'm still stuck on export location on Android with export failed error but that is Google problem not dawarich.

tylwright commented 2 weeks ago

Thanks, @Freika!

Official wiki has been updated: https://github.com/Freika/dawarich/wiki/How-to-import-your-Google-Maps-Timeline-data

ve3 commented 2 weeks ago

The step to export/download timeline data on Android is on this link.

Maybe update to wiki?

I'll close this issue as it is be able to import with new Google timeline now.