search

martin-ueding / geo-activity-playground

Data analysis and visualization based on GPS tracked outdoor activities.
https://martin-ueding.github.io/geo-activity-playground/
MIT License
35 stars 14 forks source link

Server does not start after fresh import from Strava: KeyError: 'consider_for_achievements' #136

Closed p3dda closed 3 months ago

p3dda commented 3 months ago

After a fresh import of Strava exported data, the server does not start. The relevant cache migration seems to be applied before import of activities, and after import, it crashes. It seems, the relevant key is not set during import.

I am running version 0.24.2

Full log below:

2024-08-05 16:28:04 geo_activity_playground.core.cache_migrations INFO Apply cache migration if needed …
2024-08-05 16:28:04 geo_activity_playground.core.cache_migrations INFO Applying cache migration delete_activities_per_tile …
2024-08-05 16:28:04 geo_activity_playground.core.cache_migrations INFO Applying cache migration reset_time_series_embellishment …
2024-08-05 16:28:04 geo_activity_playground.core.cache_migrations INFO Applying cache migration delete_tile_visits …
2024-08-05 16:28:04 geo_activity_playground.core.cache_migrations INFO Applying cache migration delete_heatmap_cache …
2024-08-05 16:28:04 geo_activity_playground.core.cache_migrations INFO Applying cache migration delete_activity_metadata …
2024-08-05 16:28:04 geo_activity_playground.core.cache_migrations INFO Applying cache migration delete_activity_metadata …
2024-08-05 16:28:04 geo_activity_playground.core.cache_migrations INFO Applying cache migration convert_distances_to_km …
Convert m to km: 0it [00:00, ?it/s]
2024-08-05 16:28:04 geo_activity_playground.core.cache_migrations INFO Applying cache migration delete_activity_metadata …
2024-08-05 16:28:04 geo_activity_playground.core.cache_migrations INFO Applying cache migration delete_tile_visits …
2024-08-05 16:28:04 geo_activity_playground.core.cache_migrations INFO Applying cache migration delete_heatmap_cache …
2024-08-05 16:28:04 geo_activity_playground.core.cache_migrations INFO Applying cache migration add_consider_for_achievements …
2024-08-05 16:28:04 geo_activity_playground.core.cache_migrations INFO Applying cache migration delete_tile_visits …
2024-08-05 16:28:04 geo_activity_playground.core.cache_migrations INFO Applying cache migration delete_heatmap_cache …
2024-08-05 16:28:04 geo_activity_playground.core.cache_migrations INFO Applying cache migration delete_tile_visits …
2024-08-05 16:28:04 geo_activity_playground.core.cache_migrations INFO Applying cache migration delete_everything …
2024-08-05 16:28:04 geo_activity_playground.core.cache_migrations INFO Applying cache migration delete_activity_metadata …
Import from Strava export:   0%|          | 0/1 [00:00<?, ?it/s]2024-08-05 16:28:0
Import from Strava export: 100%|██████████| 1/1 [00:00<00:00,  2.83it/s]
Embellish time series data: 100%|██████████| 1/1 [00:00<00:00, 339.56it/s]
Extract explorer tile visits:   0%|          | 0/1 [00:00<?, ?it/s]
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/pandas/core/indexes/base.py", line 3805, in get_loc
    return self._engine.get_loc(casted_key)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "index.pyx", line 167, in pandas._libs.index.IndexEngine.get_loc
  File "index.pyx", line 196, in pandas._libs.index.IndexEngine.get_loc
  File "pandas/_libs/hashtable_class_helper.pxi", line 7081, in pandas._libs.hashtable.PyObjectHashTable.get_item
  File "pandas/_libs/hashtable_class_helper.pxi", line 7089, in pandas._libs.hashtable.PyObjectHashTable.get_item
KeyError: 'consider_for_achievements'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/app/geo_activity_playground/__main__.py", line 118, in <module>
    main()
  File "/app/geo_activity_playground/__main__.py", line 93, in main
    options.func(options)
  File "/app/geo_activity_playground/__main__.py", line 67, in <lambda>
    *make_activity_repository(options.basedir, options.skip_strava),
     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/geo_activity_playground/__main__.py", line 112, in make_activity_repository
    scan_for_activities(repository, tile_visit_accessor, config, skip_strava)
  File "/app/geo_activity_playground/webui/upload/controller.py", line 114, in scan_for_activities
    compute_tile_visits(repository, tile_visit_accessor)
  File "/app/geo_activity_playground/explorer/tile_visits.py", line 83, in compute_tile_visits
    if activity["consider_for_achievements"]:
       ~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/pandas/core/series.py", line 1121, in __getitem__
    return self._get_value(key)
           ^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/pandas/core/series.py", line 1237, in _get_value
    loc = self.index.get_loc(label)
          ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/pandas/core/indexes/base.py", line 3812, in get_loc
    raise KeyError(key) from err
KeyError: 'consider_for_achievements'
martin-ueding commented 3 months ago

Thanks for the pull request! Unfortunately I have refactored the code and haven't pushed it. There is now a separate layer between the raw extracted activities and the enriched ones. I now apply default values to all columns such that this shouldn't happen again.

p3dda commented 3 months ago

Your latest change from a41e508 leads to a new error:

  File "/app/geo_activity_playground/importers/strava_checkout.py", line 164, in <listcomp>
    if not (activity_extracted_meta_dir() / f"{activity_id.pickle}").exists()
                                               ^^^^^^^^^^^^^^^^^^
AttributeError: 'int' object has no attribute 'pickle'

Should probably be f"{activity_id}.pickle"?

Further errors are in the calls of activity_extracted_time_series_dir / activity_extracted_meta_dir methods. Fixed the import in PR #139