When trying to create the cache for Sebastian's release tag I get the error below which seems to be due to the fact that none of his datasets are Session associated (they are all Subject aggregates), so the dataframe in this line is empty
We should probably change the code so that datasets that are not associated to a Session will be picked up too. I am unsure if that requires changes to the cache itself, or just to the code generating the cache
(venv) root@e0966ff6424f:/var/www/alyx/alyx# python manage.py one_cache -v 2 --compress --qc --tag 2023_Q4_Bruijns_et_al -D s3://ibl-brain-wide-map-public/caches/openalyx/2023_Q4_Bruijns_et_al
Traceback (most recent call last):
File "/var/www/alyx/venv/lib/python3.8/site-packages/pandas/core/indexes/base.py", line 3802, in get_loc
return self._engine.get_loc(casted_key)
File "pandas/_libs/index.pyx", line 138, in pandas._libs.index.IndexEngine.get_loc
File "pandas/_libs/index.pyx", line 165, in pandas._libs.index.IndexEngine.get_loc
File "pandas/_libs/hashtable_class_helper.pxi", line 5745, in pandas._libs.hashtable.PyObjectHashTable.get_item
File "pandas/_libs/hashtable_class_helper.pxi", line 5753, in pandas._libs.hashtable.PyObjectHashTable.get_item
KeyError: 'all_projects'
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "manage.py", line 10, in <module>
execute_from_command_line(sys.argv)
File "/var/www/alyx/venv/lib/python3.8/site-packages/django/core/management/__init__.py", line 446, in execute_from_command_line
utility.execute()
File "/var/www/alyx/venv/lib/python3.8/site-packages/django/core/management/__init__.py", line 440, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/var/www/alyx/venv/lib/python3.8/site-packages/django/core/management/base.py", line 402, in run_from_argv
self.execute(*args, **cmd_options)
File "/var/www/alyx/venv/lib/python3.8/site-packages/django/core/management/base.py", line 448, in execute
output = self.handle(*args, **options)
File "/var/www/alyx/alyx/misc/management/commands/one_cache.py", line 132, in handle
self.generate_tables(tables, export_qc=qc, tags=options.get('tag'))
File "/var/www/alyx/alyx/misc/management/commands/one_cache.py", line 150, in generate_tables
tbl, filename = self._save_table(generate_sessions_frame(**kwargs), table, dry=dry)
File "/var/www/alyx/alyx/misc/management/commands/one_cache.py", line 40, in wrapper
res = func(*arg, **kwargs)
File "/var/www/alyx/alyx/misc/management/commands/one_cache.py", line 331, in generate_sessions_frame
df['all_projects'] = df['all_projects'].map(lambda x: ','.join(filter(None, set(x))))
File "/var/www/alyx/venv/lib/python3.8/site-packages/pandas/core/frame.py", line 3807, in __getitem__
indexer = self.columns.get_loc(key)
File "/var/www/alyx/venv/lib/python3.8/site-packages/pandas/core/indexes/base.py", line 3804, in get_loc
raise KeyError(key) from err
KeyError: 'all_projects'
When trying to create the cache for Sebastian's release tag I get the error below which seems to be due to the fact that none of his datasets are Session associated (they are all Subject aggregates), so the dataframe in this line is empty
https://github.com/cortex-lab/alyx/blob/master/alyx/misc/management/commands/one_cache.py#L328
We should probably change the code so that datasets that are not associated to a Session will be picked up too. I am unsure if that requires changes to the cache itself, or just to the code generating the cache