search

catalystneuro / tye-lab-to-nwb

NWB Conversion project for the Tye lab at the Salk Institute.
MIT License
0 stars 0 forks source link

error running convert_all_sessions.py script #47

Closed laurelrr closed 9 months ago

laurelrr commented 9 months ago

I have an excel file that contains the column nwbfile_path and have updated convert_all_session.py to read from that file. However, python convert_all_session.py produces this error:

  File "/nadata/snlkt/home/lkeyes/Projects/GIT/tye-lab-to-nwb/src/tye_lab_to_nwb/ast_ophys/convert_all_sessions.py", line 74, in <module>
    parallel_convert_sessions(
  File "/nadata/snlkt/home/lkeyes/Projects/GIT/tye-lab-to-nwb/src/tye_lab_to_nwb/ast_ophys/convert_all_sessions.py", line 32, in parallel_convert_sessions
    config = read_session_config(excel_file_path=excel_file_path)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nadata/snlkt/home/lkeyes/Projects/GIT/tye-lab-to-nwb/src/tye_lab_to_nwb/tools/read_session_config.py", line 14, in read_session_config
    assert "nwbfile_path" in config.columns, "The excel file does not contain the expected 'nwbfile_path' column."
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: The excel file does not contain the expected 'nwbfile_path' column.

However, inside a python terminal I can do the following and not generate the error:

>>> from tye_lab_to_nwb.tools import read_session_config, parallel_execute
>>> from pathlib import Path
>>> excel_file_path = Path("/snlkt/ast/ASt_NWB/NWB_Masters/calcium_ophys/ophys_master_excel_file_locations_20231211_D2.xlsx")
>>> config = read_session_config(excel_file_path=excel_file_path)
>>> config
                                        nwbfile_path                             miniscope_folder_path   ...  genotype subject_id
0  /snlkt/ast/ASt_NWB/NWB_Masters/calcium_ophys/n...  /snlkt/ast/Miniscope/expAnalysis/20220601_D2_D...  ...  drd2-cre       J642
1  /snlkt/ast/ASt_NWB/NWB_Masters/calcium_ophys/n...  /snlkt/ast/Miniscope/expAnalysis/20220601_D2_D...  ...  drd2-cre       J588
2  /snlkt/ast/ASt_NWB/NWB_Masters/calcium_ophys/n...  /snlkt/ast/Miniscope/expAnalysis/20220601_D2_D...  ...  drd2-cre       J650
3  /snlkt/ast/ASt_NWB/NWB_Masters/calcium_ophys/n...  /snlkt/ast/Miniscope/expAnalysis/20220601_D2_D...  ...  drd2-cre       J633
4  /snlkt/ast/ASt_NWB/NWB_Masters/calcium_ophys/n...  /snlkt/ast/Miniscope/expAnalysis/20220601_D2_D...  ...  drd2-cre       J634

[5 rows x 13 columns]
weiglszonja commented 9 months ago

There are a couple things we can double check here, first is whether tye_lab_to_nwb/ast_ophys/convert_all_sessions.py is really working with the correct excel file, we can try to call parallel_convert_sessions directly with that excel file:

>>> from tye_lab_to_nwb.ast_ophys.convert_all_sessions import parallel_convert_sessions
>>> excel_file_path = "/snlkt/ast/ASt_NWB/NWB_Masters/calcium_ophys/ophys_master_excel_file_locations_20231211_D2.xlsx"
>>> parallel_convert_sessions(excel_file_path=excel_file_path)

Also make sure the columns doesn't contain any spaces or tabs, but since it worked when directly calling read_session_config I think the file is fine.

laurelrr commented 9 months ago

I did have some trailing whitespace on a few of the column headers.
After removing them, it seems to be running, thank you!