search

hassonlab / 247-pickling

Contains code to create pickles from raw/processed data
1 stars 9 forks source link

failed create-sig-pickle for 777 (podcast) #105

Closed leoniekerken closed 1 year ago

leoniekerken commented 1 year ago

When trying to create-sig-pickle for 777 (podcast), several errors occurred:

1) In tfspkl_build_matrices.py, line 57: TypeError: read_csv() got an unexpected keyword argument 'columns' -> pd.read_csv does not take 'columns' as keyword argument, should be changed to 'names' ->https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.read_csv.html

2) In tfspkl_build_matrices.py, line 137: zip* fails to unpack values -> ValueError: not enough values to unpack (expected 2, got 0) (this is related to error 3)

3) There seems to be a mismatch between electrode_labels and electrode_names for subject 662

Screenshot 2022-12-06 at 16 07 12
hvgazula commented 1 year ago

I think I am still working on fixing this issue. Hopefully, this should be done tonight or by the meeting time tomorrow.

leoniekerken commented 1 year ago

Oh okay, great. Thanks! :)

hvgazula commented 1 year ago

If possible, could you also paste the command you ran? That will help reproduce the error. Thanks.

leoniekerken commented 1 year ago

'make create-sig-pickle' for podcast (777) gave me the first two errors. Then during debugging we inserted a breakpoint after line 137 and checked the contents of electrode_names and electrode_labels, this gave the following output:

Screenshot 2022-12-06 at 16 56 17
hvgazula commented 1 year ago

Hye @leoniekerken can you try doing the same with this https://github.com/hassonlab/247-pickling/tree/cf50fafda2c81573d3b8f788bf6f13065ac74890

leoniekerken commented 1 year ago

So I tried on the older branch and now make create-sig-pickle for podcast (777) works. However, I do get an error when running make create-pickle for podcast (661): File "/scratch/gpfs/ln1144/247-pickling/code/tfspkl_utils.py", line 191, in get_electrode_labels raise ValueError('Header File Missing') ValueError: Header File Missing

hvgazula commented 1 year ago

Wow..what a mess the code is :)

leoniekerken commented 1 year ago

Oh wait... that was my bad - had the wrong specifications in the makefile. Now I corrected it and it worked fine. Sorry!

hvgazula commented 1 year ago

Good. Thanks for trying this. Now, we know the source of the error. I will fix it as soon as possible.

leoniekerken commented 1 year ago

Thanks!

hvgazula commented 1 year ago

Resolved. Duplicates https://github.com/hassonlab/247-pickling/issues/97

hvgazula commented 1 year ago

@leoniekerken Did you generate any contextual embeddings (say gpt2), or did you stop with pickling? If yes, do you mind sharing the path to where your embeddings are so I can copy them over and rewrite the targets for copying embeddings?

hvgazula commented 1 year ago

Please ignore this request. I am doing this myself.