code for filter_data broken?

akhanf commented 5 months ago

Got a bit farther after running without epoch extraction, but failing when running filter_data - looking at the script where it fails, it seems you are using a variable before defining it?

i.e. elec_edf here: https://github.com/mcespedes99/sEEGPrep/blob/8712f12013aa149c2e629139d3b6adbf6fe61d9f/clean_seeg/clean_seeg/edf_utils.py#L11

log below:

[Thu Jun  6 21:28:28 2024]
rule filter_data:
    input: work/sub-W2/ses-iemu/ieeg/sub-W2_ses-iemu_task-timit5_acq-B8_rec-dn_run-01_clip-01_ieeg.edf, /local/scratch/ds004993/sub-W2/ses-iemu/ieeg/sub-W2_ses-iemu_task-timit5_acq-B8_run-01_channels.tsv
    output: work/sub-W2/ses-iemu/ieeg/sub-W2_ses-iemu_task-timit5_acq-B8_rec-denoise_run-01_clip-01_ieeg.edf, work/sub-W2/ses-iemu/ieeg/sub-W2_ses-iemu_task-timit5_acq-B8_rec-denoise_run-01_clip-01_report.tsv, work/sub-W2/ses-iemu/ieeg/sub-W2_ses-iemu_task-timit5_acq-B8_rec-denoise_run-01_clip-01_report.json
    log: logs/sub-W2/ses-iemu/sub-W2_ses-iemu_task-timit5_acq-B8_run-01_clip-01_filtering.log
    jobid: 3
    benchmark: benchmark/sub-W2/ses-iemu/sub-W2_ses-iemu_task-timit5_acq-B8_run-01_clip-01_benchmarkFilter.txt
    reason: Missing output files: work/sub-W2/ses-iemu/ieeg/sub-W2_ses-iemu_task-timit5_acq-B8_rec-denoise_run-01_clip-01_ieeg.edf; Input files updated by another job: work/sub-W2/ses-iemu/ieeg/sub-W2_ses-iemu_task-timit5_acq-B8_rec-dn_run-01_clip-01_ieeg.edf
    wildcards: subject=W2, session=iemu, task=timit5, acq=B8, run=01, clip=01
    threads: 12
    resources: tmpdir=/tmp, mem_mb=16000, mem_mib=15259

/local/scratch/sEEGPrep/.venv_310/bin/python3 /local/scratch/sEEGPrep/seegprep/test_out_ds004993/.snakemake/scripts/tmp6z4x03o6.filter_signal.py
Traceback (most recent call last):
  File "/local/scratch/sEEGPrep/seegprep/test_out_ds004993/.snakemake/scripts/tmp6z4x03o6.filter_signal.py", line 50, in <module>
    main()
  File "/local/scratch/sEEGPrep/seegprep/test_out_ds004993/.snakemake/scripts/tmp6z4x03o6.filter_signal.py", line 36, in main
    _, df_report_filt, report_filt = seegTF.drift_correction(
  File "/local/scratch/sEEGPrep/.venv_310/lib/python3.10/site-packages/clean_seeg/clean_seeg.py", line 375, in drift_correction
    chn_labels, discarded_labels, signal = extract_signal(channels_tsv, self.edf_path)
  File "/local/scratch/sEEGPrep/.venv_310/lib/python3.10/site-packages/clean_seeg/edf_utils.py", line 11, in extract_signal
    chn_labels, discarded_labels = get_chn_labels(channels_tsv, elec_edf)
UnboundLocalError: local variable 'elec_edf' referenced before assignment
[Thu Jun  6 21:28:29 2024]
Error in rule filter_data:
    jobid: 3
    input: work/sub-W2/ses-iemu/ieeg/sub-W2_ses-iemu_task-timit5_acq-B8_rec-dn_run-01_clip-01_ieeg.edf, /local/scratch/ds004993/sub-W2/ses-iemu/ieeg/sub-W2_ses-iemu_task-timit5_acq-B8_run-01_channels.tsv
    output: work/sub-W2/ses-iemu/ieeg/sub-W2_ses-iemu_task-timit5_acq-B8_rec-denoise_run-01_clip-01_ieeg.edf, work/sub-W2/ses-iemu/ieeg/sub-W2_ses-iemu_task-timit5_acq-B8_rec-denoise_run-01_clip-01_report.tsv, work/sub-W2/ses-iemu/ieeg/sub-W2_ses-iemu_task-timit5_acq-B8_rec-denoise_run-01_clip-01_report.json
    log: logs/sub-W2/ses-iemu/sub-W2_ses-iemu_task-timit5_acq-B8_run-01_clip-01_filtering.log (check log file(s) for error details)

RuleException:
CalledProcessError in file /local/scratch/sEEGPrep/seegprep/workflow/rules/filter.smk, line 69:
Command 'set -euo pipefail;  /local/scratch/sEEGPrep/.venv_310/bin/python3 /local/scratch/sEEGPrep/seegprep/test_out_ds004993/.snakemake/scripts/tmp6z4x03o6.filter_signal.py' returned non-zero exit status 1.
  File "/local/scratch/sEEGPrep/seegprep/workflow/rules/filter.smk", line 69, in __rule_filter_data
  File "/usr/lib/python3.10/concurrent/futures/thread.py", line 58, in run
Shutting down, this might take some time.
Exiting because a job execution failed. Look above for error message
Complete log: .snakemake/log/2024-06-06T212818.251621.snakemake.log

mcespedes99 commented 5 months ago

Seems to be a dumb bug that I introduced in the last commit. Will fix tomorrow!

mcespedes99 commented 5 months ago

Just pushed the corresponding updates: https://github.com/mcespedes99/sEEGPrep/commit/3b4560a08c9d576a678685581d1e73b620b0ed47

mcespedes99 / sEEGPrep

code for filter_data broken? #21