int-brain-lab / iblenv

Unified environment and Issue tracker for all IBL
MIT License
10 stars 10 forks source link

Problem doing ALF conversion on phy sorted files which have merged or split clusters #308

Open antaramaj16 opened 2 years ago

antaramaj16 commented 2 years ago

For ephys sessions that have been phy curated and there are spikes either merged or split - while converting into ALF files you may run into this error: image

How exactly do we resolve this because a lot of our sessions for analysis have already been phy sorted and normally it doesn't give us problems if we don't merge or split clusters?

oliche commented 2 years ago

Hi, Yes it is probable since exporting after manual curation is not something we are super familiar with. Would you mind sharing the dataset with us so we can reproduce this locally ?

If you do not want to post the dataset publicly, you can contact me by email or at support@internationatlbrainlab.org

antaramaj16 commented 2 years ago

Hi,

Please find attached the dataset for a session. Thank you!

Sincerely, Antara amplitudes.npy https://drive.google.com/file/d/1rG7tFTWHUJllMziL7Hb7kVKZ7Im3UZEO/view?usp=drive_web channel_map.npy https://drive.google.com/file/d/15gLPJ70YCY0gFQqhBdV_kKqHz9XkC_7-/view?usp=drive_web channel_positions.npy https://drive.google.com/file/d/1slz5e3oVdA64qqPYQnm22er5YG8OdBzh/view?usp=drive_web cluster_Amplitude.tsv https://drive.google.com/file/d/1FQi1PI3oD1wGD_eC2MblJzThCmDFOiTZ/view?usp=drive_web cluster_ContamPct.tsv https://drive.google.com/file/d/1tQ-k8_nWCBnGZy4pUZnEO5kZGHDwQk0m/view?usp=drive_web cluster_group.tsv https://drive.google.com/file/d/1Fvg0lOTm5JhtYCObJCKQ5ko0av86TXaZ/view?usp=drive_web cluster_info.tsv https://drive.google.com/file/d/12LXsBh1HPTXHmIR6P5KPn3qDCLLM0wAT/view?usp=drive_web cluster_KSLabel.tsv https://drive.google.com/file/d/1XmQ_LO16bQimNSRvnuCzu6Uw35mRz1sx/view?usp=drive_web params.py https://drive.google.com/file/d/1I9gZuxjl_T4N8e0pX6GFn-GrAhiJAA4Q/view?usp=drive_web phy.log https://drive.google.com/file/d/1XnXZCC4XtEmDaxNFpr-Dtl9tmk_xzz0_/view?usp=drive_web rez.mat https://drive.google.com/file/d/11k3b9YuF238gGjhn_0PvDUWOIMU-9N3r/view?usp=drive_web similar_templates.npy https://drive.google.com/file/d/1LLJ3UFwDH7OfeHrLMA0Upi6TK-y2r7P7/view?usp=drive_web spike_clusters.npy https://drive.google.com/file/d/1JDgeb-A39dJ8VfdFYu__bChWWeEagRAP/view?usp=drive_web spike_templates.npy https://drive.google.com/file/d/1DC8ZuVfyEtBy0VGVZ6--rYB-m_dmnMjK/view?usp=drive_web spike_times.npy https://drive.google.com/file/d/1JZ7MMVIDZ7jQJw7bb3d8C0FSZ1kFem7n/view?usp=drive_web templates.npy https://drive.google.com/file/d/1JZQ1XkkcsbzqxBkOXhbjsnxXCfAPZzoU/view?usp=drive_web templates_ind.npy https://drive.google.com/file/d/1nQmFoLNpZPiEOG-EBLdwxOo1jlfMQOwd/view?usp=drive_web whitening_mat.npy https://drive.google.com/file/d/10Pdj2LxK5WsSFu1X18GnBCz-YaDBpKZz/view?usp=drive_web whitening_mat_inv.npy https://drive.google.com/file/d/1EC3zKVck4JV1X_qh4oWBjGa4zs7xLJem/view?usp=drive_web

On Thu, Jun 2, 2022 at 8:12 AM Olivier Winter @.***> wrote:

Hi, Yes it is probable since exporting after manual curation is not something we are super familiar with. Would you mind sharing the dataset with us so we can reproduce this locally ?

If you do not want to post the dataset publicly, you can contact me by email or at @.***

— Reply to this email directly, view it on GitHub https://github.com/int-brain-lab/iblenv/issues/308#issuecomment-1144525601, or unsubscribe https://github.com/notifications/unsubscribe-auth/ARV7RIU5TK7GCU4UKIO7KGTVNBND7ANCNFSM5WI7F6YA . You are receiving this because you authored the thread.Message ID: @.***>

--

_____

Antara Majumdar

DPhil Physiology, Anatomy and Genetics | Kellogg College

University of Oxford | Lak Lab Department of Physiology, Anatomy and Genetics Sherrington Building, Sherrington Rd, Oxford OX1 3PT Tel: +44 07709252462

antaramaj16 commented 2 years ago

Here are the raw files:

2022-05-06_AMR019_g0_t0.imec0.lf.bin https://drive.google.com/file/d/18JLlyIse9rJJzbPx1u8du9oHgTJshlpV/view?usp=drive_web 2022-05-06_AMR019_g0_t0.imec0.lf.meta https://drive.google.com/file/d/1z7yGkgrSbHCM-_bsOT-lvG-nSooaW3rZ/view?usp=drive_web 2022-05-06_AMR019_g0_t0.nidq.bin https://drive.google.com/file/d/1Im1IMgznuG8rX6FAfMoSnpS_2hTNGg7R/view?usp=drive_web 2022-05-06_AMR019_g0_t0.nidq.meta https://drive.google.com/file/d/1cAOfiIn8s3Q_1Si3qfO41qqHravW--wH/view?usp=drive_web 2022-05-06_AMR019_g0_tcat.imec0.ap.bin https://drive.google.com/file/d/1akcPg8RkGPC57teMLVoAMA_vh5ifAm9U/view?usp=drive_web 2022-05-06_AMR019_g0_tcat.imec0.ap.meta https://drive.google.com/file/d/1-Z1xzKrEgfE4jP_k5BX2dA0matARgmTA/view?usp=drive_web 2022-05-06_AMR019_g0_tcat.imec0.ap.SY_384_6_500... https://drive.google.com/file/d/1TZofXCeJ0mgOhpCUXUDzrDMiEkFsgs8H/view?usp=drive_web 2022-05-06_AMR019_g0_tcat.nidq.meta https://drive.google.com/file/d/1yn1JDk5ODvkjya1h-DOD7CQNWfcs6T9p/view?usp=drive_web 2022-05-06_AMR019_g0_tcat.nidq.XA_0_0.txt https://drive.google.com/file/d/1UkNW9f4Wiyl0Eert4IZ0mfMAZEzMk3cG/view?usp=drive_web 2022-05-06_AMR019_g0_tcat.nidq.XA_1_500.txt https://drive.google.com/file/d/1CymsBh_X2ZMXY_2OvLpw1w7VDoOIRy-E/view?usp=drive_web

On Fri, Jun 3, 2022 at 10:59 AM Antara Majumdar @.***> wrote:

Hi,

Please find attached the dataset for a session. Thank you!

Sincerely, Antara amplitudes.npy https://drive.google.com/file/d/1rG7tFTWHUJllMziL7Hb7kVKZ7Im3UZEO/view?usp=drive_web channel_map.npy https://drive.google.com/file/d/15gLPJ70YCY0gFQqhBdV_kKqHz9XkC_7-/view?usp=drive_web channel_positions.npy https://drive.google.com/file/d/1slz5e3oVdA64qqPYQnm22er5YG8OdBzh/view?usp=drive_web cluster_Amplitude.tsv https://drive.google.com/file/d/1FQi1PI3oD1wGD_eC2MblJzThCmDFOiTZ/view?usp=drive_web cluster_ContamPct.tsv https://drive.google.com/file/d/1tQ-k8_nWCBnGZy4pUZnEO5kZGHDwQk0m/view?usp=drive_web cluster_group.tsv https://drive.google.com/file/d/1Fvg0lOTm5JhtYCObJCKQ5ko0av86TXaZ/view?usp=drive_web cluster_info.tsv https://drive.google.com/file/d/12LXsBh1HPTXHmIR6P5KPn3qDCLLM0wAT/view?usp=drive_web cluster_KSLabel.tsv https://drive.google.com/file/d/1XmQ_LO16bQimNSRvnuCzu6Uw35mRz1sx/view?usp=drive_web params.py https://drive.google.com/file/d/1I9gZuxjl_T4N8e0pX6GFn-GrAhiJAA4Q/view?usp=drive_web phy.log https://drive.google.com/file/d/1XnXZCC4XtEmDaxNFpr-Dtl9tmk_xzz0_/view?usp=drive_web rez.mat https://drive.google.com/file/d/11k3b9YuF238gGjhn_0PvDUWOIMU-9N3r/view?usp=drive_web similar_templates.npy https://drive.google.com/file/d/1LLJ3UFwDH7OfeHrLMA0Upi6TK-y2r7P7/view?usp=drive_web spike_clusters.npy https://drive.google.com/file/d/1JDgeb-A39dJ8VfdFYu__bChWWeEagRAP/view?usp=drive_web spike_templates.npy https://drive.google.com/file/d/1DC8ZuVfyEtBy0VGVZ6--rYB-m_dmnMjK/view?usp=drive_web spike_times.npy https://drive.google.com/file/d/1JZ7MMVIDZ7jQJw7bb3d8C0FSZ1kFem7n/view?usp=drive_web templates.npy https://drive.google.com/file/d/1JZQ1XkkcsbzqxBkOXhbjsnxXCfAPZzoU/view?usp=drive_web templates_ind.npy https://drive.google.com/file/d/1nQmFoLNpZPiEOG-EBLdwxOo1jlfMQOwd/view?usp=drive_web whitening_mat.npy https://drive.google.com/file/d/10Pdj2LxK5WsSFu1X18GnBCz-YaDBpKZz/view?usp=drive_web whitening_mat_inv.npy https://drive.google.com/file/d/1EC3zKVck4JV1X_qh4oWBjGa4zs7xLJem/view?usp=drive_web

On Thu, Jun 2, 2022 at 8:12 AM Olivier Winter @.***> wrote:

Hi, Yes it is probable since exporting after manual curation is not something we are super familiar with. Would you mind sharing the dataset with us so we can reproduce this locally ?

If you do not want to post the dataset publicly, you can contact me by email or at @.***

— Reply to this email directly, view it on GitHub https://github.com/int-brain-lab/iblenv/issues/308#issuecomment-1144525601, or unsubscribe https://github.com/notifications/unsubscribe-auth/ARV7RIU5TK7GCU4UKIO7KGTVNBND7ANCNFSM5WI7F6YA . You are receiving this because you authored the thread.Message ID: @.***>

--

_____

Antara Majumdar

DPhil Physiology, Anatomy and Genetics | Kellogg College

University of Oxford | Lak Lab Department of Physiology, Anatomy and Genetics Sherrington Building, Sherrington Rd, Oxford OX1 3PT Tel: +44 07709252462

--

_____

Antara Majumdar

DPhil Physiology, Anatomy and Genetics | Kellogg College

University of Oxford | Lak Lab Department of Physiology, Anatomy and Genetics Sherrington Building, Sherrington Rd, Oxford OX1 3PT Tel: +44 07709252462

antaramaj16 commented 2 years ago

Here is a folder with the dataset in case it is hard to access this: https://drive.google.com/drive/folders/15fkZI244jwedu6KOB5krK1qu8D02Opz5?usp=sharing