Base tridesclous on spiketoolkit + try opencl

[samuelgarcia]

I try to make change for tridesclous. I don't known how to test.

[magland]

see working/test_sorters/test_sorters.py You can run one of these by doing, for example,

pytest -m tridesclous_magland_c8 -s

[magland]

@samuelgarcia . Let me know once you have tested and I will merge. Otherwise, do you want me to test?

[samuelgarcia]

@magland Hi Jeremy. I am not able to run test for download permission problems. I need to release tridesclous tomorow and spikeinterface just after.

When it will be done, could you run test ?

And if tests pass I think you can merge.

Thanks a lot.

[magland]

@samuelgarcia . Sorry about the download problems. Yes, I will test it once it is ready. (I will also see if I can fix the permissions issue)

[samuelgarcia]

@magland Hi Jeremy, spiketoolkit 0.3.2 have been released. Could you try to make the docker and run test of this new implementation SI based ?

best

Samuel

[magland]

Trying it now... pocl package not found for ubuntu 18.04. Do you have a workaround? I can try it without that package...

[samuelgarcia]

oups. Sorry it is in fact pocl-opencl-icd now.

[magland]

@samuelgarcia I built the singularity container and made some minor tweaks to get it to run using singularity container. It seemed to run on the test but I don't think it worked because it gave 0% accuracy. See the output below.

I pushed my changes to the samuelgarcia-tridesclous branch. The test command I ran was

cd working/test_sorters
pytest -m tridesclous_magland_c8 -s

If you want to run it outside the container (good for debugging), go into the test_sorters.py and change container='default' to container=None for that particular test.

(devel) magland@dub:~/src/spikeforest/working/test_sorters$ pytest -m tridesclous_magland_c8 -s
======================================= test session starts ========================================
platform linux -- Python 3.6.8, pytest-4.3.0, py-1.7.0, pluggy-0.8.1
rootdir: /home/magland/src/spikeforest, inifile: pytest.ini
collected 35 items / 34 deselected / 1 selected                                                    

test_sorters.py Job: Tridesclous (version: 0.2.2)
Timestamp: 2019-06-13 07:02:23
Realizing container file: sha1://07a59aaf83bcf2a2cf4b5e7fbd05aed784b34671/2019-06-13/tridesclous.simg
RUNNING SHELL SCRIPT: /home/magland/sha1-cache-new/tmp/tmp_execute_Tridesclouso_v8d7e3/singularity_run.sh
>> Running processor (class=Tridesclous) ...
>> Running Tridesclous...
>> Loading recording...
>> Running TridesclousSorter...
>> 'group' property is not available and it will not be saved.
========== Tridesclous exited with code 0 after 28.7817440032959 sec
================================================================================
Job: GenSortingComparisonTable (version: 0.2.6)
Timestamp: 2019-06-13 07:02:52
Realizing container file: sha1://0944f052e22de0f186bb6c5cb2814a71f118f2d1/spikeforest_basic.simg
RUNNING SHELL SCRIPT: /home/magland/sha1-cache-new/tmp/tmp_execute_GenSortingComparisonTabley4b5h827/singularity_run.sh
>> Running processor (class=GenSortingComparisonTable) ...
>> GenSortingComparisonTable: firings=/processor_inputs/firings.in, firings_true=/processor_inputs/firings_true.in, units_true=[]
========== GenSortingComparisonTable exited with code 0 after 1.3765480518341064 sec
================================================================================
Average accuracy: 0.0

[samuelgarcia]

Thank you I will have a look.

[samuelgarcia]

@magland I still have this

E           Exception: sha1 of downloaded file does not match expected http://kbucket.flatironinstitute.org/get/sha1/fb52d510d2543634e247e0d2d1d4390be9ed9e20 fb52d510d2543634e247e0d2d1d4390be9ed9e20

[samuelgarcia]

@magland I don't anderstand why I can't run the test inside my spikeforest fork.

In spiketoolkit==0.3.2 the tridesclous wrapper on datasets_noise10_K10_C4/001_synth give accuracy quite good for at least 8 units.

I don't anderstand.

[magland]

@samuelgarcia Okay, I'll work on this tomorrow. Could you give me an example script using spiketoolkit? Thanks

[samuelgarcia]

This script is running OK on my machine

import spiketoolkit as st
from spiketoolkit.comparison.groundtruthstudy import (setup_comparison_study, run_study_sorters,
            aggregate_sorting_comparison, aggregate_performances_table, get_recordings, get_ground_truths)

'sha1dir://fb52d510d2543634e247e0d2d1d4390be9ed9e20.synth_magland/datasets_noise10_K10_C4/001_synth',   #####

recording = SFMdaRecordingExtractor(dataset_directory=url, download=True)
sorting_gt = SFMdaSortingExtractor(firings_file=url + '/firings_true.mda') 
gt_dict = {rec_name : (recording, sorting_gt) }

sorter_list = ['tridesclous', 'moutainsort', ]

# run sorter
study_folder = 'test_tridesclous_on_datasets_noise10_K10_C4'
setup_comparison_study(study_folder, gt_dict)
st.run_study_sorters(study_folder, sorter_list)

# plot basics results
comparisons = aggregate_sorting_comparison(study_folder, exhaustive_gt=True)
dataframes = aggregate_performances_table(study_folder, exhaustive_gt=True)

for (rec_name, sorter_name), comp in comparisons.items():
    print()
    print(rec_name, sorter_name)
    print(comp.count)
    comp.print_summary()
    comp.plot_confusion_matrix()

plt.show()

[magland]

@samuelgarcia . I just confirmed that my unit test produces decent results when applying the new Tridesclous wrapper to the tetrode dataset

sha1dir://fb52d510d2543634e247e0d2d1d4390be9ed9e20.synth_magland/datasets_noise10_K10_C4/001_synth

When applying it to the 8-channel recording, it runs, but produces zero accuracy

sha1dir://fb52d510d2543634e247e0d2d1d4390be9ed9e20.synth_magland/datasets_noise10_K10_C8/001_synth

On the other hand, the old Tridesclous wrapper (the one used to generate the current table) produces a decent accuracy for both.

Could you try to reproduce that discrepancy using your script? I have a hard time running your script because I get:

AttributeError: module 'spikeextractors' has no attribute 'NpzSortingExtractor'

I am working on a new cleaned-up release of mountaintools and spikeforest packages. At that point I will push my latest wrappers.

[samuelgarcia]

Hi Jeremy. Thank you for this. I will have a look. If the wrapper work you can then safely merge it. I will update parameters later on on SI and TDC side to optimize results.

Sorry about this NpzSortingExtractor. It has been release with latest spikeextractors. It is needed somewhere in spiketoolkit but not in tridesclouswrapper. I think getting last version should fix.

[samuelgarcia]

I confim that results for tridesclous on datasets_noise10_K10_C8 is totally bad. I need to change paramerts again.

[magland]

Okay. You can also see website for latest numbers with the current wrapper.

On Friday, June 14, 2019, Garcia Samuel notifications@github.com wrote:

I confim that results for tridesclous on datasets_noise10_K10_C8 is totally bad. I need to change paramerts again.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/flatironinstitute/spikeforest/pull/36?email_source=notifications&email_token=AA4CIQAILB4WHGQSE543GILP2P7Y3A5CNFSM4HUGRS72YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODXX5T4Y#issuecomment-502258163, or mute the thread https://github.com/notifications/unsubscribe-auth/AA4CIQASA7UVL3WAMGEDURDP2P7Y3ANCNFSM4HUGRS7Q .

-- Sent from my phone

[samuelgarcia]

I close this one because you already put it in master. i will open another one.