'N.' in start time in example .kwik file (default?)

[lepmik]

Hi, dont know where this issue belong so i posted it here as well. In the test_hybrid_120sec.kwik recordings/0 start_time = N., is this default when no value is given? could this be zero in stead e.g. start_time = 0? The reason I ask is that I write an IO for NEO which will take both .kwik from open ephys and phy.

[rossant]

I think there must be a bug somewhere where this value is NaN or None and this is converted to N in HDF5. The problem might have disappeared in the phy rewrite. Do you see this problem on files created with phy too?

[lepmik]

Can you send me a working .prm file, tried two different:

1: https://github.com/klusta-team/kwiklib/blob/master/kwiklib/utils/params_default.py

phy spikesort params.prm 
Traceback (most recent call last):
  File "/home/mikkel/.virtualenvs/kwikteam/bin/phy", line 9, in <module>
    load_entry_point('phy==0.2.0.dev0', 'console_scripts', 'phy')()
  File "/home/mikkel/apps/KwikTeam/phy/phy/scripts/phy_script.py", line 384, in main
    out = func(args)
  File "/home/mikkel/apps/KwikTeam/phy/phy/scripts/phy_script.py", line 313, in spikesort
    kwik_path=kwik_path)
  File "/home/mikkel/apps/KwikTeam/phy/phy/io/kwik/creator.py", line 305, in create_kwik
    prm = _read_python(prm_file) if prm_file else {}
  File "/home/mikkel/apps/KwikTeam/phy/phy/utils/_misc.py", line 132, in _read_python
    exec_(contents, {}, metadata)
  File "/home/mikkel/apps/KwikTeam/phy/phy/ext/six.py", line 638, in exec_
    exec("""exec _code_ in _globs_, _locs_""")
  File "<string>", line 1, in <module>
  File "<string>", line 3, in <module>
NameError: name 'sample_rate' is not defined

2: https://drive.google.com/folderview?id=0BwTrbfNJNihcaHZjTXEwdk44cFE&usp=sharing

phy spikesort params.prm 
Traceback (most recent call last):
  File "/home/mikkel/.virtualenvs/kwikteam/bin/phy", line 9, in <module>
    load_entry_point('phy==0.2.0.dev0', 'console_scripts', 'phy')()
  File "/home/mikkel/apps/KwikTeam/phy/phy/scripts/phy_script.py", line 384, in main
    out = func(args)
  File "/home/mikkel/apps/KwikTeam/phy/phy/scripts/phy_script.py", line 313, in spikesort
    kwik_path=kwik_path)
  File "/home/mikkel/apps/KwikTeam/phy/phy/io/kwik/creator.py", line 308, in create_kwik
    assert 'spikedetekt' in prm
AssertionError

[rossant]

we have changed slightly the PRM format

here is an example

experiment_name = 'hybrid_10sec'
prb_file = '1x32_buzsaki'

traces = dict(
    raw_data_files=[experiment_name + '.dat'],
    voltage_gain=10.,
    sample_rate=20000,
    n_channels=32,
)

spikedetekt = dict(
    filter_low=500.,  # Low pass frequency (Hz)
    filter_high_factor=0.95 * .5,
    filter_butter_order=3,  # Order of Butterworth filter.

    filter_lfp_low=0,  # LFP filter low-pass frequency
    filter_lfp_high=300,  # LFP filter high-pass frequency

    chunk_size_seconds=1,
    chunk_overlap_seconds=.015,

    n_excerpts=50,
    excerpt_size_seconds=1,
    threshold_strong_std_factor=4.5,
    threshold_weak_std_factor=2.,
    use_single_threshold=False,
    detect_spikes='negative',

    connected_component_join_size=1,

    extract_s_before=16,
    extract_s_after=16,

    n_features_per_channel=3,  # Number of features per channel.
    pca_n_waveforms_max=10000,
)

[lepmik]

Now I get

phy spikesort params.prm --overwrite
11:29:59 [W] The `dtype` parameter is mandatory. Using a default value of `int16` for now. Please update your `.prm` file.
Traceback (most recent call last):
  File "/home/mikkel/.virtualenvs/kwikteam/bin/phy", line 9, in <module>
    load_entry_point('phy==0.2.0.dev0', 'console_scripts', 'phy')()
  File "/home/mikkel/apps/KwikTeam/phy/phy/scripts/phy_script.py", line 384, in main
    out = func(args)
  File "/home/mikkel/apps/KwikTeam/phy/phy/scripts/phy_script.py", line 313, in spikesort
    kwik_path=kwik_path)
  File "/home/mikkel/apps/KwikTeam/phy/phy/io/kwik/creator.py", line 377, in create_kwik
    dtype=dtype,
  File "/home/mikkel/apps/KwikTeam/phy/phy/io/kwik/creator.py", line 228, in add_recordings_from_dat
    assert op.splitext(filename)[1] == '.dat'
AssertionError

with .prm file

experiment_name = 'test_hybrid_120sec'
prb_file = '1x32_buzsaki'

traces = dict(
    raw_data_files=[experiment_name + '.raw.kwd'],
...

[rossant]

can you remove the brackets in raw_data_files?

[lepmik]

11:57:58 [I] Saving a backup of the Kwik file in /home/mikkel/apps/KwikTeam/klusta-example/test_hybrid_120sec.kwik.bak.
11:57:58 [I] Finding the thresholds...
Traceback (most recent call last):
  File "/home/mikkel/.virtualenvs/kwikteam/bin/phy", line 9, in <module>
    load_entry_point('phy==0.2.0.dev0', 'console_scripts', 'phy')()
  File "/home/mikkel/apps/KwikTeam/phy/phy/scripts/phy_script.py", line 409, in main
    exec_(cmd, {}, ns)
  File "/home/mikkel/apps/KwikTeam/phy/phy/ext/six.py", line 638, in exec_
    exec("""exec _code_ in _globs_, _locs_""")
  File "<string>", line 1, in <module>
  File "<string>", line 1, in <module>
  File "/home/mikkel/apps/KwikTeam/phy/phy/cluster/session.py", line 318, in detect
    out = sd.run_serial(traces, interval_samples=interval_samples)
  File "/home/mikkel/apps/KwikTeam/phy/phy/cluster/algorithms.py", line 656, in run_serial
    thresholds=thresholds,
  File "/home/mikkel/apps/KwikTeam/phy/phy/cluster/algorithms.py", line 474, in step_detect
    dead_channels=self._dead_channels
  File "/home/mikkel/apps/KwikTeam/phy/phy/cluster/algorithms.py", line 258, in detect
    weak = thresholder.detect(traces_t, 'weak')
  File "/home/mikkel/apps/KwikTeam/phy/phy/traces/detect.py", line 102, in detect
    threshold = float(threshold)
TypeError: only length-1 arrays can be converted to Python scalars

[rossant]

my bad, two options:

• use use_single_threshold=True
• otherwise keep the parameter to False, but use this PR

[lepmik]

ok, now the attributes are good, thanks