Numpy error - numpy.bytes

[pdmadeira]

Hey, I tried to use REMoDNaV for the first time, running the example command in the terminal:

Command: remodnav “input.tsv” “trial5_events.tsv” 0.028847918928415837 1000.0

adapted to my project, but I get an error (stated below). Can you give me any help?

File "c:\users\...\remodnav\__init__.py", line 145, in main
    pp = clf.preproc(
  File "c:\users\...\remodnav\clf.py", line 813, in preproc
    data = filter_spikes(data)
  File "c:\users\...\remodnav\clf.py", line 165, in filter_spikes
    data['x'] = _filter(data['x'])
  File "c:\users\...\remodnav\clf.py", line 157, in _filter
    prev_dist = abs(arr[i - 1] - arr[i])
TypeError: unsupported operand type(s) for -: 'numpy.bytes_' and 'numpy.bytes_'

Please find attached my input file (.txt format because GitHub doesn't support .tsv): input.txt

[adswa]

Hi, thanks for a report that includes example data! This is helpful. :)

As far as I can see, the error originates because the data is being read in as strings. This is due to the fact that your input data contains a header (x_pos, y_pos), but we don't expect a header. When remodnav reads in your data, it sets a header itself:

    data = np.recfromcsv(
        args.infile,
        delimiter='\t',
        names=['x', 'y'],
        usecols=[0, 1])

As a result, your data contains a mix of data types, which then all get interpreted as strings. When I remove the header from your input file, remodnav runs fine. Can you try if this solves your issue?

[pdmadeira]

Yep, it works fine now! I think I clearly missed the Example Usage part when you say "(no header)". Great explanation from you, thank you!

And by the way, thanks for all the work put together in the development of this tool, it will be a huge help in my research!

[adswa]

Cool, glad it works! :)

[pdmadeira]

Any suggestion on the most efficient method to apply the command to multiple files in a folder, generating different output files?

[adswa]

You could write a bash wrapper script that calls the command in a loop. An example is here: https://github.com/psychoinformatics-de/studyforrest-data-eyemovementlabels/blob/master/code/compute_all.sh. You could also turn to job scheduling/batch processing when you're doing this on a computational cluster and do not want to sequentially call the command, but parallelize.

[jliebers]

To anybody in the future encountering this issue, in my file I had the following line somewhere in the middle causing these troubles which came from pandas.to_csv(): -1,29E+01 0.1144016

CTRL+F your file for some comma :)