Warning: KS2.5 default output directory same as working directory and may delete primary Open-Ephys .npy files.

[linussun]

This issue is a warning to anyone who expects .npy files in the output folder to be preserved (e.g. Open Ephys users). To avoid this be sure to change the default output directory to an EMPTY directory when using the KS2.5 GUI. I can understand if the developers don't wish to change behavior of how data is written to the output directory. However, the current default behavior of the GUI may lead to a rude awaking to OEP users who will lose primary data.

Steps to reproduce

1. Installed software: Windows 10, Open Ephys 0.5.2, Kilosort 2.5
2. Open Ephys: record a session of FPGA data stored with recording note in binary format.
3. Launch KS2.5 GUI, click on "select data file" and navigate to file: "RecordNode###\experiment1\recording1\continuous\Rhythm_FPGA-101.0\continuous.dat"
4. Click Run All.

Current behaviour (bug)

1. KS2.5 GUI will default set "working directory" and "results output directory" to the same folder as the continuous.dat folder: "RecordNode###\experiment1\recording1\continuous\Rhythm_FPGA-101.0\"
2. Open Ephys raw data includes two .npy files: timestamps.npy and synchronized_timestamps.npy
3. Both these files are deleted destructively when executing preprocessing/Run All. They are not able to be recovered and may make raw data set unusable unless timestamp .npy files can be recreated.

Expected behaviour The OEP timestamps.npy, synchronized_timestamps.npy or really any .npy files that are not generated by KS2.5 are left alone. I assume a 'rm *.npy' command is being run in the processing stage by Kilosort 2.5.

Proposed solutions:

1. Simple: Only delete .npy files that are expected to be generated by KS2.5
2. Simple: Store all .npy generated files into a ./output_YYMMDDHHMM date-time named folder.
3. Less simple: Offer warning when .npy files do exist and will be over written.
4. Less simple but nicest: move all .npy files that will be destroyed into a ./prevKSoutput folder that is date-time stamped.
5. No code update, but update documentation warning users of setting the output directory to an empty folder (i.e.), or warn users that if they wish to save analysis of data, not to store them as .npy files in the output directory otherwise warn them to not re-run KS2.5 (see solution 3).

[marius10p]

Not the right repository to post in, but thanks for bringing this up. I think we'll make a "kilosort" subfolder.

[linussun]

It looks like this was fixed in 3.0. I also added the mkdir('kilosort25') line and appended this directory to rootZ before rez2Phy in main_kilosort.m.

Thank you.