Running MountainSort on my own data (.mda files converted from Neuralynx .ntt files)

[jstrohl]

Hello, I have installed the newest version on MountainSort (on Ubuntu LTS 12.04), and while it worked just fine with the example data, I am unable to run it on my own data. Our recordings were all in the form of .ntt files, and I have converted them to mda files in matlab, and then when I try to bring these into MountainSort, I have the following result and errors:

Using pipeline: mountainsort3.mlp Using data directory: /tmp/mountainlab/prvbucket Computing checksum for file: data/raw.mda child_process.js:507 throw err; ^

Error: Command failed: prv-stat data/raw.mda

at checkExecSyncError (child_process.js:464:13)
at Object.execSync (child_process.js:504:13)
at prv_stat_sync (/home/loinn/mountainlab/mlpipeline/utils/mlp/mlp-run:269:35)
at step2 (/home/loinn/mountainlab/mlpipeline/utils/mlp/mlp-run:165:12)
at /home/loinn/mountainlab/mlpipeline/utils/mlp/mlp-run:79:4
at /home/loinn/mountainlab/mlpipeline/web/mlpipeline/kuleleclient.js:325:4
at /home/loinn/mountainlab/mlpipeline/processing_server/larinet/larinetserver.js:260:4
at /home/loinn/mountainlab/mlpipeline/processing_server/larinet/larinetserver.js:544:5
at ChildProcess.<anonymous> (/home/loinn/mountainlab/mlpipeline/processing_server/larinet/larinetserver.js:713:3)
at emitTwo (events.js:87:13)

I am not sure if the problem I am having is from installing mountainsort improperly, or (more likely it seems) if it is from preparing my raw.mda file improperly.

[jstrohl]

Just a little more info about the .mda file: ~/mountainlab/packages/mountainsort/JoshData$ mda Data/raw.mda { "data_type": -3, "data_type_string": "float32", "dims": [ 4, 27352417 ], "header_size": 20, "num_bytes_per_entry": 4, "num_dims": 2 }

Finally, I am using the mountainsort3.mlp pipeline that comes in the pipelines folder

[jstrohl]

It seems I had a typo in my folder naming, I am closing this issue.

[macayladonegan]

Would you mind sharing how you convert ntt files to mdas? I'm running into some issues.

[magland]

Can you get the data into matlab or python? Then there are utilities to write out to .mda format.

[macayladonegan]

I think I figured it out. thanks!

On Fri, Apr 27, 2018 at 11:43 AM, Jeremy Magland notifications@github.com wrote:

Can you get the data into matlab or python? Then there are utilities to write out to .mda format.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/flatironinstitute/mountainsort/issues/12#issuecomment-385010052, or mute the thread https://github.com/notifications/unsubscribe-auth/AME434324wanBjpK1jwy2Q9No3baKM4Yks5tszyYgaJpZM4Q1iIp .

-- Macayla Donegan National Science Foundation Fellow Co-founder & Chair of the We the Scientists Board of Directors PhD Candidate in Neurobiology and Behavior Steve Siegelbaum Laboratory, Columbia University www.wethescientists.net

[macayladonegan]

Actually, I'm running into another hiccup. I have my neuralynx data as a 4x32xlength of the recording int32, but when I run write mda it gives me a 1x1 double with an integer value (ex: 48000, 32000, 44100)

On Fri, Apr 27, 2018 at 3:46 PM, macayla donegan macayladonegan@gmail.com wrote:

I think I figured it out. thanks!

On Fri, Apr 27, 2018 at 11:43 AM, Jeremy Magland <notifications@github.com

wrote:

Can you get the data into matlab or python? Then there are utilities to write out to .mda format.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/flatironinstitute/mountainsort/issues/12#issuecomment-385010052, or mute the thread https://github.com/notifications/unsubscribe-auth/AME434324wanBjpK1jwy2Q9No3baKM4Yks5tszyYgaJpZM4Q1iIp .

-- Macayla Donegan National Science Foundation Fellow Co-founder & Chair of the We the Scientists Board of Directors PhD Candidate in Neurobiology and Behavior Steve Siegelbaum Laboratory, Columbia University www.wethescientists.net

-- Macayla Donegan National Science Foundation Fellow Co-founder & Chair of the We the Scientists Board of Directors PhD Candidate in Neurobiology and Behavior Steve Siegelbaum Laboratory, Columbia University www.wethescientists.net

[magland]

Could you give some more detail on what exactly you are running and what is the result, more specifically?

[macayladonegan]

Here's the script for my open ephys data

clear tic LFPS=ls('CH.continuous'); for aa=1:length(LFPS) thisLFP=LFPS(aa,:); thisLFP=strtrim(thisLFP); [LFPdata(:,aa), LFPtimestamps, LFPinfo]=load_open_ephys_data_faster(thisLFP); end save LFPdata; save LFPtimestamps

%%split into mda files for each tetrode LFPdata=LFPdata'; %%for 8 tts with 4 channels-initialize params numTTs=8; numChannels=4;

mkdir('MDAs');cd('MDAs') for bb=1:numTTs thisLFP=LFPdata(4bb-3:4bb,:); thisLFP=int32(thisLFP); TTname=num2str(bb); thisdir=strcat('MDA',TTname);mkdir(thisdir);cd(thisdir) mdaname=strcat('raw','.mda'); writemda(thisLFP,mdaname,'int32')

%% add geom file to each folder
g=[0,0
    -25,25
    25,25
    0,50];
csvwrite('geom.csv',g)
%% copy param files into each folder
thisdir=pwd;
copyfile

C:\Users\macayla\Documents\GitHub\mountainsort_examples\examples\example1_mlp\params.json params.json copyfile C:\Users\macayla\Documents\GitHub\mountainsort_examples\examples\example1_mlp\mountainsort3.mlp mountainsort3.mlp cd .. end cd ..

toc

and here's the one for the neuralynx data clear tic TTs=ls('TT*.ntt'); mkdir('MDAs'); for aa=1:length(TTs) thisTT=TTs(aa,:); data=read_ntt_files(thisTT); TTname=num2str(aa); cd('MDAs') TTdata=complex32(data.waveforms); thisdir=strcat('MDA',TTname);mkdir(thisdir);cd(thisdir) mdaname=strcat('raw','.mda'); writemda(TTdata,mdaname,'int32')

%% add geom file to each folder
g=[0,0
    -25,25
    25,25
    0,50];
csvwrite('geom.csv',g)
%% copy param files into each folder
thisdir=pwd;
copyfile

C:\Users\macayla\Documents\GitHub\mountainsort_examples\examples\example1_mlp\params.json params.json copyfile C:\Users\macayla\Documents\GitHub\mountainsort_examples\examples\example1_mlp\mountainsort3.mlp mountainsort3.mlp cd ..;cd ..

end cd ..

toc

I'm having the same issue with both.

On Wed, May 9, 2018 at 10:19 AM, Jeremy Magland notifications@github.com wrote:

Reopened #12 https://github.com/flatironinstitute/mountainsort/issues/12 .

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/flatironinstitute/mountainsort/issues/12#event-1617741402, or mute the thread https://github.com/notifications/unsubscribe-auth/AME43z5FYhQVLFjegKCW3srSTnTpHRQCks5twvsNgaJpZM4Q1iIp .

-- Macayla Donegan National Science Foundation Fellow Co-founder & Chair of the We the Scientists Board of Directors PhD Candidate in Neurobiology and Behavior Steve Siegelbaum Laboratory, Columbia University www.wethescientists.net

[magland]

It's difficult for me to answer that right now. Are you on the slack site? If not, send me an email and I will invite you.