Closed petersenpeter closed 3 years ago
That sounds good. We can use such structure to store for example wavelets and other types of lfp channel analysis outputs. How are you doing it now DanL?
On Feb 28, 2019, at 6:10 PM, Peter Petersen notifications@github.com wrote:
I am in need of a standardization of channel level data in buzcode. To follow the nomenclature of cellInfo and sessionInfo, I propose a .channelInfo.mat files that contains channel specific metrics such as impedance measures, average ripple across channels and so fourth.
I propose the mat file should contain a structure that includes list of channels: .channels, and how the data was derived similar to how the FindRipples have the fields .detectorName and .detectorParams
Do you have any input on this? For it or do you have a better solution?
— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/buzsakilab/buzcode/issues/300, or mute the thread https://github.com/notifications/unsubscribe-auth/AM8oQ1BCxwye8ufLkWXHetAiJ8EBEv5_ks5vSGHpgaJpZM4bX7gu.
A channelInfo.mat filetype sounds very useful to me! I don’t have anything like that right now, but can think of quite a few things that would fit in that box as well… for example a population spike-phase coupling depth profile or CSD depth profile used for laminar detection.
Want to start a section in the git wiki and we can update/add conventional fields as they seem necessary? We should make a load function similar to bz_LoadCellinfo and each detector should make its own channelInfo.mat file that follows the conventions.
Dan
On Feb 28, 2019, at 6:53 PM, AntonioFR notifications@github.com wrote:
That sounds good. We can use such structure to store for example wavelets and other types of lfp channel analysis outputs. How are you doing it now DanL?
On Feb 28, 2019, at 6:10 PM, Peter Petersen notifications@github.com wrote:
I am in need of a standardization of channel level data in buzcode. To follow the nomenclature of cellInfo and sessionInfo, I propose a .channelInfo.mat files that contains channel specific metrics such as impedance measures, average ripple across channels and so fourth.
I propose the mat file should contain a structure that includes list of channels: .channels, and how the data was derived similar to how the FindRipples have the fields .detectorName and .detectorParams
Do you have any input on this? For it or do you have a better solution?
— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/buzsakilab/buzcode/issues/300, or mute the thread https://github.com/notifications/unsubscribe-auth/AM8oQ1BCxwye8ufLkWXHetAiJ8EBEv5_ks5vSGHpgaJpZM4bX7gu.
— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/buzsakilab/buzcode/issues/300#issuecomment-468486843, or mute the thread https://github.com/notifications/unsubscribe-auth/AG7dmHKs7vLIwfLfVMCBiuv8RzyAhK_nks5vSGv5gaJpZM4bX7gu.
I have code for reading the .csv impedances in here: ... you could have them auto-load into this new thing
tf = fullfile(basepath,AnimalMetadata.ExtracellEphys.Channels.ImpedanceFilenames{pidx}); txt = read_mixed_csv(tf); numchans = size(txt,1)-1; % channums = 1:numchans; for cidx = 1:numchans eloc = strfind(txt{cidx+1,5},'e'); n1 = str2num(txt{cidx+1,5}(1:eloc-1)); n2 = str2num(txt{cidx+1,5}(eloc+1:end)); imp(cidx) = n1*10^n2; end % AnimalMetadata.ExtracellEphys.Probes.ImpedanceByChannel{pidx} = cat(2,channums',imp'); AnimalMetadata.ExtracellEphys.Channels.ImpedanceByChannel{pidx} = imp';
On Thu, Feb 28, 2019 at 7:05 PM Dan Levenstein notifications@github.com wrote:
A channelInfo.mat filetype sounds very useful to me! I don’t have anything like that right now, but can think of quite a few things that would fit in that box as well… for example a population spike-phase coupling depth profile or CSD depth profile used for laminar detection.
Want to start a section in the git wiki and we can update/add conventional fields as they seem necessary? We should make a load function similar to bz_LoadCellinfo and each detector should make its own channelInfo.mat file that follows the conventions.
Dan
On Feb 28, 2019, at 6:53 PM, AntonioFR notifications@github.com wrote:
That sounds good. We can use such structure to store for example wavelets and other types of lfp channel analysis outputs. How are you doing it now DanL?
On Feb 28, 2019, at 6:10 PM, Peter Petersen notifications@github.com wrote:
I am in need of a standardization of channel level data in buzcode. To follow the nomenclature of cellInfo and sessionInfo, I propose a .channelInfo.mat files that contains channel specific metrics such as impedance measures, average ripple across channels and so fourth.
I propose the mat file should contain a structure that includes list of channels: .channels, and how the data was derived similar to how the FindRipples have the fields .detectorName and .detectorParams
Do you have any input on this? For it or do you have a better solution?
— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub < https://github.com/buzsakilab/buzcode/issues/300>, or mute the thread < https://github.com/notifications/unsubscribe-auth/AM8oQ1BCxwye8ufLkWXHetAiJ8EBEv5_ks5vSGHpgaJpZM4bX7gu .
— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub < https://github.com/buzsakilab/buzcode/issues/300#issuecomment-468486843>, or mute the thread < https://github.com/notifications/unsubscribe-auth/AG7dmHKs7vLIwfLfVMCBiuv8RzyAhK_nks5vSGv5gaJpZM4bX7gu .
— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.
I updated the wiki with the new channelinfo section. Feel free to add to it. https://github.com/buzsakilab/buzcode/wiki/Data-Formatting-Standards#channel-info
Oh here is this function which was called in my previous code:
function lineArray = read_mixed_csv(fileName,delimiter)
% copied from http://stackoverflow.com/questions/4747834/import-csv-file-with-mixed-data-types
% Brendon Watson 2014
if ~exist('delimiter','var')
delimiter = ',';
end
fid = fopen(fileName,'r'); %# Open the file
lineArray = cell(100,1); %# Preallocate a cell array (ideally slightly
%# larger than is needed)
lineIndex = 1; %# Index of cell to place the next line in
nextLine = fgetl(fid); %# Read the first line from the file
while ~isequal(nextLine,-1) %# Loop while not at the end of the file
lineArray{lineIndex} = nextLine; %# Add the line to the cell array
lineIndex = lineIndex+1; %# Increment the line index
nextLine = fgetl(fid); %# Read the next line from the file
end
fclose(fid); %# Close the file
lineArray = lineArray(1:lineIndex-1); %# Remove empty cells, if needed
for iLine = 1:lineIndex-1 %# Loop over lines
lineData = textscan(lineArray{iLine},'%s',... %# Read strings
'Delimiter',delimiter);
lineData = lineData{1}; %# Remove cell encapsulation
if strcmp(lineArray{iLine}(end),delimiter) %# Account for when the line
lineData{end+1} = ''; %# ends with a delimiter
end
lineArray(iLine,1:numel(lineData)) = lineData; %# Overwrite line data
end
On Fri, Mar 1, 2019 at 5:05 PM Peter Petersen notifications@github.com wrote:
I updated the wiki with the new channelinfo section. Feel free to add to it. https://github.com/buzsakilab/buzcode/wiki/Data-Formatting-Standards#channel-info
— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/buzsakilab/buzcode/issues/300#issuecomment-468828013, or mute the thread https://github.com/notifications/unsubscribe-auth/ADXrTSZkxw8OS01FO6tLdgQdqUX9ccHCks5vSaQggaJpZM4bX7gu .
Like in the code I sent in the previous email. So you can read impedances automatically into matlab from Intan impedance files
On Fri, Mar 1, 2019 at 5:05 PM Peter Petersen notifications@github.com wrote:
I updated the wiki with the new channelinfo section. Feel free to add to it. https://github.com/buzsakilab/buzcode/wiki/Data-Formatting-Standards#channel-info
— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/buzsakilab/buzcode/issues/300#issuecomment-468828013, or mute the thread https://github.com/notifications/unsubscribe-auth/ADXrTSZkxw8OS01FO6tLdgQdqUX9ccHCks5vSaQggaJpZM4bX7gu .
A few updates to make it comparable to other buzcode structures: Thoughts? Is using the standard “data” field too restrictive?
A .channelinfo.mat is a file format for storing info about recorded channels. For example, rippleCorrelogram.channelinfo.mat, csd.channelinfo.mat, electrodeImpedanceMeasures.channelinfo.mat.
File naming format: baseName.channelinfoName.channelinfo.mat
Struct Fields (required):
chanID (0-indexed, matching neuroscope) data (the info about recorded channels, channels should be in the first dimension?) baseName detectorinfo: substructure with information about the detection method (fields below) .detectorname: name of the function used for detection .detectionparms: parameters used for detection .detectiondate: date of detection (optional, depending on what is being stored):
x_bins eventName timestamps
On Mar 1, 2019, at 5:05 PM, Peter Petersen notifications@github.com wrote:
I updated the wiki with the new channelinfo section. Feel free to add to it. https://github.com/buzsakilab/buzcode/wiki/Data-Formatting-Standards#channel-info https://github.com/buzsakilab/buzcode/wiki/Data-Formatting-Standards#channel-info — You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/buzsakilab/buzcode/issues/300#issuecomment-468828013, or mute the thread https://github.com/notifications/unsubscribe-auth/AG7dmNRSQI9XYzxw_OS2ciwhNEEgJgzXks5vSaQggaJpZM4bX7gu.
.data is too restrictive as you want to be able to save multiple variables like what is being done in FindRipples. I consider detectorinfo more related to an event detection and not a calculation. As channel specific data can be triggered by events, it could inherit how the events were detected, but I would separate out the processing function from this? For behavior there is a .processingfunction which seems more relevant here?
As I see it:
-multiple variables that are detector-specific are added as optional fields as needed. but there is generally a primary results (for example, the rate in rate maps) that corresponds to what the title of the file is. I usually put that in a .data field. FindRipples is an .events.mat - so it has .timestamps (a general-purpose required field for events.mat files). Along with the other multiple variables that are specific to ripples/ripple detection.
-I see detectorinfo as a very general title. It’s just nice to have it the same across all buzcode structures for ease of other multi-use functions. If I had seen the behavior conventions I would have changes processingfunction to what I see as the “standard” detectorinfo structure.
TL;DR I’d like to maximize generality. If we can make things fit into a .data and a .detectorinfo field, I think this is ideal for generalization across types (and allows use by as many future and current multi-purpose tools as possible). Other variables can always be saved as appropriate for the specifics of the channeinfo that’s being saved.
Anyway, that’s my 2 cents. Feel free to take it or leave it.
On Mar 4, 2019, at 3:19 PM, Peter Petersen notifications@github.com wrote:
.data is too restrictive as you want to be able to save multiple variables like what is being done in FindRipples. I consider detectorinfo more related to an event detection and not a calculation. As channel specific data can be triggered by events, it could inherit how the events were detected, but I would separate out the processing function from this? For behavior there is a .processingfunction which seems more relevant here?
— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/buzsakilab/buzcode/issues/300#issuecomment-469406638, or mute the thread https://github.com/notifications/unsubscribe-auth/AG7dmOi-vEUl2DIRCjr3aIObpMbnIpwqks5vTX_sgaJpZM4bX7gu.
solved
Hi Peter,
What have you updated exactly? I guess we should update the CellExplore repository now, right?
On Wed, Nov 18, 2020 at 6:32 AM Peter C. Petersen notifications@github.com wrote:
Closed #300 https://github.com/buzsakilab/buzcode/issues/300.
— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/buzsakilab/buzcode/issues/300#event-4010193386, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADHSQQ66EC4SWHCKKJKEVGLSQOWDXANCNFSM4G27XAXA .
-- Antonio Fernandez-Ruiz, PhD Sir Henry Wellcome Post-doctoral fellow Buzsaki Lab. NYU Neuroscience Institute New York University, Langone Medical Center East River Science Park, 450 East 29th Street, 9th Floor New York, NY 10016
I just realized there were 64 open issues, so closed the two I created a while back that I consider solved.
We did this back then but never closed the issue: https://github.com/buzsakilab/buzcode/wiki/Data-Formatting-Standards#channel-info
Also implemented in CellExplorer.
Is this what you meant?
Ah ok, I have seen that already, thanks! By the way, I’m using your cell metrics always now and liked it a lot!
On Nov 18, 2020, at 9:57 AM, Peter C. Petersen notifications@github.com wrote:
We did this back then but never closed the issue: https://github.com/buzsakilab/buzcode/wiki/Data-Formatting-Standards#channel-info https://github.com/buzsakilab/buzcode/wiki/Data-Formatting-Standards#channel-info Also implemented in CellExplorer.
Is this what you meant?
— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/buzsakilab/buzcode/issues/300#issuecomment-729733873, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADHSQQ27YEOUL5KNSXDLXELSQPOHBANCNFSM4G27XAXA.
I am in need of a standardization of channel level data in buzcode. To follow the nomenclature of cellInfo and sessionInfo, I propose a .channelInfo.mat files that contains channel specific metrics such as impedance measures, average ripple across channels and so fourth.
I propose the mat file should contain a structure that includes list of channels: .channels, and how the data was derived similar to how the FindRipples have the fields .detectorName and .detectorParams
Do you have any input on this? For it or do you have a better solution?