search

SpikeInterface / spikeinterface

A Python-based module for creating flexible and robust spike sorting pipelines.
https://spikeinterface.readthedocs.io
MIT License
533 stars 187 forks source link

SpikeGLX readin problem & question about public recording recommendation #2184

Closed ZoeChen96 closed 1 year ago

ZoeChen96 commented 1 year ago

Hi, I asked the question here yesterday about spikeGLX. I have 4 spikeGLX datasets, and I found only 1 of the 4 can't work after I rename the file. However, I tried with two other resources and they both work: spikeGLX original link another link The error is below. Can you have a look?

Besides, I want to ask for a favor if you have any recommendation about widely-used neuropixel 2.0 (384channel) recording with ground truth (maybe curated ground truth) . I know there is one in your spikeinterface paper, but unfortunately there are some unconnected channels: link to dandi datasets Is there any other good dataset? Or it's common that there are unconnected channels?

{
    "name": "KeyError",
    "message": "'snsMnMaXaDw'",
    "stack": "---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
/imec/other/macaw/chen14/pysimulator/maindevelop.ipynb Cell 20 line 4
      <a href='vscode-notebook-cell://ssh-remote%2Bchen2/imec/other/macaw/chen14/pysimulator/maindevelop.ipynb#X25sdnNjb2RlLXJlbW90ZQ%3D%3D?line=0'>1</a> import spikeinterface.extractors as se
      <a href='vscode-notebook-cell://ssh-remote%2Bchen2/imec/other/macaw/chen14/pysimulator/maindevelop.ipynb#X25sdnNjb2RlLXJlbW90ZQ%3D%3D?line=2'>3</a> folder = \"/imec/other/macaw/projectdata/neuropixels_datasets/set2\"
----> <a href='vscode-notebook-cell://ssh-remote%2Bchen2/imec/other/macaw/chen14/pysimulator/maindevelop.ipynb#X25sdnNjb2RlLXJlbW90ZQ%3D%3D?line=3'>4</a> rec = se.read_spikeglx(folder_path=folder)
      <a href='vscode-notebook-cell://ssh-remote%2Bchen2/imec/other/macaw/chen14/pysimulator/maindevelop.ipynb#X25sdnNjb2RlLXJlbW90ZQ%3D%3D?line=4'>5</a> a = rec.get_traces()
      <a href='vscode-notebook-cell://ssh-remote%2Bchen2/imec/other/macaw/chen14/pysimulator/maindevelop.ipynb#X25sdnNjb2RlLXJlbW90ZQ%3D%3D?line=5'>6</a> print(a)

File /imec/other/macaw/chen14/anaconda43/envs/tfqu1016/lib/python3.8/site-packages/spikeinterface/extractors/neoextractors/spikeglx.py:51, in SpikeGLXRecordingExtractor.__init__(self, folder_path, load_sync_channel, stream_id, stream_name, all_annotations)
     49 def __init__(self, folder_path, load_sync_channel=False, stream_id=None, stream_name=None, all_annotations=False):
     50     neo_kwargs = self.map_to_neo_kwargs(folder_path, load_sync_channel=load_sync_channel)
---> 51     NeoBaseRecordingExtractor.__init__(
     52         self, stream_id=stream_id, stream_name=stream_name, all_annotations=all_annotations, **neo_kwargs
     53     )
     55     # open the corresponding stream probe for LF and AP
     56     # if load_sync_channel=False
     57     if \"nidq\" not in self.stream_id and not load_sync_channel:

File /imec/other/macaw/chen14/anaconda43/envs/tfqu1016/lib/python3.8/site-packages/spikeinterface/extractors/neoextractors/neobaseextractor.py:185, in NeoBaseRecordingExtractor.__init__(self, stream_id, stream_name, block_index, all_annotations, use_names_as_ids, **neo_kwargs)
    156 def __init__(
    157     self,
    158     stream_id: Optional[str] = None,
   (...)
    163     **neo_kwargs: Dict[str, Any],
    164 ) -> None:
    165     \"\"\"
    166     Initialize a NeoBaseRecordingExtractor instance.
    167 
   (...)
    182 
    183     \"\"\"
--> 185     _NeoBaseExtractor.__init__(self, block_index, **neo_kwargs)
    187     kwargs = dict(all_annotations=all_annotations)
    188     if block_index is not None:

File /imec/other/macaw/chen14/anaconda43/envs/tfqu1016/lib/python3.8/site-packages/spikeinterface/extractors/neoextractors/neobaseextractor.py:25, in _NeoBaseExtractor.__init__(self, block_index, **neo_kwargs)
     23 def __init__(self, block_index, **neo_kwargs):
     24     if not hasattr(self, \"neo_reader\"):  # Avoid double initialization
---> 25         self.neo_reader = self.get_neo_io_reader(self.NeoRawIOClass, **neo_kwargs)
     27     if self.neo_reader.block_count() > 1 and block_index is None:
     28         raise Exception(
     29             \"This dataset is multi-block. Spikeinterface can load one block at a time. \"
     30             \"Use 'block_index' to select the block to be loaded.\"
     31         )

File /imec/other/macaw/chen14/anaconda43/envs/tfqu1016/lib/python3.8/site-packages/spikeinterface/extractors/neoextractors/neobaseextractor.py:64, in _NeoBaseExtractor.get_neo_io_reader(cls, raw_class, **neo_kwargs)
     62 neoIOclass = getattr(rawio_module, raw_class)
     63 neo_reader = neoIOclass(**neo_kwargs)
---> 64 neo_reader.parse_header()
     66 return neo_reader

File ~/.local/lib/python3.8/site-packages/neo/rawio/baserawio.py:179, in BaseRawIO.parse_header(self)
    166 def parse_header(self):
    167     \"\"\"
    168     This must parse the file header to get all stuff for fast use later on.
    169 
   (...)
    177 
    178     \"\"\"
--> 179     self._parse_header()
    180     self._check_stream_signal_channel_characteristics()

File ~/.local/lib/python3.8/site-packages/neo/rawio/spikeglxrawio.py:81, in SpikeGLXRawIO._parse_header(self)
     80 def _parse_header(self):
---> 81     self.signals_info_list = scan_files(self.dirname)
     83     # sort stream_name by higher sampling rate first
     84     srates = {info['stream_name']: info['sampling_rate'] for info in self.signals_info_list}

File ~/.local/lib/python3.8/site-packages/neo/rawio/spikeglxrawio.py:254, in scan_files(dirname)
    252 if meta_filename.exists() and bin_filename.exists():
    253     meta = read_meta_file(meta_filename)
--> 254     info = extract_stream_info(meta_filename, meta)
    256     info['meta_file'] = str(meta_filename)
    257     info['bin_file'] = str(bin_filename)

File ~/.local/lib/python3.8/site-packages/neo/rawio/spikeglxrawio.py:415, in extract_stream_info(meta_file, meta)
    412 channel_gains = np.ones(num_chan)
    414 # there are differents kinds of channels with different gain values
--> 415 mn, ma, xa, dw = [int(e) for e in meta['snsMnMaXaDw'].split(sep=',')]
    416 per_channel_gain = np.ones(num_chan, dtype='float64')
    417 per_channel_gain[0:mn] = 1. / float(meta['niMNGain'])

KeyError: 'snsMnMaXaDw'"
}
alejoe91 commented 1 year ago

Hi, can you share the .meta files?

Regarding NP2 data with ground-truth, I'm not sure there are any. Are you looking for simulated data? In that case, you can easily generate some with MEArec

ZoeChen96 commented 1 year ago

Hi Alessio, yes sure, here it is: 

acqApLfSy=384,0,1
appVersion=20230323
fileCreateTime=2020-01-07T11:17:23
fileName=G:/bernardo_tacca_sample/NP2.0/SC035_010720_g0_t0.exported.imec1.ap.bin
fileSHA1=FF17CB550A95347677C7C414A4140D5D995F96B2
fileSizeBytes=13859737430
fileTimeSecs=599.9999833330181
firstSample=84968814
gateMode=Immediate
imAiRangeMax=0.5
imAiRangeMin=-0.5
imCalibrated=true
imDatApi=2.11
imDatBs_fw=1.1.128
imDatBsc_fw=2.0.19
imDatBsc_hw=2.1
imDatBsc_pn=NP2_QBSC_00
imDatBsc_sn=259
imDatFx_hw=1.7
imDatFx_pn=NPM_FLEX_0
imDatHs_fw=0.0
imDatHs_pn=000000000
imDatHs_sn=1440
imDatPrb_dock=1
imDatPrb_pn=PRB2_1_2_0640_0
imDatPrb_port=2
imDatPrb_slot=3
imDatPrb_sn=19011111614
imDatPrb_type=21
imLEDEnable=false
imMaxInt=8192
imRoFile=C:/SpikeGLX/Release_v20190919-phase20/IMRO_Tables/NPtype21_bank0_ref1.imro
imSampRate=29999.4325
imStdby=
imTrgRising=false
imTrgSource=0
nSavedChans=385
snsApLfSy=384,0,1
snsSaveChanSubset=0:384
syncImInputSlot=3
syncSourceIdx=2
syncSourcePeriod=1
trigMode=Immediate
typeImEnabled=3
typeNiEnabled=1
typeThis=imec
userNotes=Imec0: left ALM; Imec1: left Mdibrain; Imec2: right Medulla\nSingle shank for Midbrain, 4shank probes for ALM and Medulla
~imroTbl=(21,384)(0 1 1 0)(1 1 1 1)(2 1 1 2)(3 1 1 3)(4 1 1 4)(5 1 1 5)(6 1 1 6)(7 1 1 7)(8 1 1 8)(9 1 1 9)(10 1 1 10)(11 1 1 11)(12 1 1 12)(13 1 1 13)(14 1 1 14)(15 1 1 15)(16 1 1 16)(17 1 1 17)(18 1 1 18)(19 1 1 19)(20 1 1 20)(21 1 1 21)(22 1 1 22)(23 1 1 23)(24 1 1 24)(25 1 1 25)(26 1 1 26)(27 1 1 27)(28 1 1 28)(29 1 1 29)(30 1 1 30)(31 1 1 31)(32 1 1 32)(33 1 1 33)(34 1 1 34)(35 1 1 35)(36 1 1 36)(37 1 1 37)(38 1 1 38)(39 1 1 39)(40 1 1 40)(41 1 1 41)(42 1 1 42)(43 1 1 43)(44 1 1 44)(45 1 1 45)(46 1 1 46)(47 1 1 47)(48 1 1 48)(49 1 1 49)(50 1 1 50)(51 1 1 51)(52 1 1 52)(53 1 1 53)(54 1 1 54)(55 1 1 55)(56 1 1 56)(57 1 1 57)(58 1 1 58)(59 1 1 59)(60 1 1 60)(61 1 1 61)(62 1 1 62)(63 1 1 63)(64 1 1 64)(65 1 1 65)(66 1 1 66)(67 1 1 67)(68 1 1 68)(69 1 1 69)(70 1 1 70)(71 1 1 71)(72 1 1 72)(73 1 1 73)(74 1 1 74)(75 1 1 75)(76 1 1 76)(77 1 1 77)(78 1 1 78)(79 1 1 79)(80 1 1 80)(81 1 1 81)(82 1 1 82)(83 1 1 83)(84 1 1 84)(85 1 1 85)(86 1 1 86)(87 1 1 87)(88 1 1 88)(89 1 1 89)(90 1 1 90)(91 1 1 91)(92 1 1 92)(93 1 1 93)(94 1 1 94)(95 1 1 95)(96 1 1 96)(97 1 1 97)(98 1 1 98)(99 1 1 99)(100 1 1 100)(101 1 1 101)(102 1 1 102)(103 1 1 103)(104 1 1 104)(105 1 1 105)(106 1 1 106)(107 1 1 107)(108 1 1 108)(109 1 1 109)(110 1 1 110)(111 1 1 111)(112 1 1 112)(113 1 1 113)(114 1 1 114)(115 1 1 115)(116 1 1 116)(117 1 1 117)(118 1 1 118)(119 1 1 119)(120 1 1 120)(121 1 1 121)(122 1 1 122)(123 1 1 123)(124 1 1 124)(125 1 1 125)(126 1 1 126)(127 1 1 127)(128 1 1 128)(129 1 1 129)(130 1 1 130)(131 1 1 131)(132 1 1 132)(133 1 1 133)(134 1 1 134)(135 1 1 135)(136 1 1 136)(137 1 1 137)(138 1 1 138)(139 1 1 139)(140 1 1 140)(141 1 1 141)(142 1 1 142)(143 1 1 143)(144 1 1 144)(145 1 1 145)(146 1 1 146)(147 1 1 147)(148 1 1 148)(149 1 1 149)(150 1 1 150)(151 1 1 151)(152 1 1 152)(153 1 1 153)(154 1 1 154)(155 1 1 155)(156 1 1 156)(157 1 1 157)(158 1 1 158)(159 1 1 159)(160 1 1 160)(161 1 1 161)(162 1 1 162)(163 1 1 163)(164 1 1 164)(165 1 1 165)(166 1 1 166)(167 1 1 167)(168 1 1 168)(169 1 1 169)(170 1 1 170)(171 1 1 171)(172 1 1 172)(173 1 1 173)(174 1 1 174)(175 1 1 175)(176 1 1 176)(177 1 1 177)(178 1 1 178)(179 1 1 179)(180 1 1 180)(181 1 1 181)(182 1 1 182)(183 1 1 183)(184 1 1 184)(185 1 1 185)(186 1 1 186)(187 1 1 187)(188 1 1 188)(189 1 1 189)(190 1 1 190)(191 1 1 191)(192 1 1 192)(193 1 1 193)(194 1 1 194)(195 1 1 195)(196 1 1 196)(197 1 1 197)(198 1 1 198)(199 1 1 199)(200 1 1 200)(201 1 1 201)(202 1 1 202)(203 1 1 203)(204 1 1 204)(205 1 1 205)(206 1 1 206)(207 1 1 207)(208 1 1 208)(209 1 1 209)(210 1 1 210)(211 1 1 211)(212 1 1 212)(213 1 1 213)(214 1 1 214)(215 1 1 215)(216 1 1 216)(217 1 1 217)(218 1 1 218)(219 1 1 219)(220 1 1 220)(221 1 1 221)(222 1 1 222)(223 1 1 223)(224 1 1 224)(225 1 1 225)(226 1 1 226)(227 1 1 227)(228 1 1 228)(229 1 1 229)(230 1 1 230)(231 1 1 231)(232 1 1 232)(233 1 1 233)(234 1 1 234)(235 1 1 235)(236 1 1 236)(237 1 1 237)(238 1 1 238)(239 1 1 239)(240 1 1 240)(241 1 1 241)(242 1 1 242)(243 1 1 243)(244 1 1 244)(245 1 1 245)(246 1 1 246)(247 1 1 247)(248 1 1 248)(249 1 1 249)(250 1 1 250)(251 1 1 251)(252 1 1 252)(253 1 1 253)(254 1 1 254)(255 1 1 255)(256 1 1 256)(257 1 1 257)(258 1 1 258)(259 1 1 259)(260 1 1 260)(261 1 1 261)(262 1 1 262)(263 1 1 263)(264 1 1 264)(265 1 1 265)(266 1 1 266)(267 1 1 267)(268 1 1 268)(269 1 1 269)(270 1 1 270)(271 1 1 271)(272 1 1 272)(273 1 1 273)(274 1 1 274)(275 1 1 275)(276 1 1 276)(277 1 1 277)(278 1 1 278)(279 1 1 279)(280 1 1 280)(281 1 1 281)(282 1 1 282)(283 1 1 283)(284 1 1 284)(285 1 1 285)(286 1 1 286)(287 1 1 287)(288 1 1 288)(289 1 1 289)(290 1 1 290)(291 1 1 291)(292 1 1 292)(293 1 1 293)(294 1 1 294)(295 1 1 295)(296 1 1 296)(297 1 1 297)(298 1 1 298)(299 1 1 299)(300 1 1 300)(301 1 1 301)(302 1 1 302)(303 1 1 303)(304 1 1 304)(305 1 1 305)(306 1 1 306)(307 1 1 307)(308 1 1 308)(309 1 1 309)(310 1 1 310)(311 1 1 311)(312 1 1 312)(313 1 1 313)(314 1 1 314)(315 1 1 315)(316 1 1 316)(317 1 1 317)(318 1 1 318)(319 1 1 319)(320 1 1 320)(321 1 1 321)(322 1 1 322)(323 1 1 323)(324 1 1 324)(325 1 1 325)(326 1 1 326)(327 1 1 327)(328 1 1 328)(329 1 1 329)(330 1 1 330)(331 1 1 331)(332 1 1 332)(333 1 1 333)(334 1 1 334)(335 1 1 335)(336 1 1 336)(337 1 1 337)(338 1 1 338)(339 1 1 339)(340 1 1 340)(341 1 1 341)(342 1 1 342)(343 1 1 343)(344 1 1 344)(345 1 1 345)(346 1 1 346)(347 1 1 347)(348 1 1 348)(349 1 1 349)(350 1 1 350)(351 1 1 351)(352 1 1 352)(353 1 1 353)(354 1 1 354)(355 1 1 355)(356 1 1 356)(357 1 1 357)(358 1 1 358)(359 1 1 359)(360 1 1 360)(361 1 1 361)(362 1 1 362)(363 1 1 363)(364 1 1 364)(365 1 1 365)(366 1 1 366)(367 1 1 367)(368 1 1 368)(369 1 1 369)(370 1 1 370)(371 1 1 371)(372 1 1 372)(373 1 1 373)(374 1 1 374)(375 1 1 375)(376 1 1 376)(377 1 1 377)(378 1 1 378)(379 1 1 379)(380 1 1 380)(381 1 1 381)(382 1 1 382)(383 1 1 383)
~snsChanMap=(384,0,1)(AP0;0:0)(AP1;1:1)(AP2;2:2)(AP3;3:3)(AP4;4:4)(AP5;5:5)(AP6;6:6)(AP7;7:7)(AP8;8:8)(AP9;9:9)(AP10;10:10)(AP11;11:11)(AP12;12:12)(AP13;13:13)(AP14;14:14)(AP15;15:15)(AP16;16:16)(AP17;17:17)(AP18;18:18)(AP19;19:19)(AP20;20:20)(AP21;21:21)(AP22;22:22)(AP23;23:23)(AP24;24:24)(AP25;25:25)(AP26;26:26)(AP27;27:27)(AP28;28:28)(AP29;29:29)(AP30;30:30)(AP31;31:31)(AP32;32:32)(AP33;33:33)(AP34;34:34)(AP35;35:35)(AP36;36:36)(AP37;37:37)(AP38;38:38)(AP39;39:39)(AP40;40:40)(AP41;41:41)(AP42;42:42)(AP43;43:43)(AP44;44:44)(AP45;45:45)(AP46;46:46)(AP47;47:47)(AP48;48:48)(AP49;49:49)(AP50;50:50)(AP51;51:51)(AP52;52:52)(AP53;53:53)(AP54;54:54)(AP55;55:55)(AP56;56:56)(AP57;57:57)(AP58;58:58)(AP59;59:59)(AP60;60:60)(AP61;61:61)(AP62;62:62)(AP63;63:63)(AP64;64:64)(AP65;65:65)(AP66;66:66)(AP67;67:67)(AP68;68:68)(AP69;69:69)(AP70;70:70)(AP71;71:71)(AP72;72:72)(AP73;73:73)(AP74;74:74)(AP75;75:75)(AP76;76:76)(AP77;77:77)(AP78;78:78)(AP79;79:79)(AP80;80:80)(AP81;81:81)(AP82;82:82)(AP83;83:83)(AP84;84:84)(AP85;85:85)(AP86;86:86)(AP87;87:87)(AP88;88:88)(AP89;89:89)(AP90;90:90)(AP91;91:91)(AP92;92:92)(AP93;93:93)(AP94;94:94)(AP95;95:95)(AP96;96:96)(AP97;97:97)(AP98;98:98)(AP99;99:99)(AP100;100:100)(AP101;101:101)(AP102;102:102)(AP103;103:103)(AP104;104:104)(AP105;105:105)(AP106;106:106)(AP107;107:107)(AP108;108:108)(AP109;109:109)(AP110;110:110)(AP111;111:111)(AP112;112:112)(AP113;113:113)(AP114;114:114)(AP115;115:115)(AP116;116:116)(AP117;117:117)(AP118;118:118)(AP119;119:119)(AP120;120:120)(AP121;121:121)(AP122;122:122)(AP123;123:123)(AP124;124:124)(AP125;125:125)(AP126;126:126)(AP127;127:127)(AP128;128:128)(AP129;129:129)(AP130;130:130)(AP131;131:131)(AP132;132:132)(AP133;133:133)(AP134;134:134)(AP135;135:135)(AP136;136:136)(AP137;137:137)(AP138;138:138)(AP139;139:139)(AP140;140:140)(AP141;141:141)(AP142;142:142)(AP143;143:143)(AP144;144:144)(AP145;145:145)(AP146;146:146)(AP147;147:147)(AP148;148:148)(AP149;149:149)(AP150;150:150)(AP151;151:151)(AP152;152:152)(AP153;153:153)(AP154;154:154)(AP155;155:155)(AP156;156:156)(AP157;157:157)(AP158;158:158)(AP159;159:159)(AP160;160:160)(AP161;161:161)(AP162;162:162)(AP163;163:163)(AP164;164:164)(AP165;165:165)(AP166;166:166)(AP167;167:167)(AP168;168:168)(AP169;169:169)(AP170;170:170)(AP171;171:171)(AP172;172:172)(AP173;173:173)(AP174;174:174)(AP175;175:175)(AP176;176:176)(AP177;177:177)(AP178;178:178)(AP179;179:179)(AP180;180:180)(AP181;181:181)(AP182;182:182)(AP183;183:183)(AP184;184:184)(AP185;185:185)(AP186;186:186)(AP187;187:187)(AP188;188:188)(AP189;189:189)(AP190;190:190)(AP191;191:191)(AP192;192:192)(AP193;193:193)(AP194;194:194)(AP195;195:195)(AP196;196:196)(AP197;197:197)(AP198;198:198)(AP199;199:199)(AP200;200:200)(AP201;201:201)(AP202;202:202)(AP203;203:203)(AP204;204:204)(AP205;205:205)(AP206;206:206)(AP207;207:207)(AP208;208:208)(AP209;209:209)(AP210;210:210)(AP211;211:211)(AP212;212:212)(AP213;213:213)(AP214;214:214)(AP215;215:215)(AP216;216:216)(AP217;217:217)(AP218;218:218)(AP219;219:219)(AP220;220:220)(AP221;221:221)(AP222;222:222)(AP223;223:223)(AP224;224:224)(AP225;225:225)(AP226;226:226)(AP227;227:227)(AP228;228:228)(AP229;229:229)(AP230;230:230)(AP231;231:231)(AP232;232:232)(AP233;233:233)(AP234;234:234)(AP235;235:235)(AP236;236:236)(AP237;237:237)(AP238;238:238)(AP239;239:239)(AP240;240:240)(AP241;241:241)(AP242;242:242)(AP243;243:243)(AP244;244:244)(AP245;245:245)(AP246;246:246)(AP247;247:247)(AP248;248:248)(AP249;249:249)(AP250;250:250)(AP251;251:251)(AP252;252:252)(AP253;253:253)(AP254;254:254)(AP255;255:255)(AP256;256:256)(AP257;257:257)(AP258;258:258)(AP259;259:259)(AP260;260:260)(AP261;261:261)(AP262;262:262)(AP263;263:263)(AP264;264:264)(AP265;265:265)(AP266;266:266)(AP267;267:267)(AP268;268:268)(AP269;269:269)(AP270;270:270)(AP271;271:271)(AP272;272:272)(AP273;273:273)(AP274;274:274)(AP275;275:275)(AP276;276:276)(AP277;277:277)(AP278;278:278)(AP279;279:279)(AP280;280:280)(AP281;281:281)(AP282;282:282)(AP283;283:283)(AP284;284:284)(AP285;285:285)(AP286;286:286)(AP287;287:287)(AP288;288:288)(AP289;289:289)(AP290;290:290)(AP291;291:291)(AP292;292:292)(AP293;293:293)(AP294;294:294)(AP295;295:295)(AP296;296:296)(AP297;297:297)(AP298;298:298)(AP299;299:299)(AP300;300:300)(AP301;301:301)(AP302;302:302)(AP303;303:303)(AP304;304:304)(AP305;305:305)(AP306;306:306)(AP307;307:307)(AP308;308:308)(AP309;309:309)(AP310;310:310)(AP311;311:311)(AP312;312:312)(AP313;313:313)(AP314;314:314)(AP315;315:315)(AP316;316:316)(AP317;317:317)(AP318;318:318)(AP319;319:319)(AP320;320:320)(AP321;321:321)(AP322;322:322)(AP323;323:323)(AP324;324:324)(AP325;325:325)(AP326;326:326)(AP327;327:327)(AP328;328:328)(AP329;329:329)(AP330;330:330)(AP331;331:331)(AP332;332:332)(AP333;333:333)(AP334;334:334)(AP335;335:335)(AP336;336:336)(AP337;337:337)(AP338;338:338)(AP339;339:339)(AP340;340:340)(AP341;341:341)(AP342;342:342)(AP343;343:343)(AP344;344:344)(AP345;345:345)(AP346;346:346)(AP347;347:347)(AP348;348:348)(AP349;349:349)(AP350;350:350)(AP351;351:351)(AP352;352:352)(AP353;353:353)(AP354;354:354)(AP355;355:355)(AP356;356:356)(AP357;357:357)(AP358;358:358)(AP359;359:359)(AP360;360:360)(AP361;361:361)(AP362;362:362)(AP363;363:363)(AP364;364:364)(AP365;365:365)(AP366;366:366)(AP367;367:367)(AP368;368:368)(AP369;369:369)(AP370;370:370)(AP371;371:371)(AP372;372:372)(AP373;373:373)(AP374;374:374)(AP375;375:375)(AP376;376:376)(AP377;377:377)(AP378;378:378)(AP379;379:379)(AP380;380:380)(AP381;381:381)(AP382;382:382)(AP383;383:383)(SY0;384:384)
~snsGeomMap=(PRB2_1_2_0640_0,1,0,70)(0:27:0:1)(0:59:0:1)(0:27:15:1)(0:59:15:1)(0:27:30:1)(0:59:30:1)(0:27:45:1)(0:59:45:1)(0:27:60:1)(0:59:60:1)(0:27:75:1)(0:59:75:1)(0:27:90:1)(0:59:90:1)(0:27:105:1)(0:59:105:1)(0:27:120:1)(0:59:120:1)(0:27:135:1)(0:59:135:1)(0:27:150:1)(0:59:150:1)(0:27:165:1)(0:59:165:1)(0:27:180:1)(0:59:180:1)(0:27:195:1)(0:59:195:1)(0:27:210:1)(0:59:210:1)(0:27:225:1)(0:59:225:1)(0:27:240:1)(0:59:240:1)(0:27:255:1)(0:59:255:1)(0:27:270:1)(0:59:270:1)(0:27:285:1)(0:59:285:1)(0:27:300:1)(0:59:300:1)(0:27:315:1)(0:59:315:1)(0:27:330:1)(0:59:330:1)(0:27:345:1)(0:59:345:1)(0:27:360:1)(0:59:360:1)(0:27:375:1)(0:59:375:1)(0:27:390:1)(0:59:390:1)(0:27:405:1)(0:59:405:1)(0:27:420:1)(0:59:420:1)(0:27:435:1)(0:59:435:1)(0:27:450:1)(0:59:450:1)(0:27:465:1)(0:59:465:1)(0:27:480:1)(0:59:480:1)(0:27:495:1)(0:59:495:1)(0:27:510:1)(0:59:510:1)(0:27:525:1)(0:59:525:1)(0:27:540:1)(0:59:540:1)(0:27:555:1)(0:59:555:1)(0:27:570:1)(0:59:570:1)(0:27:585:1)(0:59:585:1)(0:27:600:1)(0:59:600:1)(0:27:615:1)(0:59:615:1)(0:27:630:1)(0:59:630:1)(0:27:645:1)(0:59:645:1)(0:27:660:1)(0:59:660:1)(0:27:675:1)(0:59:675:1)(0:27:690:1)(0:59:690:1)(0:27:705:1)(0:59:705:1)(0:27:720:1)(0:59:720:1)(0:27:735:1)(0:59:735:1)(0:27:750:1)(0:59:750:1)(0:27:765:1)(0:59:765:1)(0:27:780:1)(0:59:780:1)(0:27:795:1)(0:59:795:1)(0:27:810:1)(0:59:810:1)(0:27:825:1)(0:59:825:1)(0:27:840:1)(0:59:840:1)(0:27:855:1)(0:59:855:1)(0:27:870:1)(0:59:870:1)(0:27:885:1)(0:59:885:1)(0:27:900:1)(0:59:900:1)(0:27:915:1)(0:59:915:1)(0:27:930:1)(0:59:930:1)(0:27:945:1)(0:59:945:0)(0:27:960:1)(0:59:960:1)(0:27:975:1)(0:59:975:1)(0:27:990:1)(0:59:990:1)(0:27:1005:1)(0:59:1005:1)(0:27:1020:1)(0:59:1020:1)(0:27:1035:1)(0:59:1035:1)(0:27:1050:1)(0:59:1050:1)(0:27:1065:1)(0:59:1065:1)(0:27:1080:1)(0:59:1080:1)(0:27:1095:1)(0:59:1095:1)(0:27:1110:1)(0:59:1110:1)(0:27:1125:1)(0:59:1125:1)(0:27:1140:1)(0:59:1140:1)(0:27:1155:1)(0:59:1155:1)(0:27:1170:1)(0:59:1170:1)(0:27:1185:1)(0:59:1185:1)(0:27:1200:1)(0:59:1200:1)(0:27:1215:1)(0:59:1215:1)(0:27:1230:1)(0:59:1230:1)(0:27:1245:1)(0:59:1245:1)(0:27:1260:1)(0:59:1260:1)(0:27:1275:1)(0:59:1275:1)(0:27:1290:1)(0:59:1290:1)(0:27:1305:1)(0:59:1305:1)(0:27:1320:1)(0:59:1320:1)(0:27:1335:1)(0:59:1335:1)(0:27:1350:1)(0:59:1350:1)(0:27:1365:1)(0:59:1365:1)(0:27:1380:1)(0:59:1380:1)(0:27:1395:1)(0:59:1395:1)(0:27:1410:1)(0:59:1410:1)(0:27:1425:1)(0:59:1425:1)(0:27:1440:1)(0:59:1440:1)(0:27:1455:1)(0:59:1455:1)(0:27:1470:1)(0:59:1470:1)(0:27:1485:1)(0:59:1485:1)(0:27:1500:1)(0:59:1500:1)(0:27:1515:1)(0:59:1515:1)(0:27:1530:1)(0:59:1530:1)(0:27:1545:1)(0:59:1545:1)(0:27:1560:1)(0:59:1560:1)(0:27:1575:1)(0:59:1575:1)(0:27:1590:1)(0:59:1590:1)(0:27:1605:1)(0:59:1605:1)(0:27:1620:1)(0:59:1620:1)(0:27:1635:1)(0:59:1635:1)(0:27:1650:1)(0:59:1650:1)(0:27:1665:1)(0:59:1665:1)(0:27:1680:1)(0:59:1680:1)(0:27:1695:1)(0:59:1695:1)(0:27:1710:1)(0:59:1710:1)(0:27:1725:1)(0:59:1725:1)(0:27:1740:1)(0:59:1740:1)(0:27:1755:1)(0:59:1755:1)(0:27:1770:1)(0:59:1770:1)(0:27:1785:1)(0:59:1785:1)(0:27:1800:1)(0:59:1800:1)(0:27:1815:1)(0:59:1815:1)(0:27:1830:1)(0:59:1830:1)(0:27:1845:1)(0:59:1845:1)(0:27:1860:1)(0:59:1860:1)(0:27:1875:1)(0:59:1875:1)(0:27:1890:1)(0:59:1890:1)(0:27:1905:1)(0:59:1905:1)(0:27:1920:1)(0:59:1920:1)(0:27:1935:1)(0:59:1935:1)(0:27:1950:1)(0:59:1950:1)(0:27:1965:1)(0:59:1965:1)(0:27:1980:1)(0:59:1980:1)(0:27:1995:1)(0:59:1995:1)(0:27:2010:1)(0:59:2010:1)(0:27:2025:1)(0:59:2025:1)(0:27:2040:1)(0:59:2040:1)(0:27:2055:1)(0:59:2055:1)(0:27:2070:1)(0:59:2070:1)(0:27:2085:1)(0:59:2085:1)(0:27:2100:1)(0:59:2100:1)(0:27:2115:1)(0:59:2115:1)(0:27:2130:1)(0:59:2130:1)(0:27:2145:1)(0:59:2145:1)(0:27:2160:1)(0:59:2160:1)(0:27:2175:1)(0:59:2175:1)(0:27:2190:1)(0:59:2190:1)(0:27:2205:1)(0:59:2205:1)(0:27:2220:1)(0:59:2220:1)(0:27:2235:1)(0:59:2235:1)(0:27:2250:1)(0:59:2250:1)(0:27:2265:1)(0:59:2265:1)(0:27:2280:1)(0:59:2280:1)(0:27:2295:1)(0:59:2295:1)(0:27:2310:1)(0:59:2310:1)(0:27:2325:1)(0:59:2325:1)(0:27:2340:1)(0:59:2340:1)(0:27:2355:1)(0:59:2355:1)(0:27:2370:1)(0:59:2370:1)(0:27:2385:1)(0:59:2385:1)(0:27:2400:1)(0:59:2400:1)(0:27:2415:1)(0:59:2415:1)(0:27:2430:1)(0:59:2430:1)(0:27:2445:1)(0:59:2445:1)(0:27:2460:1)(0:59:2460:1)(0:27:2475:1)(0:59:2475:1)(0:27:2490:1)(0:59:2490:1)(0:27:2505:1)(0:59:2505:1)(0:27:2520:1)(0:59:2520:1)(0:27:2535:1)(0:59:2535:1)(0:27:2550:1)(0:59:2550:1)(0:27:2565:1)(0:59:2565:1)(0:27:2580:1)(0:59:2580:1)(0:27:2595:1)(0:59:2595:1)(0:27:2610:1)(0:59:2610:1)(0:27:2625:1)(0:59:2625:1)(0:27:2640:1)(0:59:2640:1)(0:27:2655:1)(0:59:2655:1)(0:27:2670:1)(0:59:2670:1)(0:27:2685:1)(0:59:2685:1)(0:27:2700:1)(0:59:2700:1)(0:27:2715:1)(0:59:2715:1)(0:27:2730:1)(0:59:2730:1)(0:27:2745:1)(0:59:2745:1)(0:27:2760:1)(0:59:2760:1)(0:27:2775:1)(0:59:2775:1)(0:27:2790:1)(0:59:2790:1)(0:27:2805:1)(0:59:2805:1)(0:27:2820:1)(0:59:2820:1)(0:27:2835:1)(0:59:2835:1)(0:27:2850:1)(0:59:2850:1)(0:27:2865:1)(0:59:2865:1)
~snsShankMap=(1,2,640)(0:0:0:1)(0:1:0:1)(0:0:1:1)(0:1:1:1)(0:0:2:1)(0:1:2:1)(0:0:3:1)(0:1:3:1)(0:0:4:1)(0:1:4:1)(0:0:5:1)(0:1:5:1)(0:0:6:1)(0:1:6:1)(0:0:7:1)(0:1:7:1)(0:0:8:1)(0:1:8:1)(0:0:9:1)(0:1:9:1)(0:0:10:1)(0:1:10:1)(0:0:11:1)(0:1:11:1)(0:0:12:1)(0:1:12:1)(0:0:13:1)(0:1:13:1)(0:0:14:1)(0:1:14:1)(0:0:15:1)(0:1:15:1)(0:0:16:1)(0:1:16:1)(0:0:17:1)(0:1:17:1)(0:0:18:1)(0:1:18:1)(0:0:19:1)(0:1:19:1)(0:0:20:1)(0:1:20:1)(0:0:21:1)(0:1:21:1)(0:0:22:1)(0:1:22:1)(0:0:23:1)(0:1:23:1)(0:0:24:1)(0:1:24:1)(0:0:25:1)(0:1:25:1)(0:0:26:1)(0:1:26:1)(0:0:27:1)(0:1:27:1)(0:0:28:1)(0:1:28:1)(0:0:29:1)(0:1:29:1)(0:0:30:1)(0:1:30:1)(0:0:31:1)(0:1:31:1)(0:0:32:1)(0:1:32:1)(0:0:33:1)(0:1:33:1)(0:0:34:1)(0:1:34:1)(0:0:35:1)(0:1:35:1)(0:0:36:1)(0:1:36:1)(0:0:37:1)(0:1:37:1)(0:0:38:1)(0:1:38:1)(0:0:39:1)(0:1:39:1)(0:0:40:1)(0:1:40:1)(0:0:41:1)(0:1:41:1)(0:0:42:1)(0:1:42:1)(0:0:43:1)(0:1:43:1)(0:0:44:1)(0:1:44:1)(0:0:45:1)(0:1:45:1)(0:0:46:1)(0:1:46:1)(0:0:47:1)(0:1:47:1)(0:0:48:1)(0:1:48:1)(0:0:49:1)(0:1:49:1)(0:0:50:1)(0:1:50:1)(0:0:51:1)(0:1:51:1)(0:0:52:1)(0:1:52:1)(0:0:53:1)(0:1:53:1)(0:0:54:1)(0:1:54:1)(0:0:55:1)(0:1:55:1)(0:0:56:1)(0:1:56:1)(0:0:57:1)(0:1:57:1)(0:0:58:1)(0:1:58:1)(0:0:59:1)(0:1:59:1)(0:0:60:1)(0:1:60:1)(0:0:61:1)(0:1:61:1)(0:0:62:1)(0:1:62:1)(0:0:63:1)(0:1:63:0)(0:0:64:1)(0:1:64:1)(0:0:65:1)(0:1:65:1)(0:0:66:1)(0:1:66:1)(0:0:67:1)(0:1:67:1)(0:0:68:1)(0:1:68:1)(0:0:69:1)(0:1:69:1)(0:0:70:1)(0:1:70:1)(0:0:71:1)(0:1:71:1)(0:0:72:1)(0:1:72:1)(0:0:73:1)(0:1:73:1)(0:0:74:1)(0:1:74:1)(0:0:75:1)(0:1:75:1)(0:0:76:1)(0:1:76:1)(0:0:77:1)(0:1:77:1)(0:0:78:1)(0:1:78:1)(0:0:79:1)(0:1:79:1)(0:0:80:1)(0:1:80:1)(0:0:81:1)(0:1:81:1)(0:0:82:1)(0:1:82:1)(0:0:83:1)(0:1:83:1)(0:0:84:1)(0:1:84:1)(0:0:85:1)(0:1:85:1)(0:0:86:1)(0:1:86:1)(0:0:87:1)(0:1:87:1)(0:0:88:1)(0:1:88:1)(0:0:89:1)(0:1:89:1)(0:0:90:1)(0:1:90:1)(0:0:91:1)(0:1:91:1)(0:0:92:1)(0:1:92:1)(0:0:93:1)(0:1:93:1)(0:0:94:1)(0:1:94:1)(0:0:95:1)(0:1:95:1)(0:0:96:1)(0:1:96:1)(0:0:97:1)(0:1:97:1)(0:0:98:1)(0:1:98:1)(0:0:99:1)(0:1:99:1)(0:0:100:1)(0:1:100:1)(0:0:101:1)(0:1:101:1)(0:0:102:1)(0:1:102:1)(0:0:103:1)(0:1:103:1)(0:0:104:1)(0:1:104:1)(0:0:105:1)(0:1:105:1)(0:0:106:1)(0:1:106:1)(0:0:107:1)(0:1:107:1)(0:0:108:1)(0:1:108:1)(0:0:109:1)(0:1:109:1)(0:0:110:1)(0:1:110:1)(0:0:111:1)(0:1:111:1)(0:0:112:1)(0:1:112:1)(0:0:113:1)(0:1:113:1)(0:0:114:1)(0:1:114:1)(0:0:115:1)(0:1:115:1)(0:0:116:1)(0:1:116:1)(0:0:117:1)(0:1:117:1)(0:0:118:1)(0:1:118:1)(0:0:119:1)(0:1:119:1)(0:0:120:1)(0:1:120:1)(0:0:121:1)(0:1:121:1)(0:0:122:1)(0:1:122:1)(0:0:123:1)(0:1:123:1)(0:0:124:1)(0:1:124:1)(0:0:125:1)(0:1:125:1)(0:0:126:1)(0:1:126:1)(0:0:127:1)(0:1:127:1)(0:0:128:1)(0:1:128:1)(0:0:129:1)(0:1:129:1)(0:0:130:1)(0:1:130:1)(0:0:131:1)(0:1:131:1)(0:0:132:1)(0:1:132:1)(0:0:133:1)(0:1:133:1)(0:0:134:1)(0:1:134:1)(0:0:135:1)(0:1:135:1)(0:0:136:1)(0:1:136:1)(0:0:137:1)(0:1:137:1)(0:0:138:1)(0:1:138:1)(0:0:139:1)(0:1:139:1)(0:0:140:1)(0:1:140:1)(0:0:141:1)(0:1:141:1)(0:0:142:1)(0:1:142:1)(0:0:143:1)(0:1:143:1)(0:0:144:1)(0:1:144:1)(0:0:145:1)(0:1:145:1)(0:0:146:1)(0:1:146:1)(0:0:147:1)(0:1:147:1)(0:0:148:1)(0:1:148:1)(0:0:149:1)(0:1:149:1)(0:0:150:1)(0:1:150:1)(0:0:151:1)(0:1:151:1)(0:0:152:1)(0:1:152:1)(0:0:153:1)(0:1:153:1)(0:0:154:1)(0:1:154:1)(0:0:155:1)(0:1:155:1)(0:0:156:1)(0:1:156:1)(0:0:157:1)(0:1:157:1)(0:0:158:1)(0:1:158:1)(0:0:159:1)(0:1:159:1)(0:0:160:1)(0:1:160:1)(0:0:161:1)(0:1:161:1)(0:0:162:1)(0:1:162:1)(0:0:163:1)(0:1:163:1)(0:0:164:1)(0:1:164:1)(0:0:165:1)(0:1:165:1)(0:0:166:1)(0:1:166:1)(0:0:167:1)(0:1:167:1)(0:0:168:1)(0:1:168:1)(0:0:169:1)(0:1:169:1)(0:0:170:1)(0:1:170:1)(0:0:171:1)(0:1:171:1)(0:0:172:1)(0:1:172:1)(0:0:173:1)(0:1:173:1)(0:0:174:1)(0:1:174:1)(0:0:175:1)(0:1:175:1)(0:0:176:1)(0:1:176:1)(0:0:177:1)(0:1:177:1)(0:0:178:1)(0:1:178:1)(0:0:179:1)(0:1:179:1)(0:0:180:1)(0:1:180:1)(0:0:181:1)(0:1:181:1)(0:0:182:1)(0:1:182:1)(0:0:183:1)(0:1:183:1)(0:0:184:1)(0:1:184:1)(0:0:185:1)(0:1:185:1)(0:0:186:1)(0:1:186:1)(0:0:187:1)(0:1:187:1)(0:0:188:1)(0:1:188:1)(0:0:189:1)(0:1:189:1)(0:0:190:1)(0:1:190:1)(0:0:191:1)(0:1:191:1)
ZoeChen96 commented 1 year ago

Hi, can you share the .meta files?

Regarding NP2 data with ground-truth, I'm not sure there are any. Are you looking for simulated data? In that case, you can easily generate some with MEArec

For the datasets, I don't want simulated data, I already know and generated some simulated datasets using MEArec, thanks for mentioning that! Instead, I want some real recordings with curated results (which can be used as ground truth), like what I see in here used in your SpikeInterface paper 😄

alejoe91 commented 1 year ago

What is the name of this .meta file? do you have more than one in the folder?

alejoe91 commented 1 year ago

Note that for how the reader is structured, the meta file must contain the imec substring in its name. Can you try to add it if it's not there?

See here: https://github.com/NeuralEnsemble/python-neo/blob/master/neo/rawio/spikeglxrawio.py#L378

The filename needs to be in the form of: whatevername.imec#.ap.meta (or lf instead of ap)

ZoeChen96 commented 1 year ago

The name is SC035_010720_g0_t0.exported.imec1.ap.meta, I think it is with imec substring. There is only this meta here

alejoe91 commented 1 year ago

right, you need to remove .exported

ZoeChen96 commented 1 year ago

Ok, I will try! I don't have permission to change the name, it will take some time

ZoeChen96 commented 1 year ago

It works, thank you!

alejoe91 commented 1 year ago

Actually, it should work with your file name..not sure why it's not working! @samuelgarcia any idea?

samuelgarcia commented 1 year ago

I think we have to change rules using regexp instead of spliting on dot for getting fields like (ap vs lf) in spikeglx.