plot_headers.ipynb not running

[t-sasatani]

It's probably something around handling the header. I will look into it later.

IndexError: index 10 is out of bounds for axis 0 with size 10

at

frames.append(sd.read(return_header=True))

[sneakers-the-rat]

can i see the full traceback?

[sneakers-the-rat]

I would imagine this is from not using a matching SDCardLayout.

So here we construct the header array in two steps, first by reading a single word that should have the buffer length in it and then reading n_fields * word_len after that

https://github.com/Aharoni-Lab/miniscope-io/blob/88d9e7c6b6b2ae5ccbd92f53e337a2fad3e279fa/miniscope_io/io.py#L217-L224

Then we use the Header layout as a data model to be able to address header fields by name (because otherwise their meaning is implicit):

https://github.com/Aharoni-Lab/miniscope-io/blob/88d9e7c6b6b2ae5ccbd92f53e337a2fad3e279fa/miniscope_io/io.py#L227-L232

So I assume you're using a SDCardLayout that has 11 fields (ie. index 10) like the Battery layout (i didn't know what to name it): https://github.com/Aharoni-Lab/miniscope-io/blob/88d9e7c6b6b2ae5ccbd92f53e337a2fad3e279fa/miniscope_io/formats.py#L64

to read data from something recorded in an older format that doesn't have the last battery_voltage field in it (ie. the header only has 10 fields, index 9)

There are a few options I can think of here:

• The source of the problem is ultimately that we don't have a way of inferring the header layout from the data itself. If we instead had some version number or etc. that could be use to match data stored on the SD Card to a data layout in the IO library then we could make this whole thing automatically, including removing the need to explicitly specify an SDCard layout when loading data, which adds friction.
• We could try and silently correct mismatches between stored data and the header data model, eg. in this case just dropping the battery_voltage field, but this would be a hack that will buy us more complexity in the future. Specifically the assumption is that only one field value would only ever be encoded in a given header position, and if that assumption ever gets violated then we will silently pass bad data (eg. calling a timestamp a battery_voltage reading) that would make downstream problems harder to diagnose.

Otherwise this is categorized as "not a bug" - it's a good thing that an error was raised for a mismatch between expected data model and data on card, but in any case we should have a more informative error message.

Or, since I am inferring the bug from just reading the code, it could be something else entirely! To diagnose, I would try this:

TestLayout = WireFreeSDLayout_Battery.copy()
TestLayout.battery_voltage = None

and use that as the layout when you read.

[t-sasatani]

will read and reply october