kemerelab / jagular

Out-of-core pre-processing of big-ish electrophysiology data
Other
3 stars 1 forks source link

benchmark: channel writes #5

Open eackermann opened 7 years ago

eackermann commented 7 years ago

How does it scale with # channels written out? Does the unpacking take most of the time, or the writing back to disk?

How about the effect of the block size?

jchutrue commented 7 years ago

As discussed before, this is a task that could be assigned to an undergrad, who would then generate the benchmark plots.

The title of the issue indicates benchmarking for channel writes. Where does the unpacking come into play?

eackermann commented 7 years ago

I didn't want to create two separate issues for that, but the title could be made more descriptive.

At any rate, the unpacking is also part of the channel writing process, since we (1) read from large rec file, (2) unpack packet to get to the relevant channel(s), (3) inspect and process the channel(s) with interpolation etc., and (4) write the [potentially] modified channel(s) out to disk.

So under the umbrella of "writing channels", I would like to understand if writing out 50 channels is 50 times slower than writing out only a single channel, or if it's only moderately slower, or whatever.

And yes, with an adequate understanding of the parameter space, an undergrad can and should absolutely do this. But before then, we need to have documentation and things in place.