search

HolyLab / ImagineInterface.jl

Read and write Imagine analog and digital recordings and commands
Other
2 stars 2 forks source link

Validation speed #60

Closed HaoyangRong closed 6 years ago

HaoyangRong commented 7 years ago

The package is doing a very good job appending commands, but the signal validate_allvalidation takes a longer time to run. Validating signals last \~45min (With a sampling rate of 50000) took about 40\~45s, while appending those signals to the rigtemplate object took just a few seconds. Is it possible to have some improvements in the validation speed? Thanks!

HaoyangRong commented 7 years ago

Or maybe we can have an option fed into the write_commands function to allow partial validation or no validation.

Cody-G commented 7 years ago

It's not surprising that appending is faster because if you append an RLEVector then it doesn't have to read or copy any samples. Validation requires that we look at each sample at least once. 45s of validation for a 45 minute experiment is consistent with what I was seeing. I'm sure we can optimize it further. I wouldn't be surprised if we could get it down to 10s or so with a few hours of work. But can I ask what is your typical workflow? For me the 1s validation per 1minute of recording feels tolerable because I only write the full file once per experiment. (I may write some smaller files as tests before doing the full recording).

I'm OK with the idea of adding a keyword argument to disable validation, but keep in mind that if there is a problem with the command then Imagine will refuse to run it, which can be pretty frustrating in the middle of an experiment.

HaoyangRong commented 7 years ago

I usually generate a JSON file for a test trial first, using the same settings for the experiment. I run that test trial on OCPI2 and count camera pulses from .di file to see if it matches with what's in JSON file and .imagine file. If it works, then I generate the script for the entire experiment. I agree the validation time is probably tolerable if it happens only once for each experiment. I'm just not sure if the validation step is still necessary after the test trial validation.

Cody-G commented 6 years ago

Just fyi, I added a keyword argument to turn off validation. Currently you'll have to check out the cjg/imaginehardware branch to get that. That branch contains lots of other changes, some of which may break your scripts, so I would be careful with it. That branch will eventually become the new version of the package, but it may be a while.

Cody-G commented 6 years ago

Partially addressed by #67 but it's still recommended not to turn validation off unless you're quite sure that you don't have bugs.