search

NeuroBench / neurobench

Benchmark harness and baseline results for the NeuroBench algorithm track.
https://neurobench.readthedocs.io
Apache License 2.0
52 stars 12 forks source link

Regression Dataloaders #94

Closed korneelf1 closed 11 months ago

korneelf1 commented 11 months ago

Primate reaching: Standardize output to 3D version, 2D version corresponds to lookback window of 1 timestep, this makes it consistent to the NeuroBench standard of (batch, number of steps, *).

MackeyGlass: Augment dataloader with lookback window functionality, similar to primate reaching

@jasonlyik @dlits21 @denkle @YounesBouhadjar Test compatibility with existing models, update if necessary.

jasonlyik commented 11 months ago

@YounesBouhadjar @denkle The lookback window we are discussing in the email thread corresponds to the primate reaching "3D" mode, which formats data batches with dimensions (# predictions to make, bin_width, feature). No history (only one sample given), as currently implemented by the MG dataloader, would here correspond to (# samples, 1, value), which is still different from what you were mentioning in #78 @YounesBouhadjar.

I think we should update both the MG and primate reaching dataloaders to follow a consistent parameterization of bin_width, rather than mode = 2D / 3D, where bin_width refers to number of input points per prediction, and is 1 for the 2d mode

jasonlyik commented 11 months ago

Changed for MG in #104, will be done for primate reaching in #88