I guess first I would ask what exactly you mean by a "batch" of data? I'm guessing that this would mean processing multiple "streams" of data (i.e. batches) at once.
At the moment, RTNeural models only support processing a single input stream for a single "timestep" at one time (the meaning of "timestep" may vary depending on your application). There's two reasons for this: first all optimizations are designed to happen "across" one timestep, rather than across multiple timesteps or batches, so if you do need to process multiple timesteps or batches at one time, you should be able to do so pretty easily with a for loop. The other reason is that RTNeural is designed so that users can construct their own network architectures when needed, so for those purposes it's sometimes necessary to give the user control over what RTNeural is doing at the level of a single batch or timestep.
That said, if you have some ideas for a clean, zero-overhead abstraction for doing batch processing, that's definitely something we could add to RTNeural!
Hello!
I guess first I would ask what exactly you mean by a "batch" of data? I'm guessing that this would mean processing multiple "streams" of data (i.e. batches) at once.
At the moment, RTNeural models only support processing a single input stream for a single "timestep" at one time (the meaning of "timestep" may vary depending on your application). There's two reasons for this: first all optimizations are designed to happen "across" one timestep, rather than across multiple timesteps or batches, so if you do need to process multiple timesteps or batches at one time, you should be able to do so pretty easily with a
forloop. The other reason is that RTNeural is designed so that users can construct their own network architectures when needed, so for those purposes it's sometimes necessary to give the user control over what RTNeural is doing at the level of a single batch or timestep.That said, if you have some ideas for a clean, zero-overhead abstraction for doing batch processing, that's definitely something we could add to RTNeural!