Is your feature request related to a problem? Please describe.
The perdayvolume generator creates + stores multiple event lists/dicts while populating the output queue.
Describe the solution you'd like
We should look at using a buffered io stream to mitigate the memory usage.
Describe alternatives you've considered
Re-architect perdayvolume generator? Any alternative would be complex and require more effort
Additional context
This is not currently blocking customers from generating their desired data (hence the feature/improvement tag), but this could drastically reduce the maximum capacity of perdayvolume's generation. We should also measure + any report any performance gain associated with this improvement.
Is your feature request related to a problem? Please describe. The perdayvolume generator creates + stores multiple event lists/dicts while populating the output queue.
Describe the solution you'd like We should look at using a buffered io stream to mitigate the memory usage.
Describe alternatives you've considered Re-architect perdayvolume generator? Any alternative would be complex and require more effort
Additional context This is not currently blocking customers from generating their desired data (hence the feature/improvement tag), but this could drastically reduce the maximum capacity of perdayvolume's generation. We should also measure + any report any performance gain associated with this improvement.