The current StreamingFeatherWriter class lacks file rotation functionality, which may lead to oversized files during long-running operations or high data volume scenarios. Additionally, there's room for improvement in time management and file naming.
Suggested Improvements
Implement file rotation functionality, supporting rotation based on size, time interval, and scheduled dates.
Use a Clock object for time management to improve flexibility and testability.
Add timestamps to filenames for better organization and identification.
Track file sizes and creation times to support rotation functionality and provide more information.
Add methods to query current file information and next rotation time.
Expected Benefits
Better file management, avoiding oversized single files.
Improved system reliability and performance.
More flexible time handling, facilitating testing and simulation of different time zones.
Better file organization and information query capabilities.
Issue Description
The current
StreamingFeatherWriter
class lacks file rotation functionality, which may lead to oversized files during long-running operations or high data volume scenarios. Additionally, there's room for improvement in time management and file naming.Suggested Improvements
Clock
object for time management to improve flexibility and testability.Expected Benefits
Related Files
nautilus_trader/persistence/writer.py