Qlib is an AI-oriented quantitative investment platform that aims to realize the potential, empower research, and create value using AI technologies in quantitative investment, from exploring ideas to implementing productions. Qlib supports diverse machine learning modeling paradigms. including supervised learning, market dynamics modeling, and RL.
I have 1TB trade data, turn it into qlib .bin format is quite time and space consuming, and memory is too small to load it once, I want to use dask or polars to support big data that is iarger than my memory. I don't know how, can anyone give some suggestions or examples?
Thx.
I have 1TB trade data, turn it into qlib .bin format is quite time and space consuming, and memory is too small to load it once, I want to use dask or polars to support big data that is iarger than my memory. I don't know how, can anyone give some suggestions or examples? Thx.