Nautilus SQL schema and database connection

[filipmacek]

Feature Request

• [ ] add SQL schema for Nautilus trader objects and types
• [ ] add appropriate script/actions that will be used in init and drop of schema when connected to the target database
• [ ] create a rust database object with sqlx and export it with pyo3
• [ ] implement a new cache class that will be based on SQL (sqlite, postgres) so we can save both historical, live cached data and strategy data in relational SQL database

[roguespyder]

Is this request for storing data in postgres instead of the parquet catalog? Maybe a beginner question, but does Nautilus have a data connector for databases such as snowflake, databricks, and most importantly postgres ( my need ) to read/write to/from?

[cjdsellers]

Hi @roguespyder

Thanks for your interest.

The short answer is, yes data is in scope for the SQL implementation of the cache database backing - but there will be an initial focus on the execution objects such as instruments, orders and positions etc. Parquet is serving its purpose well as a data store, but we'll potentially look at expanding to SQL in the future if it makes sense.

There are no cloud style connectors current and they're very likely to remain out of scope for the open-source project.

Hope that helps.

[roguespyder]

Thanks for the explanation. My 1.0 project with IBKR's API and full data is stored into postgres for 10k symbols. I look forward to this enhancement as I try to leverage this framework within my ecosystem. ( since I work with cloud dbs and new tech , i threw those in there :D )

[muratbeser]

I would suggest checking https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.to_sql.html