QuantConnect / lean-cli

CLI for running the LEAN engine locally and in the cloud
https://www.lean.io/cli
Apache License 2.0
189 stars 101 forks source link

Add new `data download` command to lean-cli for data retrieval #447

Closed Romazes closed 2 months ago

Romazes commented 2 months ago

Description

This pull request introduces a new command, data download, to the lean-cli tool. This command facilitates downloading various data types such as Trade, Quote, and Open Interest from multiple Historical Data Providers like Polygon, Binance, Interactive Brokers, etc. The command supports different resolutions like tick, second, minute, hour, and day, as well as different security types depending on the support available from the Data Provider. The command invokes an executable app from Lean to perform the data download.

Related Issue

https://github.com/QuantConnect/lean-cli/issues/388

Related Pull Request

Motivation and Context

The motivation behind this feature is to enhance the user experience by providing an easy-to-use command for downloading different types of data required for developing trading algorithms in Lean. By incorporating this feature into lean-cli, users can effortlessly download data and proceed with algorithm development without needing to manually handle data retrieval from various providers.

How Has This Been Tested?

This feature has been thoroughly tested to ensure its functionality and reliability:

Additional manual test cases:

Interactive Brokers [Security type: Equity]

lean data download --data-provider-historical "Interactive Brokers" --data-type Quote --resolution Daily --security-type Equity --tickers NVDA,AMD --start-date 20240303 --end-date 20240404
Result:

image

Binance [Security type: Crypto]

lean data download --data-provider-historical Binance --data-type Trade --resolution Daily --security-type Crypto --tickers BTCUSDT,ETHUSDT --start-date 20240303 --end-date 20240404
Result

image

FactSet [Security type: IndexOption]

lean data download --data-provider-historical FactSet --data-type Trade --resolution Daily --security-type IndexOption --tickers SPX --start-date 20240422 --end-date 20240423
Result:

image

Polygon [Security type: Equity]

lean data download --data-provider-historical Polygon --data-type Trade --resolution Hour --security-type Equity --tickers SOFI,TSLA,NIO --start-date 20200101 --end-date 20240404
Result:

image

IEX [Security type: Equity]

lean data download --data-provider-historical IEX --data-type Trade --resolution Hour --security-type Equity --tickers AMD --start-date 20200101 --end-date 20240404
Result: [wrong resolution]

image

IEX [Security type: Equity]

lean data download --data-provider-historical IEX --data-type Trade --resolution Daily --security-type Equity --tickers GOOGL --start-date 20100101 --end-date 20240404

Result

image

Coinbase Advanced Trade [Security type: Crypto]

pay attention --market Coinbase

lean data download --data-provider-historical "Coinbase Advanced Trade" --data-type Trade --resolution hour --security-type Crypto --tickers BTCUSD --market Coinbase --start-date 20230101 --end-date 20240404

image

Example of running

  1. Interactive Mode: Run ⬇ and follow the on-screen instructions to select the Data Provider, Data Type, Resolution, Security Type, tickers, start date, and end date.
    lean data download
  2. Non-Interactive Mode: Run ⬇
    lean data download --data-provider-historical Polygon --data-type Trade --resolution Minute --ticker-security-type Equity --tickers AAPL,QQQ --start-date 20240101 --end-date 20240202

    Conclusion

    The main goal of this feature is to be user-friendly and assist users in downloading various data types required for creating new trading algorithms in Lean. By providing a seamless data retrieval process, users can focus more on algorithm development and less on data management.