search

Leo4815162342 / dukascopy-node

✨ Download historical price tick data for Crypto, Stocks, ETFs, CFDs, Forex via CLI and Node.js ✨
https://dukascopy-node.app
MIT License
350 stars 66 forks source link

Please add an option：skip bad data and do not stop converting #29

Closed extremk closed 3 years ago

extremk commented 3 years ago

Please add an option：skip bad data and do not stop converting I Need this option

Leo4815162342 commented 3 years ago

@extremk how would do you define bad data?

extremk commented 3 years ago

image

extremk commented 3 years ago

image

Leo4815162342 commented 3 years ago

@extremk thanks for the screenshots, I'll take a look and try to patch it

extremk commented 3 years ago

Please fix this program. According to my test, all the quotations are not completely normal，the program will go wrong.

dukascopy-cli -i ussc2000idxusd -from 2018-06-01 -to 2018-06-02 -t tick -fl true -f csv from 2018-06-01 to 2018-06-02
All quotations are flat, the program does not get any data, the program will report an error Cannot read property '0' of undefined

dukascopy-cli -i xxxx(many Symbols) -from 2000-01-01 -to 2001-01-01 -t tick -f csv The start and end times are the same day, or not within the legal range, so that the start and end times become the same day, and the program will fail

The program downloaded Bi5 has decompression errors,it will stop running, and I hope to continue running even if errors

Leo4815162342 commented 3 years ago

@extremk the issue with empty binary files has been fixed and new version has been published. Please update dukascopy-cli to version 1.3.2.

Regarding dukascopy-cli -i xxxx(many Symbols) -from 2000-01-01 -to 2001-01-01 -t tick -f csv:

When downloading tick data, please use lower time ranges (ideally for tick I'd recommend not going higher than one day).

Year worth of tick data with one call will most likely fail, as dukascopy servers are rate-limiting the incoming requests that our library makes. At least try increasing the value of --batch-pause from 1000 (1 second - which is default) to something higher and try changing the number of --batch-size to a smaller value (https://github.com/Leo4815162342/dukascopy-tools/tree/master/packages/dukascopy-cli#options)