Python3: run by python3
command.
aiohttp: pip3 install aiohttp
collect.sh [exchange] [symbols separated by comma.] [output path]
example: collect.sh binancefutures btcusdt,ethusdt /mnt/data
collect.sh binancefuturescoin btcusd_perp /mnt/data
AWS tokyo region is recommended to minimize latency.
Python3: run by python3
command.
pandas: pip3 install pandas
`convert.sh [-s INITIAL_MARKET_DEPTH_SNAPSHOT] [-f] [-c] src_file dst_path
option:
with -f: including mark price, funding, book ticker streams
without -f: only market depth and trade streams
with -c: correct exchange timestamp to be monotonically increased
example:
convert.sh /mnt/data/btcusdt_20220811.dat /mnt/data
or with the initial market depth snapshot
convert.sh /mnt/data/btcusdt_20220811.dat /mnt/data -s /mnt/data/btcusdt_20220810.snapshot.pkl
/mnt/data/btcusdt_20220810.snapshot.pkl
is End-Of-Day market depth snapshot of 20220810 so it's initial market depth snapshot of 20220811.