bmoscon / cryptofeed

Cryptocurrency Exchange Websocket Data Feed Handler
Other
2.22k stars 685 forks source link

Reading binary data from MongoDB or Arctic #358

Closed interzonez closed 3 years ago

interzonez commented 3 years ago

How can I read the binary data from within the MongoDB or from within the Arctic store?

Using Mongo Compass all data is in binary form.

I've got this far with python:

`>>> pprint.pprint(cryptofeed2.find_one()) {'_id': ObjectId('5fd8774edae92XXXXXXXXX'), 'compressed': True, 'data': b'H\x00\x00\x00\xf3\x0e\x00.?\xdaV\XXXXXXXXXXXX' b'\x00\x00T\x00\x00\x00M\x00\x00\x00E\XXXXXXXXXXXX' b'\x00\xa1U\x00\x00\x00S\x00\x00\x00D\XXXXXXXXXXXXXXX' b'\x1e\xc6\xd4\XXXXXXXXXXXXX', 'parent': [ObjectId('5fd8774e4264XXXXXXXXX')], 'segment': 0, 'sha': b'8W6\x15\x16jq-Q\x14\xdcp\x05#\XXXXXXXXXXXX', 'symbol': 'open_interest'}

`

I've written over some of this data for privacy sake with XXXXXXX

Now I can get at the data in the base64 format above. How to spit this out in readable form from mongodb / arctic?

bmoscon commented 3 years ago

arctic compresses data so its only readable via the arctic library. if you want raw data stored, use the mongo backends.

interzonez commented 3 years ago

Got it thanks!