cryptofeed + tick_store

[lightandbright]

Arctic Version

1.60

Arctic Store

TICK_STORE

Platform and version

Sierra

Description of problem and/or code sample that reproduces the issue

I retrieve order book data with cryptofeed and would like to store as TickStore. Could you provide few lines of code on this?

I tried to place the TickStore write into the cryptofeed book callback. the write seems to stuck everything, it feels like some async waiting issue.

from arctic import Arctic, TICK_STORE
import pandas as pd
import pytz as tz
from datetime import datetime as dt

z = tz.UTC
a = Arctic('localhost')
a.initialize_library('tick.test', lib_type=TICK_STORE)
l = a['tick.test']

async def book(feed, book):
    for pair in book:
        print('feed {} book bid size is {} ask size is {}'.format(feed, len(book[pair]['bid']), len(book[pair]['ask'])))
        df = pd.DataFrame(data={'data': [1.0, 2.0, 3.0]}, index=[dt(2016, 1, 1, 1, 1, 1, 1, z), dt(2016, 1, 2, 1, 1, 1, 1, z), dt(2016, 1, 3, 1, 1, 1, 1, z)])
        l.write('test1', df)
        l.read('test1')

def main():

    f = FeedHandler()
    f.add_feed(GDAX(pairs=['BTC-USD', 'ETH-USD'], channels=['full'], callbacks={'book': BookCallback(book)}))
    f.run()

if __name__ == '__main__':
    main()

[bmoscon]

Cant say for sure what your error is, but the code works for me. Is mongodb running? You also didn't post what the version of cryptofeed is you are running. Either way, this isnt an arctic issue, so I'm going to close this. If you still have issues please open a ticket on cryptofeed.

[lightandbright]

Thank you for your response. I realised that I was assigning same id which prevented the write.

Btw, why is it better to use arctic rather then directly pymongo? does arctic make things more compressed or faster?

thank you Roman

On 11 Feb 2018, at 19:25, Bryant Moscon notifications@github.com wrote:

Cant say for sure what your error is, but the code works for me. Is mongodb running?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/manahl/arctic/issues/506#issuecomment-364779607, or mute the thread https://github.com/notifications/unsubscribe-auth/AiaZ4qkgUxXrFpvdQfBB9FZ0ztcljsfRks5tTz6QgaJpZM4SBbbw.

[bmoscon]

most those details are on the arctic readme:

Arctic can query millions of rows per second per client, 
achieves ~10x compression on network bandwidth,
~10x compression on disk, and scales to hundreds
of millions of rows per second per MongoDB instance.