man-group / arctic

High performance datastore for time series and tick data
https://arctic.readthedocs.io/en/latest/
GNU Lesser General Public License v2.1
3.05k stars 583 forks source link

Data visualisation of datasets stored with arctic #916

Open tomnewg opened 3 years ago

tomnewg commented 3 years ago

Hi guys,

first of all thank you so much for arctic and all the work that has been put into it. Such an awesome tool.

I have been using it for 1.5 years. I am currently trying to make the data accessible to non-programmers.

  1. Is it possible to use the data stored with arctic? I.e. I know one can link Power BI with the help of a connector to mongo db. Would I be able to access the datasets through that or are they stored in a certain way that requires arctic to read it out?
  2. if yes what software would you use? I was thinking about Power BI but it is quite limited in some operations I want to do for example a drop down where a user can choose different datasets.
  3. If no what other options do I have storing pandas datasets and using some tools to build user interfaces. I don't want to reinvent the wheel as there are a lot of tools out there.

Many thanks for your help or any direction you can point me to.

Tom

dunckerr commented 3 years ago

hi Tom - thanks for your interest in arctic. The data that arctic writes is written in a format that arctic understands, so the easiest tool for reading and writing arctic data is arctic itself. In theory you could read the data out of the arctic data store with a non arctic client, but that would require you to rewrite the arctic read layer.
I suspect what youre asking is how would one get data from arctic into some third party system, like PowerBI? If that's what you'd like to do, you'd need to find out if PowerBI has a python access library. If so, then you could write a program to read/write data using arctic, then export it immediately into PowerBI (or any other system) using the python API provided by that third party tool. I hope that makes sense. Duncan