json conversion function for outputting information to CSV files

[phoenix1of1]

Hi, first off, I want to say that the Unicorn Binance Websocket API is really great as it take the mess of the Binance API and structures it in to such a useful tool.

I am currently working on my own little bot and one of the things that became a little annoying was having to convert the raw data in to an exportable and usable CSV format where I can then load the CSV files as Pandas Dataframes for further calculation. The Unicorn_Fy module is awesome for taking the very raw json data and making it more readable but I think an executable function to convert that data further would be time-saving and very useful.

I am a novice Python coder but the following code is currently how I "clean" the raw data in to a usable CSV format:

``def process_message(msg): if msg['e'] == 'error': print("Error, malfunction") time.sleep(3) print("Terminating process") time.sleep(3) sys.exit() else: Data_Type = msg['e']

Datetimeindex = msg["E"] / 1000
Datetimeindex = datetime.fromtimestamp(Datetimeindex).strftime('%Y-%m-%d %H:%M:%S')

Symbol = msg['s']

Interval_Type = msg['k']['i']

Open_Price = msg['k']['o']

Close_Price = msg['k']['c']

High_Price = msg['k']['h']

Low_Price = msg['k']['l']

Volume = msg['k']['v']

Quantity = msg['k']['q']

print("{}, {}, {}, {}, {}, {}, {}, {}, {}, {}".format(Data_Type, Datetimeindex, 
                                                      Symbol, Interval_Type, Open_Price, 
                                                      Close_Price, High_Price, Low_Price, Volume, 
                                                      Quantity), file = "insert file name variable here, flush=True)``

I was wondering if this function could be inserted in to handling the kline streams or at least offer an "enable/disbale" function on a per stream basis via a boolean value which would allow each user to add the json conversion function if they wanted it.

On two side notes to this, the variable tags would need to be amended to offer users the full data-set, I am only interested in nine data points for further analysis.

The second side note is that you have to convert the UNIX timestamp to a proper datetime format as aggregation and resampling of kline data is "buggy" without a correctly formatted datetime to use as an index.

Hopefully this makes sense, I know I am "green" in terms of Python coding but I think the json conversion function would be something that would be really useful.

Either way, thanks for taking the time to think about this and consider it.

Cheers

James

[oliver-zehentleitner]

Hi Phoenix!

Thanks for your words!

I will pick this project and your suggestion up in a few weeks. Contributions are appreciated :)

Best regards, Oliver

[oliver-zehentleitner]

Hello! I think this will be too much work and I think most people are using databases for the bots not CSV ... I am going to create a 1.0 verison which will get incompatible to the current version. If you want to add such a function coded in a generic way I would merge it. Just create a pull request. Best regards, Oliver