search

oanda / v20-python

OANDA v20 bindings for Python
MIT License
225 stars 91 forks source link

Streaming pricing performance #5

Closed alekna closed 8 years ago

alekna commented 8 years ago

I would strongly suggest changing chunk_size from 1 byte to a typical size of a Price packet. This will introduce a very little delay in packet processing but at the same time it will significantly decrease CPU usage.

The line that needs to be changed is init.py#L186

Subscribe to all instruments and you'll see how hard is the performance hit with iter_lines(1)

dmpettyp commented 8 years ago

Good call, not exactly sure why it was set to 1 in the first place. The fix will show up in the next release.

alekna commented 8 years ago

Performance is quite an essential factor when processing streaming rates and Requests/httplib doesn't seem to handle it well.

There is no "correct" way to implement iter_lines() using blocking I/O, as you always have to bite the bullet and take a guess as to how much data you should read. I would strongly suggest implementing other means of streaming pricing. Protobuf would probably be the best option. Is this something you would be willing to consider?

dmpettyp commented 8 years ago

The next version of the v20 library will leave the default intact and allow you to set the chunk size to whatever you desire through the API context.

As for a non-HTTP based streaming solution, that is really a decision for the product team that is based on the demand for the feature. From a technology perspective, protobuf would make a lot of sense (plus we use it extensively internally).

alekna commented 8 years ago

When do you expect to release next version? Could you please include release notes whenever you release a new build?

How can I express my demand for non-HTTP based streaming solution? Is there a formal procedure to do this?

dmpettyp commented 8 years ago

I'll try to get a new release out tomorrow. There should already be release notes in the repo, found at:

https://github.com/oanda/v20-python/blob/master/src/ChangeLog

Let me get back to you on the product request.

alekna commented 8 years ago

There's Changelog in the repo, that's correct. What I was suggesting you is to include same information within the releases page. It is easier to access information this way.

alekna commented 8 years ago

Still waiting for feedback regarding product request.

dmpettyp commented 8 years ago

I've been told that your best options are:

  1. Post to the forum
  2. Make a request though frontdesk@oanda.com
alekna commented 8 years ago

OK. Thanks.