v20.errors.V20ConnectionError

[BernardRenaud]

Dear Yves,

you are already aware about the connection error that Oanda raise after a certain amount of time. I post it here in order to track it. At the same time, I have sent an email to Oanda and will get back to you if I get any news from their side.

  File "/root/miniconda3/lib/python3.8/site-packages/tpqoa/tpqoa.py", line 279, in stream_data

    for msg_type, msg in response.parts():

  File "/root/miniconda3/lib/python3.8/site-packages/v20/response.py", line 57, in parts

    raise V20ConnectionError(self.path)

v20.errors.V20ConnectionError: Connection to v20 REST server at https://stream-fxpractice.oanda.com:443/v3/accounts/xxxxxxx/pricing/stream?instruments=EUR_USD&snapshot=True failed

[elvisbaugh]

I have been experiencing this exact issue as well. I have gotten this error two days in a row.

[aistalker]

I have just started using tpqoa recently and am having this error also. The connection drops after many exceptions starting with urllib3, then requests, then v20.

Traceback (most recent call last):
  File "C:\Users\Administrator\anaconda3\lib\site-packages\urllib3\response.py", line 697, in _update_chunk_length
    self.chunk_left = int(line, 16)
ValueError: invalid literal for int() with base 16: b''

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\Administrator\anaconda3\lib\site-packages\urllib3\response.py", line 438, in _error_catcher
    yield
  File "C:\Users\Administrator\anaconda3\lib\site-packages\urllib3\response.py", line 764, in read_chunked
    self._update_chunk_length()
  File "C:\Users\Administrator\anaconda3\lib\site-packages\urllib3\response.py", line 701, in _update_chunk_length
    raise InvalidChunkLength(self, line)
urllib3.exceptions.InvalidChunkLength: InvalidChunkLength(got length b'', 0 bytes read)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\Administrator\anaconda3\lib\site-packages\requests\models.py", line 753, in generate
    for chunk in self.raw.stream(chunk_size, decode_content=True):
  File "C:\Users\Administrator\anaconda3\lib\site-packages\urllib3\response.py", line 572, in stream
    for line in self.read_chunked(amt, decode_content=decode_content):
  File "C:\Users\Administrator\anaconda3\lib\site-packages\urllib3\response.py", line 793, in read_chunked
    self._original_response.close()
  File "C:\Users\Administrator\anaconda3\lib\contextlib.py", line 131, in __exit__
    self.gen.throw(type, value, traceback)
  File "C:\Users\Administrator\anaconda3\lib\site-packages\urllib3\response.py", line 455, in _error_catcher
    raise ProtocolError("Connection broken: %r" % e, e)
urllib3.exceptions.ProtocolError: ("Connection broken: InvalidChunkLength(got length b'', 0 bytes read)", InvalidChunkLength(got length b'', 0 bytes read))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\Administrator\anaconda3\lib\site-packages\v20\response.py", line 52, in parts
    for line in self.lines:
  File "C:\Users\Administrator\anaconda3\lib\site-packages\requests\models.py", line 797, in iter_lines
    for chunk in self.iter_content(chunk_size=chunk_size, decode_unicode=decode_unicode):
  File "C:\Users\Administrator\anaconda3\lib\site-packages\requests\models.py", line 756, in generate
    raise ChunkedEncodingError(e)
requests.exceptions.ChunkedEncodingError: ("Connection broken: InvalidChunkLength(got length b'', 0 bytes read)", InvalidChunkLength(got length b'', 0 bytes read))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\Administrator\Desktop\Live\hedger.py", line 176, in <module>
    hedger.stream_data(hedger.instrument) # trading
  File "C:\Users\Administrator\anaconda3\lib\site-packages\tpqoa\tpqoa.py", line 279, in stream_data
    for msg_type, msg in response.parts():
  File "C:\Users\Administrator\anaconda3\lib\site-packages\v20\response.py", line 57, in parts
    raise V20ConnectionError(self.path)
v20.errors.V20ConnectionError: Connection to v20 REST server at https://stream-fxtrade.oanda.com:443/v3/accounts/001-001-5839177-001/pricing/stream?instruments=EUR_USD&snapshot=True failed

[yhilpisch]

This is not an issue with tpqoa itself but rather with the underlying v20 package.

We are nevertheless working on including a more stable and robust way to run socket connections (automatically reconnecting sub-thread that is run asynchronously).

[elvisbaugh]

I had that issue as well. It is not an issue with tpqoa but rather with urllib3. I found a fix here: https://github.com/psf/requests/issues/4248. you will need to add this line of code to your response.py file inside _update_chunk_length(self) function line = (len(line)>0 and line or "0")

[aistalker]

I had that issue as well. It is not an issue with tpqoa but rather with urllib3. I found a fix here: psf/requests#4248. you will need to add this line of code to your response.py file inside _update_chunk_length(self) function line = (len(line)>0 and line or "0")

I actually already tried that. It did not fix it. My code would stop randomly, and no longer give me an error traceback.

[aistalker]

Curious if there have been any updates to this in the last few months.

[yhilpisch]

There will be an update of the code soon that addresses the issue.

[mcar33]

Any update on fixing the streaming issue on the V20 with oanda. I am using it and it keeps dropping me out also. Could be 1-4 times a day.

[yhilpisch]

Have pushed an update. There is a new method for streaming. Usage would be like this:

from time import sleep
from tpqoa import tpqoa

def callback(instrument, time, bid, ask):
    spread = ask - bid
    print(instrument, time, bid, ask, spread)

def main():
    tpqoa_wrapper = tpqoa('oanda.cfg')
    # tpqoa_wrapper.stream_data('EUR_USD')
    price_stream_thread = tpqoa_wrapper.stream_data_failsafe('EUR_USD', callback=callback)

    while True:
        print('Doing something else')
        sleep(10)

if __name__ == '__main__':
    main()

[mcar33]

Ok looks great ty. All I need to do is update the anaconda with the first 2 lines of code and that's it or do I have to add the rest of the lines to my algo program also?

[yhilpisch]

You first need to execute pip install --upgrade git+https://github.com/yhilpisch/tpqoa.

The you can use the example to adjust your code. I do not know how your code looks like.

[mcar33]

Ok. Thanks a million for your help. Very appreciated.

[Edgar10555]

Yves, Many thanks! The stop streaming argument still works in a similar way? I can't seem to close the streaming after using the new method.

[yhilpisch]

No, this is not as trivial as before anymore. A separate, independent Python process needs to be stopped. This is not yet implemented.

[Tormeygit]

Thanks, Yves. Any chance that a stop streaming argument will be implemented to the new streaming method? If not, what would be an example of a simple method to stop?

[ramanathanhari]

The Job model needs a hook to interrupt the stream and exit. We will need to wrap that hook within a stop_streaming method. This is not very trivial and we will try to implement it in the future.

[emmpiiee]

hi all! already an update on this hook to interrupt the stream and exit?

[yhilpisch]

This is still open/not implemented.

[Nohealz]

@yhilpisch You mention the issue is with the underlying v20 package. It looks like v20 isn't supported anymore... could we collab to figure out what needs updated and make those changes to our local files?

This is not an issue with tpqoa itself but rather with the underlying v20 package.

We are nevertheless working on including a more stable and robust way to run socket connections (automatically reconnecting sub-thread that is run asynchronously).

[Nohealz]

I have been using the failsafe stream, and I know urllib3 has made many updates in the last couple years. My error looks a little different than what it used to.

Traceback (most recent call last):
  File "C:\ProgramData\Anaconda3\lib\site-packages\urllib3\response.py", line 444, in _error_catcher
    yield
  File "C:\ProgramData\Anaconda3\lib\site-packages\urllib3\response.py", line 828, in read_chunked
    self._update_chunk_length()
  File "C:\ProgramData\Anaconda3\lib\site-packages\urllib3\response.py", line 758, in _update_chunk_length
    line = self._fp.fp.readline()
  File "C:\ProgramData\Anaconda3\lib\socket.py", line 704, in readinto
    return self._sock.recv_into(b)
  File "C:\ProgramData\Anaconda3\lib\ssl.py", line 1241, in recv_into
    return self.read(nbytes, buffer)
  File "C:\ProgramData\Anaconda3\lib\ssl.py", line 1099, in read
    return self._sslobj.read(len, buffer)
ConnectionResetError: [WinError 10054] An existing connection was forcibly closed by the remote host

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\ProgramData\Anaconda3\lib\site-packages\requests\models.py", line 816, in generate
    yield from self.raw.stream(chunk_size, decode_content=True)
  File "C:\ProgramData\Anaconda3\lib\site-packages\urllib3\response.py", line 624, in stream
    for line in self.read_chunked(amt, decode_content=decode_content):
  File "C:\ProgramData\Anaconda3\lib\site-packages\urllib3\response.py", line 857, in read_chunked
    self._original_response.close()
  File "C:\ProgramData\Anaconda3\lib\contextlib.py", line 137, in __exit__
    self.gen.throw(typ, value, traceback)
  File "C:\ProgramData\Anaconda3\lib\site-packages\urllib3\response.py", line 461, in _error_catcher
    raise ProtocolError("Connection broken: %r" % e, e)
urllib3.exceptions.ProtocolError: ("Connection broken: ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None)", ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\ProgramData\Anaconda3\lib\site-packages\v20\response.py", line 52, in parts
    for line in self.lines:
  File "C:\ProgramData\Anaconda3\lib\site-packages\requests\models.py", line 865, in iter_lines
    for chunk in self.iter_content(
  File "C:\ProgramData\Anaconda3\lib\site-packages\requests\models.py", line 818, in generate
    raise ChunkedEncodingError(e)
requests.exceptions.ChunkedEncodingError: ("Connection broken: ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None)", ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\ProgramData\Anaconda3\lib\site-packages\tpqoa\tpqoa.py", line 373, in _stream_data_failsafe_thread
    self.stream_data(args[0], callback=args[1])
  File "C:\ProgramData\Anaconda3\lib\site-packages\tpqoa\tpqoa.py", line 346, in stream_data
    for msg_type, msg in response.parts():
  File "C:\ProgramData\Anaconda3\lib\site-packages\v20\response.py", line 57, in parts
    raise V20ConnectionError(self.path)
v20.errors.V20ConnectionError: Connection to v20 REST server at https://stream-fxpractice.oanda.com:443/v3/accounts/101-001-19867486-002/pricing/stream?instruments=EUR_USD&snapshot=True failed

[yhilpisch]

@yhilpisch You mention the issue is with the underlying v20 package. It looks like v20 isn't supported anymore... could we collab to figure out what needs updated and make those changes to our local files?

Where did you see that it should not be supported anymore?

[yhilpisch]

I have been using the failsafe stream, and I know urllib3 has made many updates in the last couple years. My error looks a little different than what it used to.

It seems that the remote API server is forcibly closing the connection several times. Does this happen regularly ("always") or just sometimes?

[Nohealz]

@yhilpisch You mention the issue is with the underlying v20 package. It looks like v20 isn't supported anymore... could we collab to figure out what needs updated and make those changes to our local files?

Where did you see that it should not be supported anymore?

I just was assuming since there are open issues on their github with people saying the project is abandoned 🤷. And the source files haven't been updated in the last 5 years.

As far as the server connection issues go, It happens in irregular intervals and about a dozen times a day. It's worth mentioning that I've recently also experienced random timeout issues using 'get_transactions()'. And sometimes retrieving activity on the Oanda iPhone app errors out periodically since their last updates were made. Maybe they need some backend upgrades.

I still see this error as well.

Traceback (most recent call last):
  File "C:\Users\Administrator\anaconda3\lib\site-packages\urllib3\response.py", line 697, in _update_chunk_length
    self.chunk_left = int(line, 16)
ValueError: invalid literal for int() with base 16: b''