Atem18 / kraky

Python asyncio client for Kraken API REST and Kraken Websockets API using httpx and websockets
https://kraky.readthedocs.io/en/latest/
MIT License
26 stars 9 forks source link

Getting "socket.send() raised exception." after some time #70

Closed Kiessar closed 1 year ago

Kiessar commented 2 years ago

Hi there, thank you for the cool tool. I'm trying to gather data with kraky and getting the erro:

socket.send() raised exception.

Here the base of my code to make sure I didn't mess up anything else.

import asyncio
from operator import index
from kraky import KrakyApiAsyncClient, KrakyWsClient
import pandas as pd
import numpy as np
from datetime import datetime
api_depth = 25
def dicttofloat(keyvalue):
        return float(keyvalue[0])
import os
import argparse

ticker_cols= [ "ask_price", "ask_wlot_volume", "ask_lot_volume","bid_price", "bid_wlot_volume", "bid_lot_volume","last_trade_closed_price", "last_trade_closed_lot_volume","volume_today", "volume_24h","vwav_today", "vwav_24h","no_trades_today", "no_trades_24h","low_today", "low_24","high_today", "high_24", "open_today", "open_24"]
ohlc_cols = ["time", "etime", "open", "high", "low", "close", "vwap", "volume", "count"]
class PairData:
    def __init__(self, pair : str) -> None:
        self.pair = pair
        self.datasets = dict()
        self.book = {"bid":{}, "ask":{}}
        self.currentData=None

def save():
[...]

async def public_handler(api_data):

    if type(api_data) == dict:
        if( "event"  in api_data and api_data["event"] == "subscriptionStatus"  and api_data["status"] == "subscribed" ):
            pass

    if type(api_data) == list:
        try:
            if api_data[-1] not in pairdatas:
                pairdatas[api_data[-1]] = PairData(api_data[-1])
            if(api_data[-2].startswith( "book")):
                if  "as" in api_data[1]:
                        api_update_book("ask", api_data[1]["as"],api_data[3] )
                        api_update_book("bid", api_data[1]["bs"],api_data[3] )
                        # signal.alarm(1)
                elif "a" in api_data[1] or "b" in api_data[1]:
                    for x in api_data[1:len(api_data[1:])-1]:
                        if "a" in x:
                            api_update_book("ask", x["a"],api_data[3] )
                        elif "b" in x:
                            api_update_book("bid", x["b"],api_data[3] )
            else:
                pairdatas[api_data[3]].setData(api_data[2], api_data[1])
        except Exception as e:
            print(e)
            print(api_data, type(api_data))

async def main(ws_pairs= ["RARI/USD"] ):

    kraky_public_ws_client = KrakyWsClient("production")

    asyncio.create_task(
            kraky_public_ws_client.connect(public_handler, connection_name="public")    
    )
    await kraky_public_ws_client.subscribe(
        {"name": "book", "depth": api_depth},
        ws_pairs,
        connection_name="public",
    )
    await kraky_public_ws_client.subscribe(
        {"name": "ticker"},
        ws_pairs,
        connection_name="public",
    )
    await kraky_public_ws_client.subscribe(
        {"name": "ohlc", "interval": 1},
        ws_pairs,
        connection_name="public",
    )
    await kraky_public_ws_client.subscribe(
        {"name": "ohlc", "interval": 60},
        ws_pairs,
        connection_name="public",
    )
    await kraky_public_ws_client.subscribe(
        {"name": "ohlc", "interval": 1440},
        ws_pairs,
        connection_name="public",
    )

async def periodic_storeing(time = 10):

    while True:

        await asyncio.sleep(time)

        for a in pairdatas:
            pairdatas[a].save()

def parse_args(pargs=None):

    parser = argparse.ArgumentParser(
        formatter_class=argparse.ArgumentDefaultsHelpFormatter,
        description='Sample for Signal concepts')

    parser.add_argument('--pairs', '-p', default="ALGOXBT",
                        help='Assetpair or comma seperated pairs (no spaces) that should be fetched')
    parser.add_argument('--sleep', '-s', type=int, default= 60,
                        help='time between fetches')

    if pargs is not None:
        return parser.parse_args(pargs)

    return parser.parse_args()

if __name__ == "__main__":
    argss =parse_args()

    pairs = argss.pairs.split(",")
    time  = int(argss.sleep)
    try:
        loop = asyncio.get_event_loop()
        loop.create_task(main(pairs))
        loop.create_task(periodic_storeing(time))
        loop.run_forever()
    except Exception as e:
        loop.close()
Atem18 commented 2 years ago

@Kiessar In your script, the pairs is not "ALGOXBT" but "ALGO/XBT". Also pairdatas is not defined so you should fix that also. Finally when you have the "socket.send() raised exception." at which time does it occurs ? Because I don't see it, maybe it's in your save method that you send something badly formatted.

Kiessar commented 2 years ago

@Atem18 thx for your fast reply. The missing parts are, because I tried to focus on the kraky part. For thefull code please look here: WSDatefetcher

As you may have noticed I tried to put a lot of "try:except" block in the code but I cannot make out where the issue arises. If would need to guess, I'd say it looses connection and getting the error on reconnecting.

Installed packages (if something is wrong with a version I use):

aiodns==2.0.0
aiohttp==3.7.4.post0
anyio==3.5.0
APScheduler==3.6.3
async-timeout==3.0.1
attrs==21.2.0
backcall==0.2.0
backtrader==1.9.76.123
bt-ccxt-store==1.0
ccxt==1.51.21
certifi==2020.12.5
cffi==1.14.5
chardet==4.0.0
charset-normalizer==2.0.12
click==8.0.3
cryptography==3.4.7
cycler==0.10.0
deap==1.3.1
decorator==5.0.9
distlib==0.3.4
empyrical==0.5.5
filelock==3.6.0
Flask==2.0.2
greenlet==1.1.1
h11==0.12.0
httpcore==0.14.7
httpx==0.21.3
idna==2.10
ipython==7.23.1
ipython-genutils==0.2.0
itsdangerous==2.0.1
jedi==0.18.0
Jinja2==3.0.2
joblib==1.0.1
kiwisolver==1.3.1
krakenex==2.1.0
kraky==2022.1.24
lxml==4.6.3
MarkupSafe==2.0.1
matplotlib==3.2.2
matplotlib-inline==0.1.2
multidict==5.1.0
numpy==1.20.1
packaging==21.3
pandas==1.2.2
pandas-datareader==0.9.0
parso==0.8.2
patsy==0.5.1
pexpect==4.8.0
pickleshare==0.7.5
Pillow==8.2.0
platformdirs==2.5.1
pluggy==1.0.0
prompt-toolkit==3.0.18
ptyprocess==0.7.0
py==1.11.0
pycares==4.0.0
pycparser==2.20
pyfolio==0.9.2
Pygments==2.9.0
PyMySQL==1.0.2
pyparsing==2.4.7
python-dateutil==2.8.1
python-telegram-bot==13.5
pytrends==4.7.3
pytz==2021.1
requests==2.25.1
rfc3986==1.5.0
scikit-learn==0.24.1
scipy==1.6.2
seaborn==0.11.1
six==1.15.0
sklearn==0.0
sniffio==1.2.0
SQLAlchemy==1.4.23
statsmodels==0.12.2
stopit==1.1.2
threadpoolctl==2.1.0
toml==0.10.2
torch==1.8.1+cpu
torchaudio==0.8.1
torchvision==0.9.1+cpu
tornado==6.1
tox==3.24.5
TPOT==0.11.7
tqdm==4.60.0
traitlets==5.0.5
typer==0.4.0
typing-extensions==3.10.0.0
tzlocal==2.1
update-checker==0.18.0
urllib3==1.26.3
virtualenv==20.13.4
waitress==2.0.0
wcwidth==0.2.5
websockets==10.2
Werkzeug==2.0.2
xgboost==1.3.3
yarl==1.6.3
Atem18 commented 2 years ago

@Kiessar To be honest, I am not sure why you are getting this error. As you can see here: https://github.com/Atem18/kraky/blob/master/kraky/ws.py If you use my methods, I am catching the websockets errors already. I also tried already to cut my internet connection and resume and it's reconnecting, that's the principle of websockets. Even while trying your script, I tried to disconnect from the internet and reconnect and it's still working. After how many times do you have this error ?

Kiessar commented 2 years ago

@Atem18 I didn't go into the analysis this deep but I setup a new venv and installed only plain kraky. If this works I'm going to do a diff on the verisons and let you know. It seems that error starts between 1 and 6 hours. Because I started a new clean run, I will let you know how long it took. Thank you for your effort and assisstance.

Atem18 commented 2 years ago

@Kiessar no problem and let me know :)

Kiessar commented 2 years ago

@Atem18 ok happened again. After about 6 hours and 20 minutes. What looks also quite strange is, that the code should save values even when they are not updated(due to the error).

2022-03-24 19:33:26.980434,0.613562,
2022-03-24 19:33:47.029422,0.613759,
2022-03-24 20:59:49.383536,0.613759,
2022-03-24 21:00:09.455432,0.613759,
2022-03-24 21:00:29.516644,0.613759,
2022-03-24 21:00:49.596934,0.613759,
2022-03-24 21:01:09.678921,0.613759,
2022-03-24 21:01:29.766926,0.613759,
2022-03-24 21:01:49.852033,0.613759,
2022-03-24 21:02:09.930524,0.613759,
2022-03-24 21:02:30.010645,0.613759,
2022-03-24 21:02:50.085574,0.613759,
2022-03-24 21:03:10.137753,0.613759,
2022-03-24 21:03:30.186664,0.613759,
2022-03-24 21:03:50.245657,0.613759,

But as you can see in the data there is a gap. after that the error happen again. Could be like you said and the fault lies in my code but I'm not familliar enough with asyncio to know how to debug this. Any idea?

Atem18 commented 2 years ago

@Kiessar So you got again a « socket.send() raised exception. » ? Do you have the stack trace ? That way I can know if it failed when trying to reconnect.

Kiessar commented 2 years ago

@Atem18 I have clue but no prrof yet. I switched the log level to DEBUG (not that easy in the current state). What I think is happening: The log level showed that before the socket.send() raised exception. is raised the following exception is thrown: 2022-03-25 14:20:29,793 kraky.ws DEBUG WebSockets connection closed error, let's reconnect anyway... Which refers to: https://github.com/Atem18/kraky/blob/e9b9c9b947d0279eb227a7e238c6930593779776/kraky/ws.py#L73

But why is the errorhappening? My guess: The message rate limit of a single WebSocket API connection will vary depending upon the load on the system. WebSocket API clients will receive the error message {"Error": "Exceeded msg rate"} when the message rate limit is exceeded. Source

I'm using 20 subscriptions in the example and think that I'm running into the message rate limit. Unfortunately i couldn't find a way to show the error message. After that it could be that I run into the connection ban:

- Cloudflare imposes a connection/re-connection rate limit (per IP address) of approximately 150 attempts per rolling 10 minutes. If this is exceeded, the IP is banned for 10 minutes.
- Recommended reconnection behaviour is to (1) attempt reconnection instantly up to a handful of times if the websocket is dropped randomly during normal operation but (2) after maintenance or extended downtime, attempt to reconnect no more quickly than once every 5 seconds. There is no advantage to reconnecting more rapidly after maintenance during cancel_only mode.

Source

I'm not sure what happens than in this part: https://github.com/Atem18/kraky/blob/e9b9c9b947d0279eb227a7e238c6930593779776/kraky/ws.py#L50-L60

What I read about was that if the error socket.send() raised exception. occurs it is because send is called on a socket that is not open anymore. But not sure what happens. I will try to split the subscriptions onto multiple connections, but this will only mitigate the issue. I fixed some issues in the code, if you need it for testing let me know. Should trigger quite quickly if the number of pairs is increased.

Atem18 commented 2 years ago

@Kiessar The main code is the function connect. As you can see, I am doing in infite loop and try catch in case an error occurs on the socket. In the try, I am checking if the socket is open, if it's not, I am reconnecting. If it's open, I get the message. If there is an "errorMessage" in the message, I log it in error and if there is no error, I pass it to the handler function. If you would get the error of the message, it would be logger as an error. In this issue https://github.com/aaugustin/websockets/issues/84, I can see that the person tried to add await asyncio.sleep(0) after websocket.send(...). I will see if I can reproduce the issue. And yes if you have code I can try, it's always nice. :)

Kiessar commented 2 years ago

@Atem18 I splitted it up and use a connection per pair now. The errorchanged now but first let me have you the code: https://pastebin.com/EYD9JFVF Starting with this parameters: nohup python WSDataFetcher.py -p XBT/EUR,XBT/USD,PAXG/ETH,SOL/EUR -s 20 &

The new errors start with:

2022-03-26 04:00:12,974 kraky.ws     DEBUG    WebSockets connection closed error, let's reconnect anyway...
2022-03-26 04:00:12,974 kraky.ws     DEBUG    WebSockets connection closed error, let's reconnect anyway...
2022-03-26 04:00:12,976 kraky.ws     DEBUG    WebSockets connection closed error, let's reconnect anyway...
2022-03-26 04:00:12,978 kraky.ws     DEBUG    WebSockets connection closed error, let's reconnect anyway...
2022-03-26 04:00:13,867 kraky.ws     DEBUG    WebSockets connection closed error, message not sent...
2022-03-26 04:00:13,867 kraky.ws     DEBUG    WebSockets connection closed error, message not sent...
2022-03-26 04:00:13,867 kraky.ws     DEBUG    WebSockets connection closed error, message not sent...

goes on for a long while and finishs with:


2022-03-26 04:58:09,987 kraky.ws     DEBUG    WebSockets connection closed error, message not sent...
--- Logging error ---
Traceback (most recent call last):
  File "/usr/lib/python3.8/asyncio/selector_events.py", line 848, in _read_ready__data_received
    data = self._sock.recv(self.max_size)
MemoryError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.8/asyncio/base_events.py", line 1733, in call_exception_handler
    self.default_exception_handler(context)
MemoryError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.8/logging/__init__.py", line 1085, in emit
    msg = self.format(record)
  File "/usr/lib/python3.8/logging/__init__.py", line 929, in format
    return fmt.format(record)
  File "/usr/lib/python3.8/logging/__init__.py", line 676, in format
    record.exc_text = self.formatException(record.exc_info)
  File "/usr/lib/python3.8/logging/__init__.py", line 626, in formatException
    traceback.print_exception(ei[0], ei[1], tb, None, sio)
  File "/usr/lib/python3.8/traceback.py", line 103, in print_exception
    for line in TracebackException(
MemoryError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.8/asyncio/events.py", line 81, in _run
    self._context.run(self._callback, *self._args)
  File "/usr/lib/python3.8/asyncio/selector_events.py", line 805, in _read_ready
    self._read_ready_cb()
  File "/usr/lib/python3.8/asyncio/selector_events.py", line 854, in _read_ready__data_received
    self._fatal_error(exc, 'Fatal read error on socket transport')
  File "/usr/lib/python3.8/asyncio/selector_events.py", line 705, in _fatal_error
    self._loop.call_exception_handler({
  File "/usr/lib/python3.8/asyncio/base_events.py", line 1740, in call_exception_handler
    logger.error('Exception in default exception handler',
  File "/usr/lib/python3.8/logging/__init__.py", line 1475, in error
    self._log(ERROR, msg, args, **kwargs)
  File "/usr/lib/python3.8/logging/__init__.py", line 1589, in _log
    self.handle(record)
  File "/usr/lib/python3.8/logging/__init__.py", line 1599, in handle
    self.callHandlers(record)
  File "/usr/lib/python3.8/logging/__init__.py", line 1669, in callHandlers
    lastResort.handle(record)
  File "/usr/lib/python3.8/logging/__init__.py", line 954, in handle
    self.emit(record)
  File "/usr/lib/python3.8/logging/__init__.py", line 1093, in emit
    self.handleError(record)
  File "/usr/lib/python3.8/logging/__init__.py", line 1007, in handleError
    traceback.print_exception(t, v, tb, None, sys.stderr)
  File "/usr/lib/python3.8/traceback.py", line 103, in print_exception
    for line in TracebackException(
MemoryError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "WSDataFetcher.py", line 216, in <module>
    loop.run_forever()
  File "/usr/lib/python3.8/asyncio/base_events.py", line 570, in run_forever
    self._run_once()
  File "/usr/lib/python3.8/asyncio/base_events.py", line 1859, in _run_once
    handle._run()
  File "/usr/lib/python3.8/asyncio/events.py", line 85, in _run
    cb = format_helpers._format_callback_source(
  File "/usr/lib/python3.8/asyncio/format_helpers.py", line 24, in _format_callback_source
    source = _get_function_source(func)
MemoryError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "WSDataFetcher.py", line 218, in <module>
    traceback.print_exc()
  File "/usr/lib/python3.8/traceback.py", line 163, in print_exc
    print_exception(*sys.exc_info(), limit=limit, file=file, chain=chain)
  File "/usr/lib/python3.8/traceback.py", line 103, in print_exception
    for line in TracebackException(
MemoryError

Next I will check my code for issues if the memoryError is caused in my domain. Just a question, is it necessary to close the websockets graefully after websockets.open is false?

Atem18 commented 2 years ago

Just a question, is it necessary to close the websockets graefully after websockets.open is false?

Just like any connection, it's better if you can. There is a method that you can call which is disconnect, to close the websocket if you want. If not, you can just kill your program and the connection will be closed anyway. Also I pushed a new version where you can configure the logging level.

Kiessar commented 2 years ago

@Atem18 Just two things I came across while thinking what to do next: I think there is something that cloeses the conncetion form server side (e.g. message limit). My guess why I'm not able to see the error message is, that the connection is not open anymore, so websocket.open returns false but there are still messages in the pipe that could be received. By replacing the websocket, the pipe is droped.

Second thing is, wouldn't it be a good idea to add a sleep in each exception? This could prevent an IP ban, due to too many reconnections.

Atem18 commented 2 years ago

@Kiessar Indeed, that could be an explanation but that would mean that the server is closing the connection before sending you the error. And yes, it would be a good idea, I will add it and add the possibility to change it.

Kiessar commented 2 years ago

@Atem18 Yes but think of the server is sending you like 50 messages and afterwards closing the connection. The connection state is checked after each recv (my assumption is one recv for each message). At some point the connection is already closed while it still tries to process all messages. Is it actually neccessary to check the connection? recv should raise an Exception/Error when pipe is drained and connection is closed.

Atem18 commented 2 years ago

@Kiessar I pushed a new release with support of timeout and sleep_time. Please let me know if it can help.

Kiessar commented 2 years ago

@Atem18 Just a small report. First time it ran about 24h after that the socket.send() was raised again. Unfortunately I had forgot to reset the log level (updating overwrote my changes). I corrected that and now it runs more than 24h. I think if it crashes one more time, I will increase the timeout to 10 seconds. If than still happens anything, I will let you know.

Atem18 commented 2 years ago

@Kiessar No problem. :)

Atem18 commented 2 years ago

@Kiessar Any news ?

Kiessar commented 2 years ago

@Kiessar Any news ?

@Atem18 It crashed again, but some time ago and I hadn't had the time to check on it. I will try to let it run again next day's. Thank you for your interest in this.

Atem18 commented 2 years ago

@Kiessar Ok good, please let me know your findings.

Atem18 commented 2 years ago

I pushed a new version with an updated version of websockets. Maybe it will help.

Kiessar commented 2 years ago

I pushed a new version with an updated version of websockets. Maybe it will help.

Ok I started in the morning but crashed caused by an error of my code. Will give it another try soon and let you know the result. Thanks again

still running...

Kiessar commented 2 years ago

@Atem18 Unfortunately error still persists. Set all the timeouts to 10 and still gettingthe error: socket.send() raised exception.

Atem18 commented 2 years ago

@Kiessar Ok thanks, I will try to reproduce it as well and see if something can be done at the client's side.

Atem18 commented 2 years ago

@Kiessar I tried to run a simple implementation and my script got killed by the OOM killer.

Did you also got an increase of memory over time ?

Kiessar commented 2 years ago

@Atem18 Yes, one time I got an memory error, but I thought it was caused by my own implementation.

Atem18 commented 2 years ago

Good to know but in that case, it's just simply listening the websocket and printing the output.

Kiessar commented 2 years ago

@Atem18 Because I'm starting to need the functionality more and more I will start to dive in more and try to solve the issue. My latest debug log is the following:

2022-06-15 08:14:59,008 kraky.ws     DEBUG    WebSockets connection closed error - retrying connection in 5 sec.
2022-06-15 08:14:59,028 kraky.ws     DEBUG    WebSockets connection closed error - retrying connection in 5 sec.
2022-06-15 08:14:59,110 kraky.ws     DEBUG    WebSockets connection closed error - retrying connection in 5 sec.
2022-06-15 08:14:59,137 kraky.ws     DEBUG    WebSockets connection closed error - retrying connection in 5 sec.
2022-06-15 08:14:59,298 kraky.ws     DEBUG    WebSockets connection closed error - retrying connection in 5 sec.
socket.send() raised exception.
socket.send() raised exception.
socket.send() raised exception.
socket.send() raised exception.

The memory error probably does occur because it prints endlessly multiple times per second the line "socket.send() raised exception.". The memory error is therefore probably not the cause but just another symptom.

Atem18 commented 2 years ago

@Kiessar For the memory error, yes it's probably that.

If I understand well, you are opening 5 websockets connections, right ?

And 4 of them did not reconnected ?

Kiessar commented 2 years ago

@Atem18 Sorry didn't make that clearly enough. It goes repeating the line

"socket.send() raised exception."

for about 500mB (so thousands of lines repeated). So the number of lines was just arbitrary cut off and are more than those 4 (so i guess that all of them don't reconnect)

Atem18 commented 2 years ago

@Kiessar Ok I see what you mean.

So maybe it's linked to https://github.com/aaugustin/websockets/issues/84 .

I will try to add await asyncio.sleep(0) after websocket.send(...) and see if it will help.

Atem18 commented 2 years ago

@Kiessar A new release is up with the sleep after the send. Can you please test it and tell me ?

Thanks a lot in advance.

Kiessar commented 2 years ago

@Atem18 New error and it crashed my server due to overload and diskspace consumption:

2022-06-20 01:54:53,144 kraky.ws     DEBUG    WebSockets connection closed error - retrying connection in 20 sec.
2022-06-20 01:56:13,824 kraky.ws     DEBUG    WebSockets connection closed error - message not sent.
2022-06-20 01:56:13,824 kraky.ws     DEBUG    WebSockets connection closed error - message not sent.
2022-06-20 01:56:13,824 kraky.ws     DEBUG    WebSockets connection closed error - message not sent.
2022-06-20 01:56:13,825 kraky.ws     DEBUG    WebSockets connection closed error - message not sent.
2022-06-20 01:56:13,825 kraky.ws     DEBUG    WebSockets connection closed error - message not sent.
2022-06-20 01:56:13,825 kraky.ws     DEBUG    WebSockets connection closed error - message not sent.
2022-06-20 01:56:13,825 kraky.ws     DEBUG    WebSockets connection closed error - message not sent.
2022-06-20 01:56:13,825 kraky.ws     DEBUG    WebSockets connection closed error - message not sent.
[...]

Diskspace was used because the error was written in the log with an insane rate.

Atem18 commented 2 years ago

@Kiessar It seems another error this time. Unless I am wrong, it went into an infinite loop.

Let me check again my logic and see if I can find the flaw.

Atem18 commented 2 years ago

@Kiessar quick question, can I have a snippet of your code that raise the issue ?

Thanks a lot in advance.

Kiessar commented 2 years ago

@Atem18 Script: https://pastebin.com/scZ0qA27 Command: nohup python WSDataFetcher.py -p RARI/EUR,ADA/EUR,PAXG/ETH,SOL/EUR,RARI/XBT,XBT/EUR,ADA/USD -s 10 &

My best guess is still, that it runs into rate limits. Therefore it highly depends on the current market usage. If I'm correct adding more pairs would lead to quicke errors.

Atem18 commented 2 years ago

@Kiessar Thanks I will have a look.

Also there is still the issue that at some point, the memory usage is increasing but now that I have proper monitoring I can see that it’s not increasing over time but in less than 5 minutes !

grafana atemlaroute landSIKovxCnkzabbix-template-linux-server-customorgId=1 from=1655936100000 to=1655937900000

So maybe there is a bug in my library, maybe there is one in the websockets library but maybe Kraken’s Websocket API have some issues that the libraries are not enjoying.

So let’s see if we can find anything useful…

Kiessar commented 2 years ago

@Atem18 Are you sure thats not my code whats causing the memory consumption? Just don't want to waste your time. If you like I could strip down everything regarding storing the data, so that only the connection is used to get data.

Atem18 commented 2 years ago

@Kiessar Yes, I also have the issue with the example in my README.md

But yes, if you can test/monitor on your side, it would be great !

Atem18 commented 2 years ago

Hi @Kiessar, do you still have the issue ?

Thanks a lot in advance.

Kiessar commented 2 years ago

Hi @Atem18 , I will check in coming days and update as soon as I have news.

Atem18 commented 2 years ago

@Kiessar Thanks a lot

Kiessar commented 1 year ago

Hi @Atem18, sorry for the late reply: It ran for about 4 days and than into the same situation:

2022-09-16 19:28:38,741 kraky.ws     DEBUG    WebSockets connection closed ok - retrying connection in 20 sec.
2022-09-16 19:29:59,465 kraky.ws     DEBUG    WebSockets connection closed error - message not sent.
2022-09-16 19:29:59,465 kraky.ws     DEBUG    WebSockets connection closed error - message not sent.
2022-09-16 19:29:59,466 kraky.ws     DEBUG    WebSockets connection closed error - message not sent.
2022-09-16 19:29:59,466 kraky.ws     DEBUG    WebSockets connection closed error - message not sent.
2022-09-16 19:29:59,466 kraky.ws     DEBUG    WebSockets connection closed error - message not sent.
2022-09-16 19:29:59,466 kraky.ws     DEBUG    WebSockets connection closed error - message not sent.
[...]

Still thinking, it is some rate limit issue that is building up

Atem18 commented 1 year ago

@Kiessar Thanks for the feedback. Yes, there could be some rate limit from Kraken.

I propose to close this issue as there is no longer the error "socket.send() raised exception".