Failed to establish a new connection: [Errno -3] Temporary failure in name resolution')': /botxxxxxxxxxxxxxxxxxxxxxxxx/getUpdates

[rooom13]

Issue I am facing

I'm running a telegram bot in a AWS ec2 instances. In the instance there's only the bot docker container & its database container. They barely (< 10%) consume CPU or Memory.

Suddenly, after 3-7 days of running without any issues, the CPU raises up to 99% and whole instance freezes making the application suck. (I can't even ssh in there)

Investigating the issue I could see that there are some WARNING logs that the application spitted, exactly at the time the CPU starts raising.

This is the log:

telegram.vendor.ptb_urllib3.urllib3.connectionpool - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None)) after connection broken by 'NewConnectionError('<telegram.vendor.ptb_urllib3.urllib3.connection.VerifiedHTTPSConnection object at 0x7fe290ef2520>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution')': /botxxxxxxxxxxxxxxxxxxxxxxxx/getUpdates

This log actually happens 3 times in a row during 1 h until the instance is completely frozen.

I know this actually sounds more like a consequence than a cause but wanted to document it here to see if anybody has any idea or saw the same thing :)

Thanks beforehand

Traceback to the issue

Failed to establish a new connection: [Errno -3] Temporary failure in name resolution')': /botxxxxxxxxxxxxxxxxxxxxxxxx/getUpdates

Related part of your code

No response

Operating System

Linux (FROM python:3.8)

Version of Python, python-telegram-bot & dependencies

python-telegram-bot 13.11 (1.0.7-2-g29da7c5)
Bot API 5.7
certifi 2021.10.08
Python 3.8.12 (default, Jan 15 2022, 18:39:47)  [GCC 7.5.0]

[harshil21]

Hi,

Unfortunately, there is not much PTB can do to fix network issues, you can have a look at this wiki page which will tell you about ways to handle/avoid these errors.

[KompetenzAirbag]

I have the same issue. So I have coded a trading bot and he downloads every 5mins price data. But after 1-5 days of running clean without any errors I suddenly get the same error. It completely breaks the bot because the code freezes. Unfortunately the wiki for handling errors is not helping much at all. I never had this error before and the bot has been running for months now but since 3 weeks I get this error. I didn't update any lib nor the code.

[rooom13]

@KompetenzAirbag

I have the same issue. So I have coded a trading bot and he downloads every 5mins price data. But after 1-5 days of running clean without any errors I suddenly get the same error. It completely breaks the bot because the code freezes. Unfortunately the wiki for handling errors is not helping much at all. I never had this error before and the bot has been running for months now but since 3 weeks I get this error. I didn't update any lib nor the code.

Are you running it in AWS EC2 as well? Does the CPU spikes up as well?

[KompetenzAirbag]

I actually don't know if my CPU spikes. I'm running it on my own server (raspberrypi).

[KompetenzAirbag]

I asked in the telegram group and someone had the idea to re-initialize the updater on Exception which I'm currently testing. I will keep you updated.

[KompetenzAirbag]

Hey man, I tried to reinitialize the updater on Exception and it worked! The bot continues to work after that like normally and the code doesn't freeze any more!

[rooom13]

Hey man, I tried to reinitialize the updater on Exception and it worked! The bot continues to work after that like normally and the code doesn't freeze any more!

Excellent, glad that it worked!, Which exception did you use in your except statement? Would you mind sharing the piece of code?

[KompetenzAirbag]

def download_candles():
    global updater
    while True:
        try:
            strategy(symbol)
        except:
            updater = None
            updater = Updater(config.tele_key)
            continue
        break

I just use a while true and try statement. The download_candles() function gets called every 5min because i download 5min candlestick data. The try statement executes the strategy() function and in that function I got my whole download data, technical analysis stuff and obviously broker api stuff.

[rooom13]

Hmm, there's something I don't understand. Why would your strategy function affect the Updater connection? It doesn't make sense to me right now 🤔

[KompetenzAirbag]

Oh right i forgot to specify, in the strategy() function there is a bunch of telegram stuff etc sending messages when entering a position etc. I also have a second loop for polling new messages (commands like "whats my portfolio worth" etc) but its just the exact same try statement

[rooom13]

I see, have you thought of instantiating the updater in the error handler

    def error_handler(): # this method reinstantiates the Updater
        ...

     dispatcher.add_error_handler(error_handler)

[KompetenzAirbag]

What exactly does the telegram error handler do? Does it auto reconnects the updater? And where do i call the function? I honestly have never worked with error handlers before.

[KompetenzAirbag]

Oh i think i understand You can set an error handler that gets executed on every error so i dont need that try thing right?

[rooom13]

Corect See an example in one of my repos -> https://github.com/rooom13/dailyWord-bot/blob/master/daily_word_bot/app.py#L334

There I might check the error/exception type and reinitialize the updater. In my case is a method so I can access the updater as it is an attributed of the class. Maybe in your case you set it up as a global variable

[KompetenzAirbag]

Corect See an example in one of my repos -> https://github.com/rooom13/dailyWord-bot/blob/master/daily_word_bot/app.py#L334

There I might check the error/exception type and reinitialize the updater. In my case is a method so I can access the updater as it is an attributed of the class. Maybe in your case you set it up as a global variable

But technically I don't need an error type right? I can just reinitialize the updater as single line in the error_handler can't I?

[stale[bot]]

This issue has been automatically closed due to inactivity. Feel free to comment in order to reopen or ask again in our Telegram support group at https://t.me/pythontelegrambotgroup.