Refactor Telegram bot

[ttpro1995]

Telegram bot does not reply anything to command.

Tried:

/start /help /progress

[yudilee]

sometimes bot get some error while running script that will need it to be restarted. You can join our discord server to get link to our other telegram bot or get info about bot or maybe reporting issue regarding bot or this project. Currently we have 5 telegram bot so if 1 bot is offline you can try your luck by using other bot. note: as for now i have already restarted the bot so you can tried it again. Thanks for informing us

[ttpro1995]

I suggest using celery.

The bot is only listener, it have nothing to do with crawler. By that way, crawler error cannot kill bot.

When bot receive command, it will dispatch task (user or chat id, the command) to Celery. Celery then execute task by calling lightnovel-crawler.

When task done, Celery will send message to user (by user id or chat id) with result.

Each task should call an independent instant of lightnovel-crawler. Thus, it does not require manually restart when thing go wrong.

[yudilee]

Really great idea, i have know celery from spotify downloader bot before. really interesting if it can be applied in telegram bot. it can serve multiple user simultaneously because asynchronous and we can add more machine into single bot instance. have read the guide before. i think telegram bot need to be rewritten using asynchronous method before applied celery. @ttpro1995 do you know any example celery with python telegram bot?

[ttpro1995]

You can use this https://github.com/dokku/dokku (15$/month server can handle that) because you will need multiple worker, which is expensive with heroku.

1. The bot itself, who listening to user chat. The bot will send user id and command to the Queue.
2. Celery with lightnovel-crawler code base. Only Celery should be runninng. Celery will listening to the Queue .Celery will call lightnovel-crawler function to execute command. Each call should be independent and with try-except wrapper.

For the queue, I suggest using RabitMQ (https://www.cloudamqp.com/, free plan work fine). Or you can host yourself.

[ttpro1995]

For reference, I have dijango chat bot project that can be deploy with Dokku. https://bitbucket.org/ttpro1995/djangochatbot/src/master/ This is not telegram chat bot. This is chatbot for another platform. But you might got an idea.

The bot will fetch CAT or Dog image for each user request. If I put the fetching in main bot flow, the bot will have to complete each request (take command, fetch cat, show cat) before taking another request. The platform does not send another callback if bot does not response to current callback. So, user cannot rapid fire cat.

Solution: I offset the request job to celery. When bot receive command, it will QUEUE a command to celery, then response to callback now, so it can receive new command.

Celery will receive command and user id. Celery will download an image and send to user by user id when it is done.

In this example, I use REDIS as queue. Dokku can support you host redis too.

Celery here: https://bitbucket.org/ttpro1995/djangochatbot/src/master/tasks.py

This is deploy file work with heroku and dokku https://bitbucket.org/ttpro1995/djangochatbot/src/master/Procfile

I suggest Dokku on self-host vps because you will need at least 2 worker (1 for bot, one for worker that does the download job), and more worker if you want more user.

[dipu-bd]

237 and #236 should be merged with this issue