search

TelegramBots / Telegram.Bot

.NET Client for Telegram Bot API
https://telegrambots.github.io/book
MIT License
3.17k stars 686 forks source link

Integration Tests #377

Closed poulad closed 7 years ago

poulad commented 7 years ago

Hi

Please have a look at this and let me know what do you think about it. I hope this saves us all the time we spend on manually testing our work.

EDIT:

See new wiki for tests here: https://github.com/TelegramBots/telegram.bot/blob/integ-tests/docs/wikis/tests/integ-tests.md

Integration Tests

Integration tests are meant to test the project with real data. This is a semi-automated test and a tester needs to interact with bot for some cases during the test execution. However, tester can select specific test(s) and run them.

Code is available in tests branch of this repo: https://github.com/TelegramBots/Telegram.Bot.Core/tree/dev

For testing, a .NET Core xUnit project is used. Test project could be run in VS, CLI, or a CI hosting.

See sample output on Travis-CI here: https://travis-ci.org/TelegramBots/Telegram.Bot.Core/builds/251359910

And this is what bot posted in Telegram chat while running those tests:

Sample test output

Those arrows indicate the test cases that tester needs to interact with bot.

Test Setup

A Telegram bot, of course, is needed and its token should be provided to the runner in appsettings.json file or from environment variable prefixed by TelegramBot_.

Other optional parameters such as the UserId and ChatId could be used to send output to a specific chat/user.

Test Initiation

Bot needs to know the chat it should post to. There are 2 ways to tell it.

/test Command

When test project starts, bot waits for 2 minutes to receive a /test command update from any chat. Once gets that, starts the process and outputs to that chat.

Specific Chat ID

If Chat id is specified in the configurations, bot doesn't wait for any initiation command and sends the output to that chat.

MrRoundRobin commented 7 years ago

I like this very much, always wanted some tests 👍

Olfi01 commented 7 years ago

@pouladpld I like them, I have tried them, but for some reason, it shows they all failed when I finish. Every single one was terminated by a TaskCancelledException or sth xD Though I saw that everything worked fine during execution, that seems a little bit strange to me :)

poulad commented 7 years ago

@Olfi01 This is unusual! There is a timeout of mostly 2 minutes that tests wait for tester to interact. Passing that, async operation tokens are going to be cancelled.

Make sure your test bot has the followings:

Unfortunately, it is easy to miss some settings so verify that the configuration values are read correctly in tests. Just put some breakpoints and check values such as PaymentProviderToken.

Olfi01 commented 7 years ago

@pouladpld I have done all of that, the test executed perfectly fine and even showed a success status, but after i finished all, the errors came up. Probably it's just a bug of visual studio and not your fault :)

poulad commented 7 years ago

@Olfi01 Try running them from dotnet core CLI. Does it behave the same?

Olfi01 commented 7 years ago

@pouladpld err im busy with sth els rn i will do that later

MrRoundRobin commented 7 years ago

@pouladpld sorry had to delete the core repo to Transfer the main repo (I have a local copy) would you like to create a branch for developing the test Project?

poulad commented 7 years ago

That's fine. I pushed them here: https://github.com/pouladpld/telegram.bot/tree/dev I am merging them right now. Too bad I didn't make a copy of notes in Project section of Core repo 😞 .

poulad commented 7 years ago

I merged them and all test cases are passing. https://github.com/TelegramBots/telegram.bot/blob/integ-tests/docs/wikis/tests/integ-tests.md