Tests

[huumanoid]

Introduction

Often our code break down after some changes. The main cause of bugs is stupid mistakes, which i have made a lot while working on this project. I'm tired of being scary and suggest to add tests. I've tried to implement some integration tests.

Notice

This document is incomplete, and It take some time to complete it. Also, maybe i need to change something in my solution, and maybe i need to explain something to you.

How to use

1. Perform npm install for installing dev dependencies
2. Run coffee src/test/db-init.coffee for setting up database for tests.
3. Run npm test.

   Implementation

   Tools

   • Mochajs & Chaijs Mocha is popular framework for nodejs tests, Chai is BDD framework which helps to specify requirements for result.
   • xat-client My nodejs implementation of xat client. Contains class XatUser, which implements basic interaction with xat server, and some helpers, which implements additional features.

     Files structure

   • Tests themselves are in test folder
   • Test helpers are in src/test

     Tests structure

Tests are logically divided into multiple files. Currently, there are four files, which tests different functionality

• interaction of regular user and guest user (user with temporary id between 199990000 and 2000000000),
• sending private messages,
• user signed into two different chats,
• user sends message to main chat, other users should receive it.

Before performing tests, database structure should be set up. db-init.coffee performs database initialization for tests purposes. It creates users and chats for tests. There are some reserved userids and chatids: users from 50 to 100 chats from 100 to 200.

Typical flow of test in every file:

1. Firstly, xat server is instantiated.
2. Then, time to create test-case environment. For example, in our case we want to test interaction of two users in the same chat. In this phase, we creating two instances of clients, setting up additional features, required in test, and connecting users to chat.
3. Performing test-cases and checking results.
4. Disconnecting users from server and shutting down server.

   Problems

   Kind of tests

The implemented tests are integration tests. Unit tests can be more reliable, but they require suitable program design, so i don't think we are ready for them. Moreover, may be integration tests are enough?

"Should not receive a message" test-case

Often we need to cut illegal messages: for example, if non-admin user tries to ban someone, no one should be banned, so we want to test if victim receives 'banned' message or not. According to using test type, the easiest way to test this case is to use delay: Set up: User Mallory and user Bob connected to chat 50. They are both guests.

1. Mallory sends 'ban' message to server with 'd'(destination) = Bob's id
2. Wait for X milliseconds.
3. Check if Bob received 'banned' message Disadvantage of this approach: we don't know, how big X should be. I don't want to make X big, otherwise tests will run too long. Small X means that sometimes test will pass, sometimes not. Very small X causes test to fail every time. Is there a better way to test such cases?

   Results

This solution already helped me to fix one bug just today. May be it helped to fix some bugs in past, but i don't remember it.

[iiegor]

Thanks @HuuMaNoID for your amazing work! I will test this and merge.

Regarding the "Should not receive a message" test-case question, I can't think on a better way of doing that but since it's a local server what we are testing, the time should not vary much.

[iiegor]

@HuuMaNoID Merged! I made a few changes to your original PR, please, can you check if master branch it's working as supposed? Thanks.