search

docker-scripts-archived / dev--LTSP

Virtual LTSP server with vagrant and/or docker-scripts.
GNU General Public License v3.0
7 stars 6 forks source link

Test extensively #63

Open dashohoxha opened 6 years ago

dashohoxha commented 6 years ago

Test extensively all the possible use cases and scenarios to make sure that everything works as expected and there are no problems or bugs. Report (create issues) and fix any problems/bugs.

Create a testing document that describes all the possible cases and scenarios that need to be tested, what values of settings.sh should be used, what scripts or commands to run, etc.

This is important because extensive testing needs to be done whenever new features are added or modifications are done to the scripts, to make sure that everything works correctly.

This will also make easy for other people to perform testing correctly, without missing any steps (related also to https://github.com/docker-scripts/dev--LTSP/issues/18).

d78ui98 commented 6 years ago

I wrote a testing plan for ubuntu bionic - https://github.com/docker-scripts/dev--LTSP/wiki/TESTING-PLAN

d78ui98 commented 6 years ago

test.sh script can be improved on bionic branch. Some improvements were made in buster branch some can be done on bionic

dashohoxha commented 6 years ago

Deepanshu, you have missed things that are necessary for testing, and you have included a lot of things that are not needed for testing. Usually in a testing plan there should not be any instructions about troubleshooting and fixing things. This is not the purpose of testing.

The testing plan should be as described following (explanations in brackets maybe should not be included, they are to explain to you what is expected to be done by the scripts).

Maybe you can add here some of the things that you have described. However most of it should go to a troubleshooting document, which may be useful for the developers, but should never be used by the testers. The testers should just report whether the things work as expected or not (and we describe here what is expected on each case). You are both a tester and a developer, so I understand that it is a bit difficult for you to separate what should be done by a tester and what should be done by a developer.

Testing Plan

0 Things to be checked on each test case

In all the test cases described below the things that should be checked are almost the same. They are listed here:

  1. On the ltsp-server window make sure that you can access the internet. Create also a test user.
  2. On the ltsp-client window login with the test user and make sure that it works.
  3. On the ltsp-client window, after you login with the test user, make sure that you can connect to internet.
  4. On ltsp-server make sure that the packages defined on the list of PACKAGES on settings.sh are installed.
  5. On the ltsp-server window open epoptes and make sure that you can access the desktop of the test-client.

1 Testing internally

Internal testing is done with the help of a virtual network interface, called 'ltsp-test-if', which is created before testing starts and is destroyed after testing ends. Before testing starts edit settings.sh and make sure that LAN_IF is set to ltsp-test-if.

1.1 Standalone mode

  1. Edit settings.sh and make sure that STANDALONE is set to "yes"`.
  2. Start testing with ./test.sh start. (This should make sure to destroy dhcp-server, ltsp-server and ltsp-client, and then to start ltsp-server and after that start ltsp-client). A window for the ltsp-server and a window for the ltsp-client should be created.
  3. Perform all the checks described on the first section.
  4. Stop testing with ./test.sh stop. (This should halt and destroy ltsp-client and ltsp-server, as well as remove the virtual network interface 'ltsp-test-if' that was used for testing).

1.2 Normal mode

  1. Edit settings.sh and make sure that STANDALONE is set to "no"`.
  2. Start testing with ./test.sh start. (This should make sure to destroy dhcp-server, ltsp-server and ltsp-client, and then to start dhcp-server, ltsp-server, and ltsp-client). A window for the dhcp-server, and ltsp-server, and ltsp-client should be created.
  3. On the ltsp-server window make sure that you can access the internet (using dhcp-server as a gateway). Create also a test user.
  4. Perform all the checks described on the first section.
  5. Stop testing with ./test.sh stop. (This should halt and destroy dhcp-server, ltsp-client and ltsp-server, as well as remove the virtual network interface 'ltsp-test-if' that was used for testing).

2 Testing externally

External testing is done using a real ethernet (LAN) interface for bridging. For this to work, this interface should be connected with a cable to switch/hub, and the interface must be up (enabled/activated). Before testing starts edit settings.sh and make sure that LAN_IF is set to name of the interface that will be used for testing.

2.1 Standalone mode

Testing is done with the help of test.sh, following the same steps as for internal testing above

2.2 Normal mode

Testing is done with the help of test.sh, following the same steps as for internal testing above

3. Real testing on LAN

In this case the client and the dhcp server will be real ones, on a real LAN. Edit settings.sh and make sure that LAN_IF is set to the network interface that is connected to the LAN.

3.1 Standalone mode

This case can be tested only if there is no dhcp/gateway server on the LAN and the server can connect to internet through another interface (for example WiFi interface).

  1. Edit settings.sh and make sure that STANDALONE is set to "yes".
  2. Destroy the ltsp-server and then start it again: vagrant destry ; vagrant up
  3. Start a ltsp client on the LAN, either directly or using the virtual ltsp-client
  4. Perform all the checks described on the first section.

Note: Before starting a virtual ltsp-client on the LAN, we should edit the corresponding settings.sh and make sure that LAN_IF has the name of the interface that is actually connected to the LAN.

3.2 Normal mode

This case can be tested only if there is already a dhcp/gateway server on the LAN.

  1. Edit settings.sh and make sure that STANDALONE is set to "no". Also make sure that LAN_IP is in the range of the LAN to which we are connected.
  2. Destroy the ltsp-server and then start it again: vagrant destry ; vagrant up
  3. Start a ltsp client on the LAN, either directly or using the virtual ltsp-client
  4. Perform all the checks described on the first section.

Note: The same thing as in the standalone case applies for starting a virtual ltsp-client on the LAN

dashohoxha commented 6 years ago

I have created this wiki page as well: https://github.com/docker-scripts/dev--LTSP/wiki/Testing-Plan

d78ui98 commented 6 years ago

okay, this is impressive. I will perform the tests accordingly. And make necessary changes in settings.sh, test.sh and other files

dashohoxha commented 6 years ago

First make the required changes, then do the testing. There are changes that you already know you have to make, without having to test it first.

d78ui98 commented 6 years ago

I tested Virtual LTSP server on a real lan. Client was running on a different machine. The interface were real and connected to the internet. One of the things to make sure is to check whether the ltsp-server is in the Network. We can do it by a simple ping. Also gateway is 192.168.1.1 So ltsp server was also on same network. Rest of the process of booting goes fine. Will be testing all other cases.

dashohoxha commented 6 years ago

Rest of the process of booting goes fine.

Booting is only the minimum of testing. There are other things to be tested as well, like user login, internet connection both from the clients and from the server, maybe performance of some heavy applications (like LibreOffice etc.).

Most of the tests can be done more efficiently with internal testing. Once everything works well with internal testing, most probably it should work well on the real LAN test as well.

d78ui98 commented 6 years ago

exhaustive testing

Testing for all possible cominations. Created 2 videos of the testing. Each part has sub case shows mistake because of which the virtual ltsp does not work.

1. Standalone="yes" , LAN_IP="192.168.111.16" (19:47)

Works perfectly as soon as we run vagrant up and boot from client. condition

After this I made changes in install.sh to make video short speed up (22:24) Just removed all the packages to install added rm -rf /etc/dnsmasq.d/ltsp-server-dnsmasq.conf and removed all lines excluding the dnsmasq configuration.

2. Standalone="no", LAN_IP="192.168.111.16" (23:26)

Will not work as dhcp server(router) is present in 192.168.1 and ltsp server is not providing ip address. You can make use of dhcp server in dhcp folder to make this work.

3. Standalone="no", LAN_IP="192.168.1.16" (29:28)

Works perfectly as proxy dhcp feature of dnamasq is used. It just provide booting information and leaves task of IP leasing orignal dhcp server.

4. Standalone="yes", LAN_IP="192.168.1.16" (36:25)

Again works prefectly when no dhcp present in 192.168.1 network

5. Real lan testing (video)

Standalone="no", LAN_IP="192.168.1.16" It works.

Issues identified

dashohoxha commented 6 years ago

On the source code there is this setting: VB_IMAGE="d78ui98/linuxmint-19-xfce-32bit" How come that in your testing it is mintxfce? Even if they are identical, still this is not correct testing.

Booting is only the minimum of testing.

I am repeating myself over and over, but you still close the client as soon as it boots up, or as soon as you login. Let me repeat it once more:

0 Things to be checked on each test case

In all the test cases described below the things that should be checked are almost the same. They are listed here:

  1. On the ltsp-server window make sure that you can access the internet. Create also a test user.
  2. On the ltsp-client window login with the test user and make sure that it works.
  3. On the ltsp-client window, after you login with the test user, make sure that you can connect to internet.
  4. On ltsp-server make sure that the packages defined on the list of PACKAGES on settings.sh are installed.
  5. On the ltsp-server window open epoptes and make sure that you can access the desktop of the test-client.
d78ui98 commented 6 years ago

How come that in your testing it is mintxfce

changed that to d78ui98/linuxmint-19-xfce-32bit

On the ltsp-server window make sure that you can access the internet. Create also a test user.

test user created and added to epoptes group

On the ltsp-client window login with the test user and make sure that it works.

This part is creating an issue. If I tried login from the terminal of client with ssh test@server It works. But does not work from gui. After login it says - 

Unable to determine failsafe session name.  Possible causes: xfconfd isn't running (D-Bus setup problem); environment variable $XDG_CONFIG_DIRS is set incorrectly (must include "/etc"), or xfce4-session is installed incorrectly.

Googling about it suggests its a permission issue. But even if I run xfconfd manually with /usr/lib/i386-linux-gnu/xfce4/xfconf/xfconfd It gives error message Working on a fix.

On the ltsp-client window, after you login with the test user, make sure that you can connect to internet.

Internet on client as test user works(checked from commandline). Will also test on gui when the problem is fixed. Now working on a fix.

dashohoxha commented 6 years ago

Please create a separate issue for each problem that you encounter. This issue is already overloaded.

d78ui98 commented 6 years ago

This issue occured due to mistake on my end. As I am very new to linux mint. I am now able log in from test user. Internet works. Able to open epoptes and control the client.

Also tested with defining packages variable defined in setting.sh. Now creating a video to show to you and akash.

dashohoxha commented 6 years ago

Please don't create a video. A video does not prove anything. I get these messages:

Guest-specific operations were attempted on a machine that is not
ready for guest communication. This should not happen and a bug
should be reported.

and later:

Timed out while waiting for the machine to boot. This means that
Vagrant was unable to communicate with the guest machine within
the configured ("config.vm.boot_timeout" value) time period.

If you look above, you should be able to see the error(s) that
Vagrant had when attempting to connect to the machine. These errors
are usually good hints as to what may be wrong.

If you're using a custom box, make sure that networking is properly
working and you're able to connect to the machine. It is a common
problem that networking isn't setup properly in these boxes.
Verify that authentication configurations are also setup properly,
as well.

If the box appears to be booting properly, you may want to increase
the timeout ("config.vm.boot_timeout") value.
d78ui98 commented 6 years ago

Okay. That issue occurs due to difference occurs due to difference in vboxguest version of your virtualbox and vagrant box. Can you please run a modinfo vboxguest to let me know which version is your virtualbox using? mine is

version:        5.2.8_KernelUbuntu r120774

Same is on the vagrant box. As I result I am able to reproduce this issue.

dashohoxha commented 6 years ago

Don't close it so quickly.