search

cyberbotics / webots

Webots Robot Simulator
https://cyberbotics.com
Apache License 2.0
3.2k stars 1.68k forks source link

QA tests R2020b #1917

Closed omichel closed 4 years ago

omichel commented 4 years ago

Tests:

Installation

Test Description
Installation Install Webots from the provided package, following the installation instruction from the Webots user guide.
Simulation resset Check that when pressing the simulation reset while the simulation is running you have the same behavior at the second iteration.Check this for the 10-15 first worlds of the guided tour.
Creation of a small project From scratch and using only the Webots GUI (wizards (including VisualStudio on windows), scene tree, new, and so on), create a world
VRML import/export Check that the VRML import/export menu is working.
Make a movie Make a movie using the menu: Files > Make Movie ...
Test at least a normal screen resolution and the full screen resolution (1920x1080 Full HD).
Delete objects in the Scene Tree 1. Open a complex world
2. Select the last object in the Scene Tree
3. Hit the [Delete] button until the Scene Tree is empty (besides the 3 top nodes)
4. Revert and repeat the operation starting from the 4th node.
The test succeeds if Webots does not crash.
Copy a robot in the Scene Tree 1. Open a world that contains a robot
2. Select the robot
3. Hit the [Copy] button
4. Hit the [Paste After] button
5. At this point there should be 2 robots close to one another
6. Separate the 2 robots using the mouse (if needed)
7. Save the .wbt
8. Push [Revert]
9. Push [Run]

The test succeeds if both robots have the same appearance and behavior, etc.
Java controllers 1. Open the world: projects/languages/java/worlds/examples.wbt
2. Hit [Run]

The test succeeds if all 3 robots move, the camera images change and there is no error message.

3. Modify Help message (Driver.java at line 88)
4. Hit the [Save] button
5. Accept the relocation of the project proposed by Webots, in that order hit: [Yes], [OK], [Open], [OK].
6. Hit the [Build] button
7. When Webots proposes to 'Revert', hit [Yes].
8. Hit [Run]

The compilation test succeeds if the help message printed in the console contains the last modifications.
Python controllers 1. Open the world: projects/languages/python/worlds/examples.wbt
2. Hit [Run]

The test succeeds if all 3 robots move, the camera images change and there is no error message.

All the Python versions officially supported by the OS should be tested.
Robot window The generic robot window opens itself correctly, even if the simulation is stopped or at step 0 (by double clicking on the robot, or by selecting the corresponding item in the Robot menu)

Note: The e-puck, the Thymio II and the DARwIn-OP have a custom robot window.
Devices worlds Run all the examples in the 'projects/samples/devices/worlds' directory and check that they all work as expected.
Support of UTF-8 characters Test that a user having UTF-8 characters in his account name can:
- launch Webots
- launch controllers
- Compile controllers
Visual Studio wizard Test that the VisualStudio wizard works.
Matlab controllers 1. Open the world: projects/languages/matlab/worlds/e-puck_matlab.wbt
2. Hit [Run]

The test succeeds if:
a) the e-puck robot moves
b) the camera and display viewports show moving images
c) a Matlab Window with 4 subwindows appears and all subwindows show moving graphics
Mouse interactions Using the mouse and the keyboard, ensure that one can move the camera (translate, rotate, zoom), the objects (xz translation, lift), and apply forces/torques.

A right click in the 3D scene opens the context menu.

A double click on a robot opens the robot window.
e-puck - remote control Try to remote control an e-puck2 robot over Bluetooth
Binary compilation Test that controllers / physics plugins / controller plugins (remote control / robot windows) / libraries can be cleaned and compiled smoothly. On Windows, please make sure to test by opening Webots from the icon.
DARwIn-OP to test:
-> the simulations do the expected thing
-> controllers can be modified
-> you can remote-control and remote-compile the real robot
-> same with Robotis OP2
Make HTML5 animation Can create an animation (File / Make HTML5 animation) and play it into a web browser.
Streaming Server 1. start with the streaming server: webots --stream

2. open the web page located in resources/web/streaming_viewer/index.html and connect to Webots (using the default parameters).

3. the 3D view of Webots should display properly on the web page. It should be possible to change the viewpoint and select objects.
Extern controllers Open a simulation with a robot. Set the controller of the robot to and launch the controller from the command line. The test succeeds if the robot behaves the same as with the normal controller.
Offline documentation Check that the offline documentation is working from 'Help / offline documentation / ...' and from the context menu.
SUMO interface Open the 'projects/vehicle/highway' world:

1.Start the simulation and check that SUMO is correctly started in GUI mode.
2. Check that the version of SUMO (top of the SUMO GUI window) is the one specified in the Changelog.
3. Check in Webots that other vehicles are appearing and moving.
4. Check that the main vehicle is visible in SUMO as a green triangle (you may need to zoom).
OSM importer and SUMO exporter 1. Install the python dependencies as explained in https://www.cyberbotics.com/doc/automobile/openstreetmap-importer#dependencies

2. Convert the OSM file:

mkdir -p $HOME/my_project/resources/osm_files
mkdir -p $HOME/my_project/resources/OSM_importer
cp $WEBOTS_HOME/projects/vehicles/importer_demo.sh $HOME/my_project/importer_demo.sh
cp $WEBOTS_HOME/resources/osm_importer/config.ini $HOME/my_project/resources/OSM_importer/config.ini
cp $WEBOTS_HOME/projects/vehicles/resources/osm_files/village.osm $HOME/my_project/resources/osm_files/village.osm
cd $HOME/my_project
chmod u+x ./importer_demo.sh
./importer_demo.sh village

3. Open the generated world file and check the result is similar to the one distributed in Webots (project/vehicles/village).
4. In pause mode, add the SumoInterface node and set it's 'useNetconvert' field to FALSE.
5. Run the simulation and check that you can see vehicles appearing and correctly aligned on the roads.
Backward compatibility Open few worlds of the previous Webots version, and check they can be open with the rightful warnings if required. This test should particularly focus on the modifications of the tested release breaking the backward compatibility.
ROS controllers Test the distributed ROS simulations located in projects/languages/ros. The instructions are available at https://www.cyberbotics.com/doc/guide/tutorial-8-using-ros
Guided Tour Check that all the worlds of the guided tour behave as expected.
DavidMansolino commented 4 years ago

released