godotengine / godot

Godot Engine – Multi-platform 2D and 3D game engine
https://godotengine.org
MIT License
86.87k stars 19.45k forks source link

[TRACKER] Unit tests to add or improve #43440

Open Calinou opened 3 years ago

Calinou commented 3 years ago

Our unit test coverage is currently fairly low. We'd like to increase our unit test coverage; any help is welcome.

Interested in writing new unit tests? See the unit tests documentation and compiling instructions. If you have further questions, join the Godot Contributors Chat.

When opening a pull request, please link back to this issue (#43440) in the PR description so that we can keep track of it more easily.

If you have the appropriate permissions, feel free to edit this issue to add new items or check items for which a PR has been opened.

Classes to test

These classes are currently lacking in test coverage, and are therefore highest-priority for receiving unit tests. Deprecated classes are not listed.

[!NOTE]

When a class is listed with "and" along a given list item, it should be submitted in the same pull request whenever possible. Tests for these classes can be in the same file or a different file depending on the size and complexity of the test suite. If in doubt, follow the file organization used in the original class implementation.

Completed classes

These classes currently have good test coverage. Further improvements may be possible by testing methods that were added after the tests were merged.

- [x] **AABB** https://github.com/godotengine/godot/pull/43727 - [x] **Animation** https://github.com/godotengine/godot/pull/57490 - [x] **Array** https://github.com/godotengine/godot/pull/46227 - [x] **ArrayMesh** - [x] **AudioStreamWAV** https://github.com/godotengine/godot/pull/60736 - [x] **BitMap** - [x] **Camera2D** https://github.com/godotengine/godot/pull/88614 - [x] **Camera3D:** May not be testable in headless mode, but it's worth trying. Test `is_position_behind()` and the `project_`/`unproject_` methods in particular - [x] **CryptoKey** https://github.com/godotengine/godot/pull/89021 - [x] **Curve** https://github.com/godotengine/godot/pull/43516 - [x] **Curve2D** https://github.com/godotengine/godot/pull/71308 - [x] **Curve3D** #76812 - [x] **Dictionary** https://github.com/godotengine/godot/pull/47511 - [x] **FastNoiseLite** - [x] **Font** https://github.com/godotengine/godot/pull/81503 - [x] **GDScript** https://github.com/godotengine/godot/pull/48657 - [x] **Geometry2D** https://github.com/godotengine/godot/pull/59643 - [x] **Geometry3D** https://github.com/godotengine/godot/pull/44974 - [x] **GradientTexture** https://github.com/godotengine/godot/pull/90501 - [x] **HashingContext** https://github.com/godotengine/godot/pull/43459 - [x] **HTTPClient** https://github.com/godotengine/godot/pull/76636 - [x] **Image** https://github.com/godotengine/godot/pull/45737 - [x] **ImageTexture** https://github.com/godotengine/godot/pull/88044 - [x] **InputEvent** https://github.com/godotengine/godot/pull/79444 - [x] **InputEventKey** https://github.com/godotengine/godot/pull/59193 - [x] **JSON** https://github.com/godotengine/godot/pull/43517 - [x] **JSONRPC**: https://github.com/godotengine/godot/pull/89124 - [x] **Math** https://github.com/godotengine/godot/pull/48721 - [x] **Marshalls** https://github.com/godotengine/godot/pull/44797 - [x] **Node**: https://github.com/godotengine/godot/pull/70152 https://github.com/godotengine/godot/pull/71367 - [x] **NodePath** https://github.com/godotengine/godot/pull/43578 - [x] **NoiseTexture** https://github.com/godotengine/godot/pull/70919 - [x] **Object** https://github.com/godotengine/godot/pull/43583 (completed by https://github.com/godotengine/godot/pull/44387) - [x] **OS** https://github.com/godotengine/godot/pull/42069 - [x] **PackedScene** https://github.com/godotengine/godot/pull/79440 https://github.com/godotengine/godot/pull/80423 https://github.com/godotengine/godot/pull/80819 - [x] **Path2D** https://github.com/godotengine/godot/pull/66927 - [x] **PathFollow2D** https://github.com/godotengine/godot/pull/46277 - [x] **Path3D** - [x] **PathFollow3D** https://github.com/godotengine/godot/pull/46341 - [x] **PCKPacker** https://github.com/godotengine/godot/pull/43572 - [x] **Plane** https://github.com/godotengine/godot/pull/44492 - [x] **PrimitiveMesh** (CapsuleMesh, CubeMesh, …) - [x] **Projection** https://github.com/godotengine/godot/pull/77275 - [x] **Quaternion** https://github.com/godotengine/godot/pull/50907 - [x] **RandomNumberGenerator** https://github.com/godotengine/godot/pull/44560 - [x] **Rect2:** Not covered in the legacy math test. https://github.com/godotengine/godot/pull/43514 - [x] **Rect2i:** Not covered in the legacy math test. https://github.com/godotengine/godot/pull/43514 - [x] **RegEx** https://github.com/godotengine/godot/pull/82225 - [x] **RID** https://github.com/godotengine/godot/pull/54325 - [x] **Resource** https://github.com/godotengine/godot/pull/43731 - [x] **Shortcut** https://github.com/godotengine/godot/pull/58712 - [x] **SpriteFrames** https://github.com/godotengine/godot/pull/57742 - [x] **Transform** - [x] **Transform2D** - [x] **Translation** https://github.com/godotengine/godot/pull/48778 - [x] **TranslationServer** https://github.com/godotengine/godot/pull/79331 - [x] **Variant** https://github.com/godotengine/godot/pull/76244 - [x] **Vector** https://github.com/godotengine/godot/pull/48821 - [x] **Vector2/Vector2i** https://github.com/godotengine/godot/pull/47202 - [x] **Vector3/Vector3i** https://github.com/godotengine/godot/pull/48063 - [x] **Vector4/Vector4i** https://github.com/godotengine/godot/pull/64027 - [x] **VisualShader** https://github.com/godotengine/godot/pull/70396

Non-testable classes

These classes can't be unit-tested for technical reasons. Unit tests always run in headless mode, so they can't do things such as rendering scenes and checking the visual result.

- [x] ~~**CubeMap:**~~ Not testable without RenderingServer access. - [x] ~~**Shader:**~~ Not testable without RenderingServer access.
Calinou commented 3 years ago

If it is all right, I would like to take a crack at testing the Bitmap class.

Sure, go ahead :slightly_smiling_face:

@rmarco3e8 Are you still working on unit tests for Path2D? If not, please comment here so that another contributor can take up the task. Feel free to link to a work-in-progress branch if you've started work but are unable to complete it.

DanielKriz commented 3 years ago

Hello, I would like to make tests for Path3D. I also have two questions/suggestions:

  1. If I am using the TEST_SUITE feature of Doctest, Should I encapsulate all of Path3D unit test in one suite, or group my unit tests to different funtionality groups? I think it would be useful if we could settle on some format and stick to it.

  2. I think it would be great if we organize tests in different folders (for example corresponding to scene/ directory, so path3d would be included as "scene/3d/test_path3d.h"), that way it would be much more readable and organized, even in test_main.cpp.

Calinou commented 3 years ago
  1. If I am using the TEST_SUITE feature of Doctest, Should I encapsulate all of Path3D unit test in one suite, or group my unit tests to different funtionality groups? I think it would be useful if we could settle on some format and stick to it.

We generally use only one test suite per class. We do use several test cases per class though. I suggest grouping checks of the same method (but with different parameters) in the same test case.

  1. I think it would be great if we organize tests in different folders (for example corresponding to scene/ directory, so path3d would be included as "scene/3d/test_path3d.h"), that way it would be much more readable and organized, even in test_main.cpp.

I think the current organization was chosen as to simplify the build system around it. I don't think having all tests in the same folder is much of a problem in practice.

Xrayez commented 3 years ago

I think the current organization was chosen as to simplify the build system around it. I don't think having all tests in the same folder is much of a problem in practice.

To be honest, test cases with doctest can be written next to production code (as it was designed to be), but now we're talking about including test code in core, and I'm not sure whether this approach @reduz would be fond of. Tests are currently self-contained and mostly don't affect the main/production code.

But currently, it's possible to improve the build system to recursively collect test suite headers in tests/. This should also eliminate the need to include test suites in test_main.cpp for each new one manually.

Gorgonx7 commented 3 years ago

If you don't mind I'm gonna take a look at testing http_client, I'm also going to refactor it a little to make it more testable no major changes just breaking down those larger functions a little

jcsho commented 3 years ago

Hi friends, interested in tackling unit tests for shortcut.

Just to clarify it's the component in scene/gui/shortcut.h?

Let me know if there's anything I should be aware of or missed in the long thread!

Calinou commented 3 years ago

Hi, interested in tackling unit tests for shortcut.

Just to clarify it's the component in scene/gui/shortcut.h?

Yes, it is :slightly_smiling_face:

skimmedsquare commented 2 years ago

Planning on adding some unit tests for RID, should have a PR later today

Calinou commented 2 years ago

Asking for updates, just in case someone no longer has time to complete unit tests. It's fine if you lack time or don't know how to implement tests for a class, but please leave a comment so that others can continue working :slightly_smiling_face:

@jscho Are you still working on unit tests for Shortcut?

@resul4e Are you still working on unit tests for BitMap?

skimmedsquare commented 2 years ago

Planning on taking a stab at the *Texture classes next, starting with ImageTexture

resul4e commented 2 years ago

Asking for updates, just in case someone no longer has time to complete unit tests. It's fine if you lack time or don't know how to implement tests for a class, but please leave a comment so that others can continue working πŸ™‚

@jscho Are you still working on unit tests for Shortcut?

@resul4e Are you still working on unit tests for BitMap?

Unfortunately, I don't have the time to finish the unit tests. Most of the unit tests are implemented though. Which you can find here: https://github.com/resul4e/godot/tree/bitmap_testing. If someone wants to complete them.

PrinceDeveloperOf commented 2 years ago

I'm going to take a shot at doing the unit test for the ArrayMesh

im-deepfriedwater commented 2 years ago

want to try to take on SpriteFrames if that's alright!

Calinou commented 2 years ago

want to try to take on SpriteFrames if that's alright!

Sure, go ahead :slightly_smiling_face:

@skimmedsquare Did you manage to get tests working for ImageTexture?

im-deepfriedwater commented 2 years ago

PR up for the SpriteFrames unit tests! πŸ™‚ #57742

skimmedsquare commented 2 years ago

@Calinou - Sorry, work on my end stalled on ImageTexture, found myself going down a rathole of a bunch of different dependencies in making effective unit tests for that class due to its dependency on other classes / lack of mocking support within the project when I was looking. Feel free to remove my assignment from it for now.

rocketman1243 commented 2 years ago

Hi all, I'd like to write tests for Shortcut. @resul4e mentioned here that he worked on that, but when I checked out his work, it turns out he actually worked at Bitmap. So, for anyone wanting to work on that: here is part of that work. You can continue working on that if you want.

That being said, I will work on writing test cases for the Shortcut class. You can find my work here, so if I somehow don't finish this and don't respond anymore, you can continue where I left off.

I'll ask questions in the rocketchat if I have them!

rocketman1243 commented 2 years ago

The PR for the unit tests for Shortcut is ready for review!

kneejuicer commented 2 years ago

If it's alright, I'd like to take a stab at writing some tests for the methods in Geometry2D that are still untested.

Calinou commented 2 years ago

If it's alright, I'd like to take a stab at writing some tests for the methods in Geometry2D that are still untested.

Sure, go ahead :slightly_smiling_face:

rocketman1243 commented 2 years ago

Hi, I'd like to write tests for InputEventKey.

Calinou commented 2 years ago

Hi, I'd like to write tests for InputEventKey.

Sounds good!

rocketman1243 commented 2 years ago

The PR for the unit tests for InputEventKey is ready for review!

StrawbDev commented 2 years ago

Hi, I'd like to take on writing tests for AudioStreamSample. @Calinou is the save_to_wav() method there as an example, or is it the only thing that needs testing?

Calinou commented 2 years ago

@Calinou is the save_to_wav() method there as an example, or is it the only thing that needs testing?

I haven't checked for sure, but it might be the only method in AudioStreamSample that's easily unit testable right now. Feel free to open a pull request that only adds a test for save_to_wav() – it's better than nothing :slightly_smiling_face:

StrawbDev commented 2 years ago

@Calinou The set_data() method on AudioStreamSample takes a lock on the AudioServer singleton, which is a problem as it isn't instantiated when the unit tests run. I can get it to work if I instantiate AudioServer along with AudioDriverDummy. Do you think it would be appropriate to add that to test_main.cpp? There are other servers being set up there in test_case_start().

StrawbDev commented 2 years ago

PR for AudioStreamSample tests is now up for review!

Calinou commented 2 years ago

I'm going to take a shot at doing the unit test for the ArrayMesh

@PrinceDeveloperOf Did you manage to get tests for ArrayMesh working in the end? If not, no worries – let us know so we can update the task list. This way, another contributor can give it a look.

PrinceDeveloperOf commented 2 years ago

I'm going to take a shot at doing the unit test for the ArrayMesh

@PrinceDeveloperOf Did you manage to get tests for ArrayMesh working in the end? If not, no worries – let us know so we can update the task list. This way, another contributor can give it a look.

Let someone else do it.

georgjz commented 2 years ago

I'd like to give Curve2D a shot.

jamesfesik12 commented 1 year ago

Hey everyone! I am a first time open source contributor and wanted to take a shot at the CubeMap unit test! Where would I find the documentation and class setup for that unit test? Any help would be greatly appreciated.

Calinou commented 1 year ago

Hey everyone! I am a first time open source contributor and wanted to take a shot at the CubeMap unit test! Where would I find the documentation and class setup for that unit test? Any help would be greatly appreciated.

Cubemap is a rarely used class, so there is no documentation for it (other than the class description in doc/classes/Cubemap.xml). You'll have to read the source code and look at the methods defined in the class.

The same applies to CubemapArray.

jamesfesik12 commented 1 year ago

Cubemap is a rarely used class, so there is no documentation for it (other than the class description in doc/classes/Cubemap.xml). You'll have to read the source code and look at the methods defined in the class.

@Calinou I see thank you for your quick response! I have been researching inside the source code and have found the correct files to include however whenever I try to set the sides to a image color my IDE does not recognize the set_side method. Am I missing a file that I should include? I have the texture.h file and the math_defs.h file included

-> im trying to run this command to set each side with images but it wont recognize almost any of these methods. _cubemap->set_side(cubemap.SIDE_FRONT, createimage(Color.red));

Looking more into it I found the instantiation of the CubeMap class and it extends ImageTextureLayered which has all of its methods and variables. However, neither one of these has set_side or any kind of side enum as a variable or method.

jbcolli2 commented 1 year ago

I'd like to start working on unit tests for the Primitives, if no one else has started them.

Calinou commented 1 year ago

I'd like to start working on unit tests for the Primitives, if no one else has started them.

Feel free to open a pull request for this :slightly_smiling_face:

I'm not sure exactly how these should be tested though, so you may have to be creative here.

jbcolli2 commented 1 year ago

Yeah, I'm new to the godot codebase, but I can't seem to find a place in the code where these classes are instantiated. Am I missing something or are these classes just not currently used in the code?

jbcolli2 commented 1 year ago

I've looked through the PRs for the tests that are checked off as done and I found a few that did not #include the test files in the test_main.cpp file.

Looks like the Math is maybe being worked on in the #40659, but the PathFollow are not mentioned in any other issue I can see. Also when I do include test_path_follow_2d.h and test_path_follow_3d.h in test_main.cpp the build does not compile.

Might be worth unchecking these unit tests and letting someone take a look at them again, unless this is already mentioned in another issue that I missed. I'm currently working on the Primitive classes, but if the OP on this issue agrees, I'd be happy to take a look at the unit tests for PathFollow2D/3D after I'm done.

raulsntos commented 1 year ago

@jbcolli2 The PathFollow tests are being worked on https://github.com/godotengine/godot/pull/51372

jbcolli2 commented 1 year ago

When working on the Primitive unit tests, I've noticed that the normals for the CylinderMesh are not calculated correctly when the top and bottom radii are different. Does anyone know if this is a known issue? If not I'll create an issue and submit the fix in a different PR.

jbcolli2 commented 1 year ago

If no one else is working on it, I'll take a shot at tests for ArrayMesh.

Geometror commented 1 year ago

For those interested, here is a test coverage report generated with gcov/gcovr. The thirdparty and tests directories are excluded as well as the generated header files (*.gen.h) In case you want to generate it yourself and fiddle with it here are the steps:

  1. Compile godot with use_coverage=yes, dev_build=yes and (dev_mode=yes or tests=yes))
  2. Run the godot executable with the --test argument andt wait until the tests are done
  3. Generate the report with gcovr -e thirdparty/ -e tests/ -e .*.gen.h --html-details coverage.html

Of course there are still some files which don't make sense to include but overall I these numbers should be pretty accurate.

Summary: image

Whole report (Google drive since GitHub does not allow uploading files over 25MB): https://drive.google.com/file/d/149VlBwMaU_O5xChfwpdcOYJ8h-q6Jsgw/view?usp=sharing

LucasLaukka commented 1 year ago

@georgjz are you still working on Curve2D? Otherwise I was thinking about giving it a shot.

georgjz commented 1 year ago

@LucasLaukka I got side-tracked real badly πŸ˜„ have a go if you like, maybe I'll just move on to Curve3D....some day

jainl28patel commented 1 year ago

Hi @Calinou I am new to Godot having experience in C++ dev. Would like to work on VisualShader is no one is working on it. Can you please give some pointers on where to start from and what all to check ??

Maran23 commented 1 year ago

Hi @Calinou I am new to Godot having experience in C++ dev. Would like to work on VisualShader is no one is working on it. Can you please give some pointers on where to start from and what all to check ??

See the description. Compiling instructions: https://docs.godotengine.org/en/latest/development/compiling/ Unit testing: https://docs.godotengine.org/en/latest/development/cpp/unit_testing.html And then have a look on the other tests.

purviporwal1812 commented 1 year ago

Is this issue solved yet? Otherwise I'd like to work on it. Can you assign this issue to me?

Maran23 commented 1 year ago

Is this issue solved yet? Otherwise I'd like to work on it. Can you assign this issue to me?

This is only tracker of components which do not have any tests. It is a bit outdated, so you should checkout which components don't have any tests yet.

ARez2 commented 1 year ago

Hi, first time contributor here, I would like to get started with contributing to Godot by writing some tests. Since that one does not seem to be worked on yet, i am starting by trying to write some tests for InputEventMouse. Let's see how this goes!

ARez2 commented 1 year ago

@Calinou i've noticed that both Curve2D, Crypto and VisualShader Unit Tests have been Merged. Could you update the checklist at the top?

Sauermann commented 1 year ago

I have started working on Unit tests for Viewport::_gui_input_event in #72862.