godotengine / godot

Godot Engine – Multi-platform 2D and 3D game engine
https://godotengine.org
MIT License
88.33k stars 20k forks source link

[TRACKER] Unit tests to add or improve #43440

Open Calinou opened 3 years ago

Calinou commented 3 years ago

Our unit test coverage is currently fairly low. We'd like to increase our unit test coverage; any help is welcome.

Interested in writing new unit tests? See the unit tests documentation and compiling instructions. If you have further questions, join the Godot Contributors Chat.

When opening a pull request, please link back to this issue (#43440) in the PR description so that we can keep track of it more easily.

If you have the appropriate permissions, feel free to edit this issue to add new items or check items for which a PR has been opened.

Classes to test

These classes are currently lacking in test coverage, and are therefore highest-priority for receiving unit tests. Deprecated classes are not listed.

[!NOTE]

When a class is listed with "and" along a given list item, it should be submitted in the same pull request whenever possible. Tests for these classes can be in the same file or a different file depending on the size and complexity of the test suite. If in doubt, follow the file organization used in the original class implementation.

Completed classes

These classes currently have good test coverage. Further improvements may be possible by testing methods that were added after the tests were merged.

- [x] **AABB** https://github.com/godotengine/godot/pull/43727 - [x] **Animation** https://github.com/godotengine/godot/pull/57490 - [x] **Array** https://github.com/godotengine/godot/pull/46227 - [x] **ArrayMesh** - [x] **AudioStreamWAV** https://github.com/godotengine/godot/pull/60736 - [x] **BitMap** - [x] **Camera2D** https://github.com/godotengine/godot/pull/88614 - [x] **Camera3D:** May not be testable in headless mode, but it's worth trying. Test `is_position_behind()` and the `project_`/`unproject_` methods in particular - [x] **CryptoKey** https://github.com/godotengine/godot/pull/89021 - [x] **Curve** https://github.com/godotengine/godot/pull/43516 - [x] **Curve2D** https://github.com/godotengine/godot/pull/71308 - [x] **Curve3D** #76812 - [x] **Dictionary** https://github.com/godotengine/godot/pull/47511 - [x] **FastNoiseLite** - [x] **Font** https://github.com/godotengine/godot/pull/81503 - [x] **GDScript** https://github.com/godotengine/godot/pull/48657 - [x] **Geometry2D** https://github.com/godotengine/godot/pull/59643 - [x] **Geometry3D** https://github.com/godotengine/godot/pull/44974 - [x] **GradientTexture** https://github.com/godotengine/godot/pull/90501 - [x] **HashingContext** https://github.com/godotengine/godot/pull/43459 - [x] **HTTPClient** https://github.com/godotengine/godot/pull/76636 - [x] **Image** https://github.com/godotengine/godot/pull/45737 - [x] **ImageTexture** https://github.com/godotengine/godot/pull/88044 - [x] **InputEvent** https://github.com/godotengine/godot/pull/79444 - [x] **InputEventKey** https://github.com/godotengine/godot/pull/59193 - [x] **JSON** https://github.com/godotengine/godot/pull/43517 - [x] **JSONRPC**: https://github.com/godotengine/godot/pull/89124 - [x] **Math** https://github.com/godotengine/godot/pull/48721 - [x] **Marshalls** https://github.com/godotengine/godot/pull/44797 - [x] **Node**: https://github.com/godotengine/godot/pull/70152 https://github.com/godotengine/godot/pull/71367 - [x] **NodePath** https://github.com/godotengine/godot/pull/43578 - [x] **NoiseTexture** https://github.com/godotengine/godot/pull/70919 - [x] **Object** https://github.com/godotengine/godot/pull/43583 (completed by https://github.com/godotengine/godot/pull/44387) - [x] **OS** https://github.com/godotengine/godot/pull/42069 - [x] **PackedScene** https://github.com/godotengine/godot/pull/79440 https://github.com/godotengine/godot/pull/80423 https://github.com/godotengine/godot/pull/80819 - [x] **Path2D** https://github.com/godotengine/godot/pull/66927 - [x] **PathFollow2D** https://github.com/godotengine/godot/pull/46277 - [x] **Path3D** - [x] **PathFollow3D** https://github.com/godotengine/godot/pull/46341 - [x] **PCKPacker** https://github.com/godotengine/godot/pull/43572 - [x] **Plane** https://github.com/godotengine/godot/pull/44492 - [x] **PrimitiveMesh** (CapsuleMesh, CubeMesh, …) - [x] **Projection** https://github.com/godotengine/godot/pull/77275 - [x] **Quaternion** https://github.com/godotengine/godot/pull/50907 - [x] **RandomNumberGenerator** https://github.com/godotengine/godot/pull/44560 - [x] **Rect2:** Not covered in the legacy math test. https://github.com/godotengine/godot/pull/43514 - [x] **Rect2i:** Not covered in the legacy math test. https://github.com/godotengine/godot/pull/43514 - [x] **RegEx** https://github.com/godotengine/godot/pull/82225 - [x] **RID** https://github.com/godotengine/godot/pull/54325 - [x] **Resource** https://github.com/godotengine/godot/pull/43731 - [x] **Shortcut** https://github.com/godotengine/godot/pull/58712 - [x] **SpriteFrames** https://github.com/godotengine/godot/pull/57742 - [x] **Transform** - [x] **Transform2D** - [x] **Translation** https://github.com/godotengine/godot/pull/48778 - [x] **TranslationServer** https://github.com/godotengine/godot/pull/79331 - [x] **Variant** https://github.com/godotengine/godot/pull/76244 - [x] **Vector** https://github.com/godotengine/godot/pull/48821 - [x] **Vector2/Vector2i** https://github.com/godotengine/godot/pull/47202 - [x] **Vector3/Vector3i** https://github.com/godotengine/godot/pull/48063 - [x] **Vector4/Vector4i** https://github.com/godotengine/godot/pull/64027 - [x] **VisualShader** https://github.com/godotengine/godot/pull/70396

Non-testable classes

These classes can't be unit-tested for technical reasons. Unit tests always run in headless mode, so they can't do things such as rendering scenes and checking the visual result.

- [x] ~~**CubeMap:**~~ Not testable without RenderingServer access. - [x] ~~**Shader:**~~ Not testable without RenderingServer access.
titus125 commented 1 year ago

Hi, new contributer here. I want to add some more methods in Transform3D covering methods that are currently not being tested such as the inverse functions and interpolate_with. Or is Transform3D no longer accepting new tests?

Calinou commented 1 year ago

Hi, new contributer here. I want to add some more methods in Transform3D covering methods that are currently not being tested such as the inverse functions and interpolate_with. Or is Transform3D no longer accepting new tests?

Feel free to open a pull request to add more tests to an existing class :slightly_smiling_face:

samkueb commented 1 year ago

Hello! Another new contributor here. Was hoping to add some tests that raise the coverage of the Path2D and Projection classes.

Calinou commented 1 year ago

Hello! Another new contributor here. Was hoping to add some tests that raise the coverage of the Path2D and Projection classes.

Feel free to open a pull request for these :slightly_smiling_face:

SiddharthChillale commented 1 year ago

Hello! This is my first time contributing! I'd like to add more coverage for Plane-Point operations for the Plane class specifically to test distance_to() method of the Plane class.

SiddharthChillale commented 1 year ago

I've started work on improving coverage of the distance_of() method of Plane class in #75602

kgrubbs22 commented 1 year ago

Hello! I'm a first-time contributor and I would like to add unit tests for Curve3D and make sure the overload operators in Variant are all tested.

SiddharthChillale commented 1 year ago

I want to take up writing unit-tests for the Cubemap class. I've got the basic test file setup and will work on adding a basic test case to it. It should also serve as a good starting point for future test-cases for the Cubemap class. Please feel free to give suggestions or pitfalls to look for :)

Calinou commented 1 year ago

Following this conversation on the Godot Contributors Chat, testing CubeMap isn't feasible without RenderingServer access. This singleton isn't available in the testing environment as headless builds don't instance a RenderingServer, so I'll strike it out from the list.

SiddharthChillale commented 1 year ago

I'm taking up the task of adding coverage for HTTPClient class.

brno32 commented 1 year ago

Path2D is unchecked but wasn't it addressed in https://github.com/godotengine/godot/pull/66927?

Calinou commented 1 year ago

Path2D is unchecked but wasn't it addressed in #66927?

Good catch, fixed :slightly_smiling_face:

StefanoPetrilli commented 1 year ago

Hey @samkueb, are you still working on Projection's tests? I am also starting to work on it. If you wish we can pair and tackle it together :)

samkueb commented 1 year ago

@StefanoPetrilli Nope, I ended up just working on Path2d. Feel free to work on Projection!

brno32 commented 1 year ago

DynamicFont has been renamed FontFile in Godot 4, right? Should the checklist above reflect that?

brno32 commented 1 year ago

Input itself doesn't have any tests. Should it be on here?

azuloo commented 1 year ago

Hi, I would like to see if we can cover TranslationServer. Is anybody working on it?

noelg-cj commented 1 year ago

Hey! I would like to work on NoiseTexture. Please do tell me if someone is already working on it. I'm new to unit testing and contributing and would really appreciate it if someone gave me pointers on where to start working on.

Sauermann commented 1 year ago

When using gcovr 5.2 for coverage-reports as explained in https://github.com/godotengine/godot/issues/43440#issuecomment-1305518762, I had to experiment, in order to get it working with Clang. The following worked for me on Linux:

scons use_llvm=yes target=editor dev_mode=yes dev_build=yes use_coverage=yes '-j X'
./bin/godot.linuxbsd.editor.dev.x86_64.llvm --test
gcovr --gcov-executable "llvm-cov gcov" --exclude-directories thirdparty/ --exclude-directories tests/ -e .*.gen.h --decisions --html-details coverage.html

image

Line coverage went up from 16% to 23% since Nov 2022 🤩

BrunoArmondBraga commented 1 year ago

Hi everyone, I'm interested in making my first contribution with tests for the ReGex class. Just wanted to comment to make sure it's not being worked on!

matorin57 commented 11 months ago

Howdy everyone. Looking to making my first contribution, just noting here that I'll be grabbing Font tests

menteValidus commented 11 months ago

I looked into AnimatedTexture class and for stable(4.1) it shows Deprecated. As it won't be updated and can be removed in future, is there still any need in tests for this class? Sorry, don't know who to tag here.

Calinou commented 11 months ago

I looked into AnimatedTexture class and for stable(4.1) it shows Deprecated. As it won't be updated and can be removed in future, is there still any need in tests for this class? Sorry, don't know who to tag here.

I think we can still add tests for it. It has to remain supported throughout the 4.x cycle after all.

menteValidus commented 11 months ago

I looked into AnimatedTexture class and for stable(4.1) it shows Deprecated. As it won't be updated and can be removed in future, is there still any need in tests for this class? Sorry, don't know who to tag here.

I think we can still add tests for it. It has to remain supported throughout the 4.x cycle after all.

Ok, thanks. I'll work on this then.

azuloo commented 11 months ago

@Calinou Hi, can you please check out Translation Server so it won't confuse anyone who would want to take it? It is covered by #79331. Thank you!

bgaudiosi commented 11 months ago

I wrote some tests for ImageTexture which should be linked to this issue now (though they're not merged at the time of me writing this). I ran into a snag I was able to solve and wanted to call it out here.

Some of these classes require the RenderingServer for testing. At one point, that was not available and some of these tests were impossible to write, but I don't believe that is the case any longer - adding [SceneTree] to your unit test name would initialize the RenderingEngine: https://github.com/godotengine/godot/blob/fe5b1c8d49313d63fbe91cb7cdf463e10fb86afa/tests/test_main.cpp#L223

I think that should be refactored to be more extensible (i.e. it doesn't make sense for random tests to add [SceneTree] to their name, but it does work.

Practically for this issue, I think that means CubeMap can be reopened as a thing that can be tested. I'm happy to prove that out myself.

menteValidus commented 11 months ago

I've encountered the same problem with RenderingServer. As you say it's configured in the test_main.cpp with

https://github.com/godotengine/godot/blob/fe5b1c8d49313d63fbe91cb7cdf463e10fb86afa/tests/test_main.cpp#L223

I personally think that such configuration should happen in the test fixture itself but if we stick to the approach that is currently is in Godot tests - we need to add test environment configuration if (name.find("[YourTestingClass]") != -1) { in the test_main.cpp for each test fixture we create.

For example, I tested AnimatedTexture and my every test has [AnimatedTexture] tag, so I just add this for configuration

menteValidus commented 11 months ago

@bgaudiosi, sorry forgot to mention you in my response

AThousandShips commented 10 months ago

AABB would benefit from more comprehensive testing of intersections, no testing is done for the clip point and normal vector for example

Phoenixcresset commented 10 months ago

Hey everyone, we are a small student group looking to make our first contribution. We're looking to create a Unit Test for Camera2D, what is meant by "headless mode" ?

AThousandShips commented 10 months ago

Means there's no renderer active, so limited or no graphics code will work

chocola-mint commented 10 months ago

Hello, I'm currently working on tests for Camera3D. I'll be testing the methods that can work in headless mode, namely:

Using SubViewport should be enough to give Camera3D a fake screen to calculate aspect ratio off of.

SilicDev commented 10 months ago

I'd be interested to work more on Variant tests. From what I can see the NOT operator has been tested already and there seem to be some EQUAL comparisions already via identity_compare, although the actual == operator seems to use hash_compare (at least in C++). So I have a few questions:

  1. Would it make sense to make another test case for hash_compare?
  2. Should the comparision test cases only consider same type comparisions or should comparisions between i. e. Vector2 and Vector2i also be considered?
  3. A lot of operators only apply to a subset of valid Variant types and often only for the same type, should a failure in other cases also be tested?
chocola-mint commented 10 months ago

I'm almost done with unit tests for Camera3D. There are just a few issues left to go over before I can submit the PR for review:

brno32 commented 9 months ago

From what I can tell, there is no DynamicFont class in the codebase and it was deleted in https://github.com/godotengine/godot/commit/99666de00fb30cb86473257776504ca70b4469c3#diff-35b4cc3280195a381c5e95d48a19f9d95c968ffae2604e3cc6897d3261ca9afa

Should this be removed from the list above, or replaced with a different class?

Calinou commented 9 months ago

From what I can tell, there is no DynamicFont class in the codebase and it was deleted in 99666de#diff-35b4cc3280195a381c5e95d48a19f9d95c968ffae2604e3cc6897d3261ca9afa

Should this be removed from the list above, or replaced with a different class?

FontFile is the 4.x replacement of DynamicFont. I've updated the list accordingly.

wilsonchen02 commented 9 months ago

Hey everyone, I'm interested in trying to make some test cases for the AnimatedTexture class for my first contribution to Godot. I've seen some of the docs about unit tests and would like to know if there are any pointers to keep in mind for that class specifically. I saw in the ImageTexture PR #82086 that several test images were generated for the test suite, and I'm wondering if this needs to be done for AnimatedTexture as well. Thanks!

menteValidus commented 9 months ago

Hey everyone, I'm interested in trying to make some test cases for the AnimatedTexture class for my first contribution to Godot. I've seen some of the docs about unit tests and would like to know if there are any pointers to keep in mind for that class specifically. I saw in the ImageTexture PR #82086 that several test images were generated for the test suite, and I'm wondering if this needs to be done for AnimatedTexture as well. Thanks!

Hey, I've actually wrote tests in my fork for the all fields of AnimatedTexture but got stuck on testing _update_proxy method. It depends on OS::get_singleton()->get_ticks_usec() and I wasn't sure at the moment how to mock it. Then I got busy at job and forgot about this for a month :) If you have an idea how to mock OS and write tests for _update_proxy I'll pass it to you. In _update_proxy happens all main logic and without it these tests I wrote are really "low value". On this week I'll attempt one more on cracking this one or asking help on dev chat.

wilsonchen02 commented 9 months ago

Hey @menteValidus, apologies for not seeing your earlier comments in the issue😅. I am not familiar with mocking OS either. I'll avoid stepping on your toes and try taking a crack at GradientTexture instead. Thanks for the quick reply and letting me know about your progress.

matorin57 commented 9 months ago

Do we currently have a runtime mock system? I’ve made some before where you can register mocks at test time via a string name (or you could use a function pointer interpreted as a int) and then add a line or two of macros at the top that in test builds that keeps track of keys to mock and if set will run some closure instead of the function itself

brno32 commented 9 months ago

InputEvent is unchecked but I believe tests were added in https://github.com/godotengine/godot/pull/79444

alex-buchanan commented 8 months ago

Hello All, as a new contributor, I'd like to tackle NoiseTexture, PackedScene, and PacketPeer. If they're still available to claim, and what's the status on Font and what's still required to check off that test?

Calinou commented 8 months ago

Hello All, as a new contributor, I'd like to tackle NoiseTexture, PackedScene, and PacketPeer.

Feel free to open pull requests for these :slightly_smiling_face:

and what's the status on Font and what's still required to check off that test?

Font has already been tackled by https://github.com/godotengine/godot/pull/81503. I've updated the list accordingly.

jrobb-scottlogic commented 8 months ago

Hello, I was looking to make my first contribution to the project and thought this would be a good issue to start with. Is there any particular unit test that would be best suited to a first time contributor or should I just dive in? Thanks :)

jasperhale342 commented 7 months ago

I'd be interested to work more on Variant tests. From what I can see the NOT operator has been tested already and there seem to be some EQUAL comparisions already via identity_compare, although the actual == operator seems to use hash_compare (at least in C++). So I have a few questions:

  1. Would it make sense to make another test case for hash_compare?
  2. Should the comparision test cases only consider same type comparisions or should comparisions between i. e. Vector2 and Vector2i also be considered?
  3. A lot of operators only apply to a subset of valid Variant types and often only for the same type, should a failure in other cases also be tested?

@SilicDev Did you start this one? I was going to try it if it's still open.

TheSofox commented 7 months ago

I linked this issue but just to state for the record: I added some unit tests for UndoRedo https://github.com/godotengine/godot/pull/85434

It's a pretty important and sophisticated system so surprised tests didn't exist for it already. May add more tests in the future if these ones are accepted.

ramadm commented 7 months ago

@Phoenixcresset are you still working on Camera2D? I'd like to give it a try if not.

Phoenixcresset commented 7 months ago

@Phoenixcresset are you still working on Camera2D? I'd like to give it a try if not.

We unfortunately reached a halt in our progress, so feel free to try your luck ! I can send you what we had done so far if you wish !

ramadm commented 7 months ago

@Phoenixcresset are you still working on Camera2D? I'd like to give it a try if not.

We unfortunately reached a halt in our progress, so feel free to try your luck ! I can send you what we had done so far if you wish !

That would be great, thank you.

andrewargyrakis commented 7 months ago

Hello, as a new contributor, I would like to try adding some unit tests for FontFile if that is still available.