[DSIP-10][Unit Tests] Improve DolphinScheduler unit tests

EricGao888 commented 2 years ago

Search before asking

[X] I had searched in the issues and found no similar feature requirement.

Description

Unit Tests of high quality could not only improve the stability of Dolphin Scheduler, but also increase the readability of Dolphin Scheduler Code.

Currently we only have around 40% UT coverage. Although DS already has E2E tests and the community are working on the API tests, it never means we do not need high quality unit tests.
As a DS developer, every time after submitting a PR, we will see the current UT coverage of the whole project. If the coverage is too low, developers may have concerns about the stability of the application.
Reading unit tests is one of the best approaches for new developers to understand the code. UT cases also help reviewers to review PR more easily. Furthermore, as it is hardly possible to provide docs of every detailed feature of DS, the best way is to keep UT updated.

Use case

We could have different standards for UT for different parts of DS. For example, we have higher bar on UT for core features but lower bar for plugins.
We could offer a list of UT cases which need to be updated, added or refactored and label them with good first issue. For new developers to this community, writing UT is a good way to understand the code logic and it is more intriguing than refactoring docs.

Related issues

related: #3700
related: #10963

Action Items

[x] Add Spotless plugin into DolphinScheduler to enable developers to fix style / formatting errors incrementally: #10963
[x] Remove dependency on Powermock: #11405
[x] Upgrade jUnit dependency from 4.x to 5.x: #10976
[x] Migrate all legacy cases from jUnit 4 to jUnit 5 in the whole project: #12301
[x] Add CI checks to prevent developer from writing UTs with jUnit4 and Powermock.
[ ] Refactor typical UTs in each module as examples.
[ ] Add docs of instructions about how to write UT in DolphinScheduler project.
[ ] Create some easy UT refactoring tasks and label them with good first issue.
[ ] Refactor all UTs in each module complying to method coverage criteria and fix the style & formatting errors of related test code and tested code. This one will be divided into sub-tasks later.

Refactoring Guideline

We use method coverage as refactoring metric to decouple UT cases as much as possible.
We refactor UTs with jUnit 5 instead of jUnit 4.
We should avoid using Powermock in UTs and remove legacy UTs which contains Powermock, see: #11405 .
Scope of each PR:

Progress

Arranged by priority:

Module	Sub-Module	Class
dolphinscheduler-api
dolphinscheduler-master
dolphinscheduler-worker
dolphinscheduler-common
dolphinscheduler-service
dolphinscheduler-alert	dolphinscheduler-alert-server	AlertServer
dolphinscheduler-task-plugin

Appendix: UT coverage of each module before refactoring

Are you willing to submit a PR?

[X] Yes I am willing to submit a PR!

Code of Conduct

[X] I agree to follow this project's Code of Conduct

github-actions[bot] commented 2 years ago

Thank you for your feedback, we have received your issue, Please wait patiently for a reply.

In order for us to understand your request as soon as possible, please provide detailed information、version or pictures.
If you haven't received a reply for a long time, you can join our slack and send your question to channel #troubleshooting

SbloodyS commented 2 years ago

That's great!

EricGao888 commented 2 years ago

That's great!

@SbloodyS Thx. Hope this will help and attract more new developers to submit PRs for DS and offer them an approach to get familiar with the code smoothly.

davidzollo commented 2 years ago

great

EricGao888 commented 2 years ago

IMHO, we need some priorities, we could apply high priority to those modules where UT might play a more significant role regarding to code quality and application stability.

EricGao888 commented 2 years ago

@SbloodyS Could you please help add a discussion label to this issue? Much appreciated~

SbloodyS commented 2 years ago

IMHO, we need some priorities, we could apply high priority to those modules where UT might play a more significant role regarding to code quality and application stability.

We can make a list and mark it with priority.

EricGao888 commented 2 years ago

I got blocked by some unit test when developing a new feature and mocking some unusual stuff for nearly a day. I would like to update the UT docs with some examples which may help new developers bypass or overcome some pitfalls in UTs. May I ask whether it is possible to migrate this docs into main repo? @zhongjiajie https://dolphinscheduler.apache.org/en-us/community/development/unit-test.html

Thanks

zhongjiajie commented 2 years ago

I got blocked by some unit test when developing a new feature and mocking some unusual stuff for nearly a day. I would like to update the UT docs with some examples which may help new developers bypass or overcome some pitfalls in UTs. May I ask whether it is possible to migrate this docs into main repo? @zhongjiajie https://dolphinscheduler.apache.org/en-us/community/development/unit-test.html

Thanks

Of cause, we can, do you interesting in this migration?

EricGao888 commented 2 years ago

I got blocked by some unit test when developing a new feature and mocking some unusual stuff for nearly a day. I would like to update the UT docs with some examples which may help new developers bypass or overcome some pitfalls in UTs. May I ask whether it is possible to migrate this docs into main repo? @zhongjiajie https://dolphinscheduler.apache.org/en-us/community/development/unit-test.html Thanks

Of cause, we can, do you interesting in this migration?

Yes, I'd love to give a shot.

EricGao888 commented 2 years ago

BTW, we could fix the formatting and style errors incrementally, together with unit tests. See: #10963 One convenient way is when refactoring some UTs, we fix the formatting and style errors of both the testing and tested part.

EricGao888 commented 2 years ago

I've generated test coverage reports for every module of DS using Intellij, sorted by method coverage. Here are some results:

EricGao888 commented 2 years ago

I will update this issue with a more detailed refactoring plan as well as priorities later this week.

stalary commented 2 years ago

I can get involved in the junit upgrade~ @EricGao888

EricGao888 commented 2 years ago

I can get involved in the junit upgrade~ @EricGao888

@stalary Currently we are working on removing powermock because it does not support jUnit 5 and blocks the upgrade. Would u like to participate in #11405 ? Thanks~

stalary commented 2 years ago

Currently we are working on removing powermock because it does not support jUnit 5 and blocks the upgrade. Would u like to participate in

Okay, I can be a part of that.

EricGao888 commented 2 years ago

Currently we are working on removing powermock because it does not support jUnit 5 and blocks the upgrade. Would u like to participate in

Okay, I can be a part of that.

@stalary Great! We have #12150 open to contributors. Would u like to pick it up? Thanks

stalary commented 2 years ago

Currently we are working on removing powermock because it does not support jUnit 5 and blocks the upgrade. Would u like to participate in

Okay, I can be a part of that.

@stalary Great! We have #12150 open to contributors. Would u like to pick it up? Thanks

Please assign to me, thanks~

EricGao888 commented 2 years ago

Usage of Powermock has been fully removed from the whole project : )

zhongjiajie commented 2 years ago

Usage of Powermock has been fully removed from the whole project :

Great new

apache / dolphinscheduler