eclipse-platform / eclipse.platform

https://eclipse.dev/eclipse/
Eclipse Public License 2.0
83 stars 113 forks source link

Reactivate tests: disabled tests collection #525

Open HeikoKlare opened 1 year ago

HeikoKlare commented 1 year ago

Current Situation

Several tests have been deactivated over the years for different reasons, such as temporary incompatibilities with a specific platform/OS, bugs in the code, indeterministic behavior of the test etc.

These disablements are done by adding some prefix to test... methods in JUnit 3, by removing or commenting out @Test in JUnit 4 and 5, or by using @Ignore or @Disabled. In addition, several tests are only deactivated for specific platforms using early returns or assume statements within the test methods.

Goal

Since in most cases there is no additional, unique documentation or tagging of these disablements, they are not easy to find. This issue is supposed to be an umbrella to document disabled tests (existing ones and potentially further ones).

The purpose of this issue is to collect the disabled tests to have a central overview and provide the chance to reactivate and, if necessary, fix them. It also serves as a "good first issue" (see https://github.com/eclipse-platform/eclipse.platform.text/pull/212#issuecomment-1594576748) for interested new contributors, which can take any of the collected tests and try to fix them to learn about the Eclipse platform code while, at the same time, already having the chance to make valuable contributions to the project.

Process

I see the following ideas as good starting points for identifying disabled tests:

Examples

Disabled in Maven

Some tests are disabled in Maven builds, such that integration builds do not reveal whether they fail or not.

Wittmaxi commented 1 year ago

Performing a naive search for _test, @Ignore, @Disabled, I found quite some artifacts which are worth adding to this issue

org.eclipse.core.tests.resources

org.eclipse.core.tests.runtime

Michael5601 commented 1 year ago

Addition to https://github.com/eclipse-platform/eclipse.platform/issues/525#issuecomment-1613014550. Checked tests have pending or accepted PR:

org.eclipse.core.tests.resources

org.eclipse.core.tests.runtime

org.eclipse.ant.tests.ui

org.eclipse.core.expressions.tests

org.eclipse.e4.core.tests

org.eclipse.debug.tests

org.eclipse.core.tests.net.NetTest

org.eclipse.team.tests.core

org.eclipse.tips.tests

org.eclipse.ua.tests