search

antlr / antlr5

BSD 3-Clause "New" or "Revised" License
57 stars 5 forks source link

Fix local testing #27

Closed ericvergnaud closed 9 months ago

ericvergnaud commented 9 months ago

@ftomassetti this works for me, can you check the same ?

(this PR doesn't fix the maven plugin tests, which would require help from @jimidle)

ftomassetti commented 9 months ago

I tried to run the tests locally, I am not sure if the PR is expected to fix running tests, but just in case this is useful this is what I get.

I tried to:

This fails locally:

[INFO] ANTLR 5 ............................................ SUCCESS [  0.224 s]
[INFO] ANTLR 5 Runtime .................................... SUCCESS [  4.604 s]
[INFO] ANTLR 5 Kotlin Runtime ............................. SUCCESS [ 17.654 s]
[INFO] ANTLR 5 Tool ....................................... SUCCESS [  5.553 s]
[INFO] ANTLR 5 Maven plugin ............................... FAILURE [  5.269 s]
[INFO] ANTLR 5 Runtime Tests (4th generation) ............. SKIPPED
[INFO] ANTLR 5 Tool Tests ................................. SKIPPED

If I then move into runtime tests and run mvn test. I get some failues.

For example:

[ERROR] Tests run: 663, Failures: 28, Errors: 0, Skipped: 0, Time elapsed: 73.374 s <<< FAILURE! - in org.antlr.v5.test.runtime.java.api.TestExpectedTokens

If I then move into tool tests and run mvn test that succeeds.

ericvergnaud commented 9 months ago

Thanks We'll have to understand why the runtime tests fail locally when they succeed in the CI, but that's progress

ericvergnaud commented 9 months ago

If I run: mvn -Dtest='java.**' test or mvn -Dtest='kotlin.**' test

I suspect that mvn test fails because it also tries running ts/js tests, which haven't been migrated. (we're only keeping the code here for later use)

jimidle commented 9 months ago

I am in the middle of relocating so I will be out until next week. But I hve locally revamped the whole Maven stuff and fixed the build anomalies. I'll have to check it. There is also an old plugin that I will have to rewrite as it is for an older version of Maven. Hopefully I can get back to this soon.

Jim

On Wed, Feb 7, 2024 at 7:23 AM ericvergnaud @.***> wrote:

@ftomassetti https://github.com/ftomassetti this works for me, can you check the same ?

(this PR doesn't fix the maven plugin tests, which would require help from @jimidle https://github.com/jimidle)

You can view, comment on, or merge this pull request online at:

https://github.com/antlr/antlr5/pull/27 Commit Summary

File Changes

(12 files https://github.com/antlr/antlr5/pull/27/files)

Patch Links:

— Reply to this email directly, view it on GitHub https://github.com/antlr/antlr5/pull/27, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAJ7TMDUNVSPT3IDM4DRLV3YSO2C3AVCNFSM6AAAAABC6AX2HKVHI2DSMVQWIX3LMV43ASLTON2WKOZSGEZDGMRXG43DEOI . You are receiving this because you were mentioned.Message ID: @.***>

ftomassetti commented 9 months ago

Just as an update, I get the exact same results from two different machines (both macs)

lppedd commented 9 months ago

I'll give it ago this afternoon, let's see if I can repro too.

lppedd commented 9 months ago

I can run tests correctly with:

cd runtime-testsuite 
mvn -Dtest=kotlin.** test
mvn -Dtest=java.** test

Which is what CI does.

Running mvn test fails tho, it seems on JS tests, which we don't need anyway.

ftomassetti commented 9 months ago

Yes, that works (with the slight changes of adding " around the -Dtest parameter).

Are instead the tests of the maven plugin expected to fail?

lppedd commented 9 months ago

Are instead the tests of the maven plugin expected to fail?

Plugin's tests use takari-plugin-testing, which received its last meaningful update more than three years ago. At this point testing with anything above Maven 3.6 is likely to fail.