[Bug]: Cannot build and test on M1 Mac

smukil commented 5 days ago

Related Template(s)

spanner

Template Version

v1

What happened?

Running mvn test -f pom.xml does not run to completion. Here is the snippet of the error I'm seeing.

...
[INFO] Tests run: 22, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 9.426 s - in org.apache.beam.it.gcp.spanner.SpannerResourceManagerTest
[INFO] 
[INFO] Results:
[INFO] 
[ERROR] Errors: 
[ERROR]   FlexTemplateDataflowJobResourceManagerTest.testBuildMavenStageCommand:40 Mockito
[INFO] 
[ERROR] Tests run: 216, Failures: 0, Errors: 1, Skipped: 0
[INFO] 
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary for Google Cloud Dataflow Templates 1.0-SNAPSHOT:
[INFO] 
[INFO] Google Cloud Dataflow Templates .................... SUCCESS [  0.542 s]
[INFO] Templates Metadata Module .......................... SUCCESS [  2.252 s]
[INFO] Templates Plugins Project .......................... SUCCESS [  0.036 s]
[INFO] Templates Core Plugin .............................. SUCCESS [  2.777 s]
[INFO] Templates Maven Plugin ............................. SUCCESS [  5.181 s]
[INFO] integration-testing-lib ............................ SUCCESS [  0.153 s]
[INFO] it-google-cloud-platform ........................... FAILURE [ 57.661 s]
[INFO] it-datadog ......................................... SKIPPED
...

If I remove that test, more tests fail. I am also not able to run a single test like so: mvn test -f pom.xml -Dtest=DdlToAvroSchemaConverterTest

If I run the test directly from IntelliJ, I get the following error: java: package com.datastax.oss.driver.api.querybuilder does not exist on an unrelated file CassandraColumnSchema.java

The only way I've been able to develop so far is to have a parallel branch and delete about 30 offending tests that do not build, so that I can build and run the tests I want. Eg: https://github.com/smukil/DataflowTemplates/commit/f92f22fffcc738305d4187441e2bae3822d1141c

This doesn't seem sustainable and slows down the entire dev process. Any help here would be greatly appreciated. Thank you for your time!

Relevant log output

No response

smukil commented 5 days ago

Note that mvn install also gives the same error:

...
[INFO] Results:
[INFO] 
[ERROR] Errors: 
[ERROR]   FlexTemplateDataflowJobResourceManagerTest.testBuildMavenStageCommand:40 Mockito
[INFO] 
[ERROR] Tests run: 216, Failures: 0, Errors: 1, Skipped: 0
[INFO] 
...

damccorm commented 2 days ago

Could you try https://github.com/GoogleCloudPlatform/DataflowTemplates/blob/main/contributor-docs/code-contributions.md#apple-m1-development

GoogleCloudPlatform / DataflowTemplates