search

whylabs / whylogs-java

Profile and monitor your ML data pipeline end-to-end
Apache License 2.0
177 stars 7 forks source link

Gradle Build is Failing While Building a Fat JAR #33

Open momohime-cloud opened 3 years ago

momohime-cloud commented 3 years ago

I tried to follow the tutorial here (https://docs.whylabs.ai/docs/spark-integration) to integrate whylogs with spark. Gradle build is failing with 100 errors. Here is the some of the traceback:

/Users/jadeyun/project/whylogs/whylogs_spark/whylogs-java/core/src/main/java/com/whylogs/core/statistics/datatypes/StringTracker.java:4: error: package com.whylogs.core.message does not exist import com.whylogs.core.message.StringsMessage; ^ /Users/jadeyun/project/whylogs/whylogs_spark/whylogs-java/core/src/main/java/com/whylogs/core/statistics/NumberTracker.java:7: error: package com.whylogs.core.message does not exist import com.whylogs.core.message.NumbersMessage; ^ /Users/jadeyun/project/whylogs/whylogs_spark/whylogs-java/core/src/main/java/com/whylogs/core/statistics/datatypes/VarianceTracker.java:3: error: package com.whylogs.core.message does not exist import com.whylogs.core.message.VarianceMessage; ^ /Users/jadeyun/project/whylogs/whylogs_spark/whylogs-java/core/src/main/java/com/whylogs/core/statistics/datatypes/DoubleTracker.java:3: error: package com.whylogs.core.message does not exist import com.whylogs.core.message.DoublesMessage;

Would you please take a look?

andyndang commented 3 years ago

Yes. This is because the build cannot find the protobuf generated definitions.

Can you try:

git submodule init
git submodule update

This will update the proto submodule (under proto path in https://github.com/whylabs/whylogs-java/tree/mainline/core) - please verify that you can see protobuf definitions there

We have a WIP to unify the repos and avoid submodule workflows so hopefully this won't be as much friction in the future.