search

tony-framework / TonY

TonY is a framework to natively run deep learning frameworks on Apache Hadoop.
https://tony-project.ai
Other
708 stars 164 forks source link

Guava Conflict? #627

Closed kevincloutier closed 2 years ago

kevincloutier commented 2 years ago

Has anyone experienced the well-known Preconditions.checkArgument error (below) when running the standard mnist example? Often this points to an incompatibility with quava, though I have found two guava jars within Hadoop, both are 27 (common/lib/guava-27.0-jre.jar and hdfs/lib/guava-27.0-jre.jar).

Any help would be appreciated.

Exception in thread "main" java.lang.NoSuchMethodError: com.google.common.base.Preconditions.checkArgument(ZLjava/lang/String;Ljava/lang/Object;)V at org.apache.hadoop.conf.Configuration.set(Configuration.java:1357) at org.apache.hadoop.conf.Configuration.set(Configuration.java:1338) at org.apache.hadoop.conf.Configuration.setInt(Configuration.java:1515) at org.apache.hadoop.yarn.client.api.impl.ContainerManagementProtocolProxy.<init>(ContainerManagementProtocolProxy.java:95) at org.apache.hadoop.yarn.client.api.impl.NMClientImpl.serviceInit(NMClientImpl.java:142) at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164) at org.apache.hadoop.yarn.client.api.async.impl.NMClientAsyncImpl.serviceInit(NMClientAsyncImpl.java:140) at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164) at com.linkedin.tony.TonyApplicationMaster.prepare(TonyApplicationMaster.java:363) at com.linkedin.tony.TonyApplicationMaster.main(TonyApplicationMaster.java:309)

kevincloutier commented 2 years ago

I believe I see the issue, the jar I was using for TonY had guava 11, it was a precompiled jar. Unfortunately I can't compile TonY on my platform (arm) due to tony-core requiring com.google.protobuf:protoc:2.5.0, which doesn't exist for arm. Has anyone been able to compile TonY on arm? If so, any pointers would be appreciated.

kevincloutier commented 2 years ago

I was able to overcome my issue by building the cli on Ubuntu x86 and using that jar on ARM.