search

twitter / hadoop-lzo

Refactored version of code.google.com/hadoop-gpl-compression for hadoop 0.20
GNU General Public License v3.0
546 stars 329 forks source link

New maven version with AArch64 binary #133

Open rhenwood-arm opened 6 years ago

rhenwood-arm commented 6 years ago

I see AArch64 support arrived in this library about a year ago [1]. It seems the most recent 0.4.20 release in Maven includes this change but is missing the AArch64 native binary [2]. Can this issue be resolved?

  1. https://github.com/twitter/hadoop-lzo/commit/5dc97e902a689ce459c9025444a7715a518fe3fb.
  2. https://mvnrepository.com/artifact/com.hadoop.gplcompression/hadoop-lzo/0.4.20
sjlee commented 6 years ago

We did add support for ARM so it can build cleanly on that arch. That said, publishing an artifact to maven that includes binaries on that architecture is a separate matter. It would not be feasible to build for all kinds of OS flavors in the binary Twitter publishes to the maven central. Thus, it was a pretty deliberate decision on our part to build only for the Linux 64 bit. We don't even build for Mac or Windows.

If you need binaries for your platform, please build it from the source and use that instead.

I am no longer at Twitter, so I'd defer the follow-up to Twitter's Hadoop team.

rhenwood-arm commented 6 years ago

Thanks for the background and prompt reply.

Twitter's Hadoop team: do you have thoughs on this issue?

ctrezzo commented 6 years ago

I think @sjlee 's answer is pretty spot on. If you wanted to build and host binaries for ARM separately, I could see adding a section to the read me pointing users to those binaries.

rhenwood-arm commented 6 years ago

Thanks for the prompt reply.

My goal is for AArch64 users, like myself, to follow the same instructions as everyone else and end up with an optimal solution. To that end, can you explain why it is not feasible to build for Linux AArch64?

I am highly motivated to understand the problem here so I can help develop a solution.

best regards Richard

vielmetti commented 6 years ago

If resources are needed for the build and host process, I'll offer up the Works on Arm cluster - details at http://worksonarm.com/cluster and/or http://github.com/worksonarm/cluster .

I'm not familiar with the production build environment - are the binaries being created by Travis right now?