search

uber / cadence

Cadence is a distributed, scalable, durable, and highly available orchestration engine to execute asynchronous long-running business logic in a scalable and resilient way.
https://cadenceworkflow.io
MIT License
8.14k stars 786 forks source link

cadence-workflow 0.20.0 build issue on apple arm64 #4162

Closed chenrui333 closed 3 years ago

chenrui333 commented 3 years ago

👋 trying to build the latest release, but run into some build issue. The error log is as below:

build failure ``` ==> make cadence cadence-server cadence-canary cadence-sql-tool cadence-cassandra-tool building revive from github.com/mgechev/revive... building goimports from golang.org/x/tools/cmd/goimports... downloading protoc 3.14.0 building protoc-gen-gofast from github.com/gogo/protobuf/protoc-gen-gofast... building protoc-gen-yarpc-go from go.uber.org/yarpc/encoding/protobuf/protoc-gen-yarpc-go... [.bin/protoc.zip] End-of-central-directory signature not found. Either this file is not a zipfile, or it constitutes one disk of a multi-part archive. In the latter case the central directory and zipfile comment will be found on the last disk(s) of this archive. unzip: cannot find zipfile directory in one of .bin/protoc.zip or .bin/protoc.zip.zip, and cannot find .bin/protoc.zip.ZIP, period. make: *** [.bin/protoc-3.14.0] Error 9 make: *** Waiting for unfinished jobs.... ```

Full build log is in here, https://github.com/Homebrew/homebrew-core/runs/2372116200 relates to https://github.com/Homebrew/homebrew-core/pull/75387

longquanzheng commented 3 years ago

Looks like the issue started with proto compiling: https://github.com/Homebrew/homebrew-core/pull/75196#issuecomment-819928251

longquanzheng commented 3 years ago

It seems fine in my MackBook Pro but it's in Intel processor. Do you know how can we fix it? or how can I reproduce it without getting a new Apple MacBook? 


(cadence-0.20)$ brew install --build-from-source cadence-workflow
==> Downloading https://ghcr.io/v2/homebrew/core/go/manifests/1.16.3
Already downloaded: /Users/qlong/Library/Caches/Homebrew/downloads/a0c7d5540c8c88f612cedc7d9af128deb15aa152ce0b0f2e1bcd3d8f235020f5--go-1.16.3.bottle_manifest.json
==> Downloading https://ghcr.io/v2/homebrew/core/go/blobs/sha256:69c28f5e60612801c66e51e93d32068f822b245ab83246cb6cb374572eb59e15
Already downloaded: /Users/qlong/Library/Caches/Homebrew/downloads/d74822e0d6c473f04ff2856cab284c0f3a55bc4edbd09f63f4c542a528d0495e--go--1.16.3.catalina.bottle.tar.gz
==> Downloading https://github.com/uber/cadence/archive/refs/tags/v0.20.0.tar.gz
Already downloaded: /Users/qlong/Library/Caches/Homebrew/downloads/80871d54ac664e80d22dd435c8a08f85def64c8e02f0cb5c069247c1b2b00af2--cadence-0.20.0.tar.gz
Warning: A newer Command Line Tools release is available.
Update them from Software Update in System Preferences or run:
  softwareupdate --all --install --force

If that doesn't show you any updates, run:
  sudo rm -rf /Library/Developer/CommandLineTools
  sudo xcode-select --install

Alternatively, manually download them from:
  https://developer.apple.com/download/more/.

==> Installing dependencies for cadence-workflow: go
==> Installing cadence-workflow dependency: go
==> Pouring go--1.16.3.catalina.bottle.tar.gz
🍺  /usr/local/Cellar/go/1.16.3: 9,955 files, 503.5MB
==> Installing cadence-workflow
==> make cadence cadence-server cadence-canary cadence-sql-tool cadence-cassandra-tool

🍺  /usr/local/Cellar/cadence-workflow/0.20.0: 10 files, 211MB, built in 2 minutes 25 seconds
Removing: /Users/qlong/Library/Caches/Homebrew/cadence-workflow--0.18.2.tar.gz... (3.3MB)
qlong@/usr/local/Homebrew/Library/Taps/homebrew/homebrew-core:
longquanzheng commented 3 years ago

@chenrui333 just a update, we are working to a fix and patch to 0.20.1 so that the formula will work

longquanzheng commented 3 years ago

Hi @Groxx , do you have any update on the fixes ? It would be nice to have proto as optional for build like thrift files, since we already checked in the generated go files.

Groxx commented 3 years ago

Nope, but I can probably get that makefile change up today. then a make .fake-codegen; make [the binaries] should work.

For arm issues specifically, I'm in line for an M1 mac, and might be able to reproduce eventually. I have no ETA for that though.

Groxx commented 3 years ago

4172 is up, should fix it :)

Groxx commented 3 years ago

ಠ_ಠ I really wish it'd confirm before auto-closing referenced things.

chenrui333 commented 3 years ago

@Groxx let me give this patch a try. Thanks!

chenrui333 commented 3 years ago

@Groxx still have build issue (full build log)

It looks like I still have some build issue.

building protoc-gen-yarpc-go from go.uber.org/yarpc/encoding/protobuf/protoc-gen-yarpc-go...
[.bin/protoc.zip]
  End-of-central-directory signature not found.  Either this file is not
  a zipfile, or it constitutes one disk of a multi-part archive.  In the
  latter case the central directory and zipfile comment will be found on
  the last disk(s) of this archive.
unzip:  cannot find zipfile directory in one of .bin/protoc.zip or
        .bin/protoc.zip.zip, and cannot find .bin/protoc.zip.ZIP, period.
make: *** [.bin/protoc-3.14.0] Error 9
make: *** Waiting for unfinished jobs....
Groxx commented 3 years ago

@chenrui333 great, at least the cause is pretty clear this time:

downloading protoc 3.14.0: https://github.com/protocolbuffers/protobuf/releases/download/v3.14.0/protoc-3.14.0-osx-arm64.zip

^ that url is a 404, so it's probably just an unvalid zip file. Looks like there are just no arm64 downloads at all on any release, so we might want to special-case those and download the x86_64 one instead.
If you want to try that, you can just hard-code the OSX download, and I believe that'll work - replace the PROTOC_URL with this: PROTOC_URL = https://github.com/protocolbuffers/protobuf/releases/download/v3.14.0/protoc-3.14.0-osx-x86_64.zip

But that said, this is missing an earlier step:

cp -pR /private/tmp/d20210502-76353-1ul9f55/revive.toml /private/tmp/cadence-workflow-20210502-76353-113xqar/revive.toml
chmod -Rf +w /private/tmp/d20210502-76353-1ul9f55
==> Patching
==> make cadence cadence-server cadence-canary cadence-sql-tool cadence-cassandra-tool
building revive from github.com/mgechev/revive...

^ you need a make .fake-codegen before make cadence .... That should skip the codegen-tool-building, and then this should all work.

longquanzheng commented 3 years ago

Hi @chenrui333 Just to give more details. What we decided is to add one more command before https://github.com/Homebrew/homebrew-core/blob/bfb30f4c62d2bbfae3e20cf271ba6aa2c9fc8f89/Formula/cadence-workflow.rb#L21

system "make", ".fake-codegen" Then it won't require the protoc anymore.

Groxx commented 3 years ago

make .fake-codegen has to be separate, and run before make cadence. doing both at once does the fakery too late to work reliably.

longquanzheng commented 3 years ago

make .fake-codegen has to be separate, and run before make cadence. doing both at once does the fakery too late to work reliably.

Yeah, right, I updated my comment

chenrui333 commented 3 years ago

Sounds good!

chenrui333 commented 3 years ago

it builds fine with the latest release, closing this issue. Thanks @Groxx and @longquanzheng !