Provider v6.42.0+ crashing on Apple Silicon when running with Rosetta2

[flostadler]

Describe what happened

Multiple user reports in the Community Slack mention crashes of the AWS provider versions v.6.42.0+ on Apple Silicon.

This is the error users are getting:

Diagnostics:
  pulumi:pulumi:Stack (quickstart-dev):
    assertion failed [arm_interval().contains(address)]: code fragment does not contain the given arm address
    (CodeFragmentMetadata.cpp:48 instruction_extents_for_arm_address)

CodeFragmentMetadata.cpp seems to be part of Rosetta2, Apple's binary translator for running x86 binaries on Apple Silicon:

strings /Library/Apple/usr/libexec/oah/runtime | grep CodeFragmentMetadata.cpp
CodeFragmentMetadata.cpp

But the output of pulumi about indicates that affected users are running the arm64 version of pulumi.

As a workaround users can downgrade to the AWS provider version v.6.41.0 which is reported to be unaffected by this issue.

Sample program

n/a

Log output

n/a

Affected Resource(s)

No response

Output of pulumi about

LI
Version      3.122.0
Go Version   go1.22.4
Go Compiler  gc

Plugins
KIND      NAME    VERSION
language  nodejs  unknown

Host
OS       darwin
Version  14.5
Arch     arm64

This project is written in nodejs: executable='[redacted]/.nvm/versions/node/v18.20.1/bin/node' version='v18.20.1'

Current Stack: [redacted]

TYPE                 URN
pulumi:pulumi:Stack  [redacted]

Found no pending operations associated with [redacted]

Backend
Name           [redacted]
URL            s3://[redacted]
User           [redacted]
Organizations

Additional context

No response

Contributing

Vote on this issue by adding a 👍 reaction. To contribute a fix for this issue, leave a comment (and link to your pull request, if you've opened one already).

[flostadler]

I tried reproducing it, but wasn't able to trigger that same error on my Mac so far. Digging deeper what change in v6.42.0 could've caused this and how I could reproduce it. From a first glance it might be: https://github.com/pulumi/pulumi-aws/commit/00ba5ea53c2548fd367776345b7c52f191ce1372. That change bumped the Go version from 1.22.2 to 1.22.4

[flostadler]

More users came back to us on the community slack that they accidentally installed pulumi in x86_64 mode on Apple Silicon Macs. Reinstalling pulumi with arm64 architecture and cleaning the existing cached plugins (rm -rf ~/.pulumi) solved the problem for them.

If you're running into this issue, please check whether the installed Pulumi binary has the correct architecture (file "$(which pulumi)"). If you're installing Pulumi with brew, also check whether brew is running in x86_64 mode (brew config). If it is, you can switch to the arm64 native version of brew by following those steps: https://github.com/orgs/Homebrew/discussions/545#discussioncomment-540891

[flostadler]

I just managed to reproduce it by upgrading to Mac OS 14.5.

I was able to get an x86_64 version of the provider (v6.42.0+) working by setting GODEBUG=asyncpreemptoff=1. E.g. GODEBUG=asyncpreemptoff=1 pulumi up --yes

Please switch to an arm64 version of Pulumi and the Providers if you can. If you can't try setting GODEBUG=asyncpreemptoff=1.

[mjeffryes]

Closing as "by design"; not a perfect label, since there is technically an upstream bug in go on rossetta2 in OSX 14.5m, but we do intentionally provide binaries for apple silicon so that rosetta2 support is not necessary.

[eff-kay]

Could not get the x86 pulumi to work in rosetta.

Downloaded the arm binaries, moved them to /usr/local/bin. Did brew uninstall pulumi. Open a new arm based terminal. Ran the worker there and it works.

EDIT 1: actually nvm, the issue still persists even with the arm architecture.