search

facebook / buck

A fast build system that encourages the creation of small, reusable modules over a variety of platforms and languages.
https://buck.build
Apache License 2.0
8.56k stars 1.16k forks source link

M1 Support #2666

Open shepting opened 2 years ago

shepting commented 2 years ago

This tracking ticket can state the current issues blocking builds on M1 (Apple Silicon) machines.

Currently, when running on an M1 machine we will get output like this:

╰─ ~/.buck/buck build //ios:AirbnbApp
Unable to connect to Buck daemon, restarting it...
Not using buckd because daemon failed to start.
Exception in thread "main" java.lang.reflect.InvocationTargetException
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:566)
    at com.facebook.buck.cli.bootstrapper.ClassLoaderBootstrapper.main(ClassLoaderBootstrapper.java:55)
Caused by: java.lang.UnsatisfiedLinkError: /Users/stevenhepting/workspace/apps/buck-out/tmp/buck_run.82_1dr8k/jna-1265810726/jna2060014269842800161.tmp: dlopen(/Users/stevenhepting/workspace/apps/buck-out/tmp/buck_run.82_1dr8k/jna-1265810726/jna2060014269842800161.tmp, 0x0001): tried: '/Users/stevenhepting/workspace/apps/buck-out/tmp/buck_run.82_1dr8k/jna-1265810726/jna2060014269842800161.tmp' (fat file, but missing compatible architecture (have 'i386,x86_64', need 'arm64e')), '/usr/lib/jna2060014269842800161.tmp' (no such file)

I believe the relevant snippet is (fat file, but missing compatible architecture (have 'i386,x86_64', need 'arm64e'))

OleksiyA commented 2 years ago

Workaround is to install buck using Rosetta terminal. Afterwards buck can be used from any terminal as usual.

Follow section Configuring Rosetta for ARM64 MacBook https://github.com/valhalla/valhalla#configuring-rosetta-for-arm64-macbook . After that use Rosetta terminal with ibrew and follow section Build from Source from https://buck.build/setup/getting_started.html .

EvenChang commented 2 years ago

Excuse me, I have follow your step and also have same error msg. Does anyone knows?

MacBook-Pro:buck even$ arch
i386
MacBook-Pro:buck even$ ./bin/buck build
Unable to connect to Buck daemon, restarting it...
Not using buckd because daemon failed to start.
Exception in thread "main" java.lang.reflect.InvocationTargetException
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at com.facebook.buck.cli.bootstrapper.ClassLoaderBootstrapper.main(ClassLoaderBootstrapper.java:55)
Caused by: java.lang.UnsatisfiedLinkError: /Users/even/Library/Caches/JNA/temp/jna1285587138928287604.tmp: dlopen(/Users/even/Library/Caches/JNA/temp/jna1285587138928287604.tmp, 0x0001): tried: '/Users/even/Library/Caches/JNA/temp/jna1285587138928287604.tmp' (fat file, but missing compatible architecture (have 'i386,x86_64', need 'arm64e')), '/usr/lib/jna1285587138928287604.tmp' (no such file)
    at java.lang.ClassLoader$NativeLibrary.load(Native Method)
    at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1934)
    at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1817)
    at java.lang.Runtime.load0(Runtime.java:810)
    at java.lang.System.load(System.java:1088)
    at com.sun.jna.Native.loadNativeDispatchLibraryFromClasspath(Native.java:1018)
    at com.sun.jna.Native.loadNativeDispatchLibrary(Native.java:988)
    at com.sun.jna.Native.<clinit>(Native.java:195)
    at com.facebook.buck.cli.MainRunner.<clinit>(MainRunner.java:287)
    at com.facebook.buck.cli.AbstractMain.prepareMainRunner(AbstractMain.java:147)
    at com.facebook.buck.cli.MainWithoutNailgun.main(MainWithoutNailgun.java:56)
    ... 5 more
gabrieldonadel commented 2 years ago

Excuse me, I have follow your step and also have same error msg. Does anyone knows?

That's because you should also use an x64 JDK to build

hendych commented 2 years ago

Using rosetta installation to build buck.pex, will it able to run on native arm64 CPU or we still need to use rosetta to run?

I'm building using rosetta and when using native arm64, it's still having error (fat file, but missing compatible architecture (have 'i386,x86_64', need 'arm64e'))

Is there any way we can define in the compiler to also build specific CPU arch?

marcesengel commented 2 years ago

I succeeded doing a local build after updating a few dependencies, will create a PR later today 👍

marcesengel commented 2 years ago

@ajanuar I published my PR, feel free to try it :) but be aware, Java 8 support is dropped due to upstream changes.

hendych commented 2 years ago

Hi @marcesengel, were you able to build using iphonesimulator-arm64? I got this error when running it in some of my appex

Error scrubbing non-deterministic metadata from /Users/xxx/Widget#iphonesimulator-arm64
com.facebook.buck.io.file.FileScrubber.ScrubException: String table does not end at end of file
com.facebook.buck.io.file.FileScrubber$ScrubException: String table does not end at end of file
    at com.facebook.buck.cxx.toolchain.objectfile.OsoSymbolsContentsScrubber.scrubFile(OsoSymbolsContentsScrubber.java:41)
    at com.facebook.buck.step.fs.FileScrubberStep.execute(FileScrubberStep.java:58)
    at com.facebook.buck.step.StepRunner.runStep(StepRunner.java:58)
    at com.facebook.buck.core.build.engine.impl.CachingBuildRuleBuilder$BuildRuleSteps.executeCommands(CachingBuildRuleBuilder.java:1417)
    at com.facebook.buck.core.build.engine.impl.CachingBuildRuleBuilder$BuildRuleSteps.runWithDefaultExecutor(CachingBuildRuleBuilder.java:1394)
    at com.facebook.buck.util.concurrent.WeightedListeningExecutorService.lambda$submit$2(WeightedListeningExecutorService.java:100)
    at com.facebook.buck.util.concurrent.WeightedListeningExecutorService.lambda$submitWithSemaphore$0(WeightedListeningExecutorService.java:74)
    at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:206)
    at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:195)
    at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:115)
    at com.google.common.util.concurrent.MoreExecutors$5$1.run(MoreExecutors.java:999)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
    at java.base/java.lang.Thread.run(Thread.java:829)
marcesengel commented 2 years ago

Hi @hendych, actually you getting this error is good news for me 😅 I got the same error building react-native and assumed it was a problem with their repo. Seems like one of the version bumps broke the FileScrubber 🤔 I will try to look into it next week, do you think you could have a look as well?

Edit: to be completely honest with you I haven't ran the tests and assumed all was fine when I was able to build Buck on my M1. In case there are failing tests this could make fixing all of this a lot easier. If there aren't any I suppose we should add one.

hendych commented 2 years ago

Hi @hendych, actually you getting this error is good news for me 😅 I got the same error building react-native and assumed it was a problem with their repo. Seems like one of the version bumps broke the FileScrubber 🤔 I will try to look into it next week, do you think you could have a look as well?

Edit: to be completely honest with you I haven't ran the tests and assumed all was fine when I was able to build Buck on my M1. In case there are failing tests this could make fixing all of this a lot easier. If there aren't any I suppose we should add one.

I saw someone having the same issue in Buck slack channel without any solution though 😅. This issue should have nothing to do with the version bump of the third party libraries because this issue exist since building buck with rosetta and iphonesimulator-arm64.

Did check it before. I think there is something should be changed in the FileScrubber code. I think this related to the symbol generation in linking phase or something. Tried to use FileScrubber on the dev branch but I think the changes is too far with master. Didn't have the chance to look further due to another project. But will do after the project.

Solace-Studios commented 2 years ago

@marcesengel @hendych Any further luck using buck on M1?

gabrieldonadel commented 2 years ago

@marcesengel @hendych Any further luck using buck on M1?

@Solace-Studios I got this working on my M1, the most important part is to make sure you're using an x64 JDK and brew running with roseta

hendych commented 2 years ago

@marcesengel @hendych Any further luck using buck on M1?

@Solace-Studios I had applied code changes by @marcesengel, now I'm using Buck on M1 but cannot get it working with iphonesimulator-arm64. It can run buck CLI using native M1 but cannot compile and run tests using arm64 simulator. As workaround, I don't use buck test CLI, but using xctestrunner with iponesimulator-x86_64. And now we're considering whether to continue using Buck or migrate to Bazel 🤔 so I stopped looking for the appropriate solution atm.

@Solace-Studios I got this working on my M1, the most important part is to make sure you're using an x64 JDK and brew running with roseta

@gabrieldonadel Can you run tests on arm64 simulator?

gabrieldonadel commented 2 years ago

@gabrieldonadel Can you run tests on arm64 simulator?

@hendych I only use buck build TBH but it runs without any issues

marcesengel commented 1 year ago

Hi @marcesengel, were you able to build using iphonesimulator-arm64? I got this error when running it in some of my appex

Error scrubbing non-deterministic metadata from /Users/xxx/Widget#iphonesimulator-arm64
com.facebook.buck.io.file.FileScrubber.ScrubException: String table does not end at end of file
com.facebook.buck.io.file.FileScrubber$ScrubException: String table does not end at end of file
  at com.facebook.buck.cxx.toolchain.objectfile.OsoSymbolsContentsScrubber.scrubFile(OsoSymbolsContentsScrubber.java:41)
  at com.facebook.buck.step.fs.FileScrubberStep.execute(FileScrubberStep.java:58)
  at com.facebook.buck.step.StepRunner.runStep(StepRunner.java:58)
  at com.facebook.buck.core.build.engine.impl.CachingBuildRuleBuilder$BuildRuleSteps.executeCommands(CachingBuildRuleBuilder.java:1417)
  at com.facebook.buck.core.build.engine.impl.CachingBuildRuleBuilder$BuildRuleSteps.runWithDefaultExecutor(CachingBuildRuleBuilder.java:1394)
  at com.facebook.buck.util.concurrent.WeightedListeningExecutorService.lambda$submit$2(WeightedListeningExecutorService.java:100)
  at com.facebook.buck.util.concurrent.WeightedListeningExecutorService.lambda$submitWithSemaphore$0(WeightedListeningExecutorService.java:74)
  at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:206)
  at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:195)
  at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:115)
  at com.google.common.util.concurrent.MoreExecutors$5$1.run(MoreExecutors.java:999)
  at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
  at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
  at java.base/java.lang.Thread.run(Thread.java:829)

IIRC I got the same error after the build was successful...

@marcesengel @hendych Any further luck using buck on M1?

@Solace-Studios I got this working on my M1, the most important part is to make sure you're using an x64 JDK and brew running with roseta

My goal was to actually have it run natively, but no luck as of now.