vterm-module compiles as x86_64 instead of arm64e on macOS M1

alex-eg commented 2 years ago

After update, vterm compiled vterm-lib as x86_64 module instead of arm64 on my Apple ARM macOS 12.0.1 with Emacs 29.0.50.

What Emacs reports on M-x vterm:

Error (use-package): vterm/:catch: Module could not be opened:
"/Users/ex/.emacs.d/elpa/vterm-20220429.21/vterm-module.so",
"dlopen(/Users/ex/.emacs.d/elpa/vterm-20220429.21/vterm-module.so, 0x0009):
tried: '/Users/ex/.emacs.d/elpa/vterm-20220429.21/vterm-module.so' (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64e')),
'/usr/local/lib/vterm-module.so' (no such file),
'/usr/lib/vterm-module.so' (no such file)"

Confirmed via terminal:

» file ./vterm-module.so
./vterm-module.so: Mach-O 64-bit bundle x86_64

Install log (from Emacs buffer):

-- The C compiler identification is AppleClang 13.1.6.13160021
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- No build type selected, defaulting to RelWithDebInfo
-- System libvterm not found: libvterm will be downloaded and compiled as part of the build process
-- Configuring done
-- Generating done
-- Build files have been written to: /Users/ex/.emacs.d/elpa/vterm-20220429.21/build
[  8%] Creating directories for 'libvterm'
[ 16%] Performing download step (git clone) for 'libvterm'
Cloning into 'libvterm'...
HEAD is now at 54c03b2 Set _XOPEN_SOURCE 600 unconditionally in vterm-ctrl.c
[ 25%] Performing update step for 'libvterm'
[ 33%] No patch step for 'libvterm'
[ 41%] No configure step for 'libvterm'
[ 50%] Performing build step for 'libvterm'
TBL src/encoding/DECdrawing.tbl
TBL src/encoding/uk.tbl
CC src/encoding.c
CC src/keyboard.c
CC src/mouse.c
CC src/parser.c
CC src/pen.c
CC src/screen.c
CC src/state.c
CC src/unicode.c
src/unicode.c:215:12: warning: unused function 'mk_wcswidth' [-Wunused-function]
static int mk_wcswidth(const uint32_t *pwcs, size_t n)
           ^
src/unicode.c:306:12: warning: unused function 'mk_wcswidth_cjk' [-Wunused-function]
static int mk_wcswidth_cjk(const uint32_t *pwcs, size_t n)
           ^
2 warnings generated.
CC src/vterm.c
LINK libvterm.la
CC bin/unterm.c
CC bin/vterm-ctrl.c
bin/vterm-ctrl.c:129:10: warning: result of comparison of constant 156 with expression of type 'char' is always false [-Wtautological-constant-out-of-range-compare]
    if(c == 0x9c) // ST
       ~ ^  ~~~~
1 warning generated.
CC bin/vterm-dump.c
[ 58%] No install step for 'libvterm'
[ 66%] Completed 'libvterm'
[ 66%] Built target libvterm
[ 75%] Building C object CMakeFiles/vterm-module.dir/vterm-module.c.o
[ 83%] Building C object CMakeFiles/vterm-module.dir/utf8.c.o
[ 91%] Building C object CMakeFiles/vterm-module.dir/elisp.c.o
[100%] Linking C shared module ../vterm-module.so
[100%] Built target vterm-module
/Users/ex/.emacs.d/elpa/vterm-20220429.21

Interestingly enough, Emacs' shell reports arch to be i386:

Welcome to the Emacs shell

~ $ arch
i386
~ $ gcc -v
Apple clang version 13.1.6 (clang-1316.0.21.2.3)
Target: x86_64-apple-darwin21.1.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin

Perhaps a bug in Emacs for MacOS itself.

Current workaround is to manually compile vterm-module from terminal. After that M-x vterm starts to work properly.

~/
» cd ~/.emacs.d/elpa/vterm-20220429.21/

~/.emacs.d/elpa/vterm-20220429.21
» arch
arm64

~/.emacs.d/elpa/vterm-20220429.21
» rm -rf build

~/.emacs.d/elpa/vterm-20220429.21
» mkdir build; cmake -B build; cmake --build build
-- The C compiler identification is AppleClang 13.1.6.13160021
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- No build type selected, defaulting to RelWithDebInfo
-- System libvterm not found: libvterm will be downloaded and compiled as part of the build process
-- Configuring done
-- Generating done
<... build log ...>
[ 66%] Built target libvterm
[ 75%] Building C object CMakeFiles/vterm-module.dir/vterm-module.c.o
[ 83%] Building C object CMakeFiles/vterm-module.dir/utf8.c.o
[ 91%] Building C object CMakeFiles/vterm-module.dir/elisp.c.o
[100%] Linking C shared module ../vterm-module.so
[100%] Built target vterm-module

~/.emacs.d/elpa/vterm-20220429.21
» file ./vterm-module.so
./vterm-module.so: Mach-O 64-bit bundle arm64

danilevy1212 commented 2 years ago

Thank you for posting but this solution didn't work for me either.


$ arch                                                                                                                master
arm64
$ mkdir build; cmake -B build; cmake --build build                                                                    master
-- The C compiler identification is AppleClang 13.1.6.13160021
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /Library/Developer/CommandLineTools/usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- No build type selected, defaulting to RelWithDebInfo
-- System libvterm not found: libvterm will be downloaded and compiled as part of the build process
-- Configuring done
-- Generating done
-- Build files have been written to: /Users/dlevy/.config/emacs/.local/straight/build-28.1/vterm/build
[  8%] Creating directories for 'libvterm'
[ 16%] Performing download step (git clone) for 'libvterm'
Cloning into 'libvterm'...
HEAD is now at 54c03b2 Set _XOPEN_SOURCE 600 unconditionally in vterm-ctrl.c
[ 25%] Performing update step for 'libvterm'
[ 33%] No patch step for 'libvterm'
[ 41%] No configure step for 'libvterm'
[ 50%] Performing build step for 'libvterm'
TBL src/encoding/DECdrawing.tbl
TBL src/encoding/uk.tbl
CC src/encoding.c
CC src/keyboard.c
CC src/mouse.c
CC src/parser.c
CC src/pen.c
CC src/screen.c
CC src/state.c
CC src/unicode.c
src/unicode.c:215:12: warning: unused function 'mk_wcswidth' [-Wunused-function]
static int mk_wcswidth(const uint32_t *pwcs, size_t n)
           ^
src/unicode.c:306:12: warning: unused function 'mk_wcswidth_cjk' [-Wunused-function]
static int mk_wcswidth_cjk(const uint32_t *pwcs, size_t n)
           ^
2 warnings generated.
CC src/vterm.c
LINK libvterm.la
CC bin/unterm.c
CC bin/vterm-ctrl.c
bin/vterm-ctrl.c:129:10: warning: result of comparison of constant 156 with expression of type 'char' is always false [-Wtautological-constant-out-of-range-compare]
    if(c == 0x9c) // ST
       ~ ^  ~~~~
1 warning generated.
CC bin/vterm-dump.c
[ 58%] No install step for 'libvterm'
[ 66%] Completed 'libvterm'
[ 66%] Built target libvterm
[ 75%] Building C object CMakeFiles/vterm-module.dir/vterm-module.c.o
[ 83%] Building C object CMakeFiles/vterm-module.dir/utf8.c.o
[ 91%] Building C object CMakeFiles/vterm-module.dir/elisp.c.o
[100%] Linking C shared module ../vterm-module.so
ld: warning: ignoring file libvterm-prefix/src/libvterm/.libs/libvterm.a, building for macOS-x86_64 but attempting to link with file built for macOS-arm64
Undefined symbols for architecture x86_64:
  "_vterm_color_is_equal", referenced from:
      _refresh_lines in vterm-module.c.o
  "_vterm_free", referenced from:
      _term_finalize in vterm-module.c.o
  "_vterm_get_size", referenced from:
      _goto_col in vterm-module.c.o
  "_vterm_input_write", referenced from:
      _Fvterm_write_input in vterm-module.c.o
  "_vterm_keyboard_end_paste", referenced from:
      _Fvterm_update in vterm-module.c.o
  "_vterm_keyboard_key", referenced from:
      _Fvterm_update in vterm-module.c.o
  "_vterm_keyboard_start_paste", referenced from:
      _Fvterm_update in vterm-module.c.o
  "_vterm_keyboard_unichar", referenced from:
      _Fvterm_update in vterm-module.c.o
  "_vterm_new", referenced from:
      _Fvterm_new in vterm-module.c.o
  "_vterm_obtain_screen", referenced from:
      _Fvterm_new in vterm-module.c.o
  "_vterm_obtain_state", referenced from:
      _Fvterm_new in vterm-module.c.o
      _term_redraw in vterm-module.c.o
      _goto_col in vterm-module.c.o
      _refresh_lines in vterm-module.c.o
      _cell_rgb_color in vterm-module.c.o
  "_vterm_output_get_buffer_current", referenced from:
      _term_flush_output in vterm-module.c.o
  "_vterm_output_read", referenced from:
      _term_flush_output in vterm-module.c.o
  "_vterm_screen_enable_altscreen", referenced from:
      _Fvterm_new in vterm-module.c.o
  "_vterm_screen_flush_damage", referenced from:
      _Fvterm_update in vterm-module.c.o
      _Fvterm_write_input in vterm-module.c.o
      _Fvterm_set_size in vterm-module.c.o
  "_vterm_screen_get_cell", referenced from:
      _goto_col in vterm-module.c.o
      _refresh_lines in vterm-module.c.o
  "_vterm_screen_is_eol", referenced from:
      _goto_col in vterm-module.c.o
      _refresh_lines in vterm-module.c.o
  "_vterm_screen_reset", referenced from:
      _Fvterm_new in vterm-module.c.o
  "_vterm_screen_set_callbacks", referenced from:
      _Fvterm_new in vterm-module.c.o
  "_vterm_screen_set_damage_merge", referenced from:
      _Fvterm_new in vterm-module.c.o
  "_vterm_set_size", referenced from:
      _Fvterm_set_size in vterm-module.c.o
  "_vterm_set_utf8", referenced from:
      _Fvterm_new in vterm-module.c.o
  "_vterm_state_get_cursorpos", referenced from:
      _term_redraw in vterm-module.c.o
  "_vterm_state_get_default_colors", referenced from:
      _goto_col in vterm-module.c.o
      _refresh_lines in vterm-module.c.o
  "_vterm_state_get_palette_color", referenced from:
      _cell_rgb_color in vterm-module.c.o
  "_vterm_state_set_bold_highbright", referenced from:
      _Fvterm_new in vterm-module.c.o
  "_vterm_state_set_unrecognised_fallbacks", referenced from:
      _Fvterm_new in vterm-module.c.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
gmake[2]: *** [CMakeFiles/vterm-module.dir/build.make:130: ../vterm-module.so] Error 1
gmake[1]: *** [CMakeFiles/Makefile2:86: CMakeFiles/vterm-module.dir/all] Error 2
gmake: *** [Makefile:91: all] Error 2

andorsk commented 1 year ago

I hit the same issue.

andorsk commented 1 year ago

This is happening because arch is set not to arm64 on laptop:

Fixed it the following way:

Make sure arch is arm64. env /usr/bin/arch -arm64 /bin/zsh --login
Recompile
mkdir -p build; cd build; cmake ..; make;

Result:

➜  emacs-libvterm git:(master) file vterm-module.so
vterm-module.so: Mach-O 64-bit bundle arm64
➜  emacs-libvterm git:(master)

Logs:

➜  build git:(master) cmake ..
-- The C compiler identification is AppleClang 13.1.6.13160021
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- No build type selected, defaulting to RelWithDebInfo
-- System libvterm detected
-- Configuring done
-- Generating done
-- Build files have been written to: /Users/akmb2/Downloads/emacs-libvterm/build
➜  build git:(master) make
[ 25%] Building C object CMakeFiles/vterm-module.dir/vterm-module.c.o
[ 50%] Building C object CMakeFiles/vterm-module.dir/utf8.c.o
[ 75%] Building C object CMakeFiles/vterm-module.dir/elisp.c.o
[100%] Linking C shared module /Users/akmb2/Downloads/emacs-libvterm/vterm-module.so
[100%] Built target vterm-module

And voila, it works:

amolgawai commented 1 year ago

Interestingly doom emacs manages to compile correct vterm-module.so. All the above workarounds didn't work for me, so I am relying on doom emacs to compile the correct one and just copy the resulting so file to my personal emacs config.

amolgawai commented 1 year ago

Another solution is https://github.com/akermu/emacs-libvterm/issues/471#issuecomment-1214274231

vvzen commented 10 months ago

In my case (M1 Max machine), forcing cmake to run under arm64 worked, despite the fact that my shell was not running under rosetta:

$ cd ~/.config/emacs/.local/straight/build-29.1/vterm
$ rm -rf build
$ mkdir build
$ arch -arm64 cmake -B build
$ arch -arm64 cmake --build build

$ file vterm-module.so
vterm-module.so: Mach-O 64-bit bundle arm64

My emacs was already installed via a arm64 brew:

$ file $(which emacs)
/opt/homebrew/bin/emacs: Bourne-Again shell script text executable, ASCII text

$ cat /opt/homebrew/bin/emacs
#!/bin/bash
exec /opt/homebrew/Cellar/emacs-plus@29/29.1/Emacs.app/Contents/MacOS/Emacs "$@"

$ file /opt/homebrew/Cellar/emacs-plus@29/29.1/Emacs.app/Contents/MacOS/Emacs
/opt/homebrew/Cellar/emacs-plus@29/29.1/Emacs.app/Contents/MacOS/Emacs: Mach-O 64-bit executable arm64

I had installed it via brew install emacs-plus@29 --with-native-comp --with-poll, and I'm also using Doom.

sstepashka commented 7 months ago

Hi,

Is there a way, maybe add a flag or somehow automatically detect the current Emacs architecture and compile it in the same arch? Or, maybe, build a universal .so which contains both arch64 and x86_64?

pradyungn commented 6 months ago

Can confirm that @vvzen's fix worked on my system.

gruns commented 6 months ago

problem still persists 🙂; x86_64 version of libterm gets built on m1 mac

% file ~/.emacs.d/elpa/vterm-20240102.1640/vterm-module.so
~/.emacs.d/elpa/vterm-20240102.1640/vterm-module.so: Mach-O 64-bit x86_64 bundle

resulting in dlopen() failure:

Debugger entered--Lisp error: (module-open-failed "~/.emacs.d/elpa/vterm-20240102.1640/vterm-module.so" "dlopen(~/.emacs.d/elpa/vterm-20240102.1640/vterm-module.so, 0x0009): tried: '~/.emacs.d/elpa/vterm-20240102.1640/vterm-module.so' ...")
  require(vterm-module)
  byte-code("..." boolean defvaralias vterm-clear-scrollback vterm-clear-scrollback-when-clearing ...] 10)
  autoload-do-load((autoload "vterm" "Create an interactive Vterm buffer.\nStart a new Vt..." t nil) vterm)
  command-execute(vterm record)
  execute-extended-command(nil "vterm" "vte")
  funcall-interactively(execute-extended-command nil "vterm" "vte")
  call-interactively(execute-extended-command nil nil)
  command-execute(execute-extended-command)

@vvzen's solution here https://github.com/akermu/emacs-libvterm/issues/593#issuecomment-1754747095 above worked for me, too. thank you, @vvzen 🙏

akermu / emacs-libvterm

vterm-module compiles as x86_64 instead of arm64e on macOS M1 #593