macOS ARM builds?

[simonw]

I tried running ./datasette.com from https://github.com/ahgamut/superconfigure/releases/tag/z0.0.1 on an M2 Mac and got this message:

./datasette.com: this ape binary lacks arm64 support

Would it be possible to include arm64 support in these releases, so they can run on Apple Silicon?

[ahgamut]

(cc @jart) The linker step for constructing APE binaries is currently being updated: https://github.com/jart/cosmopolitan/commit/0105e3e2b64f03262c522162a9b9f3159edd5648

I'll add a release with Apple Silicon support as soon as I update the superconfigure scripts in this repo to build using the new linker step . Thanks!

[ahgamut]

@simonw there's a new release built with the new apelink.com linker: https://github.com/ahgamut/superconfigure/releases/tag/z0.0.2

[ingenieroariel]

Thanks @ahgamut !

z0.0.2 works for me on M2 Air with OSX.

and on a Mac Studio M1 Ultra with Asahi Linux via NixOS.

[simonw]

Just tried that - ./datasette.com got me a Python interpreter but trying to launch the application gave me this error:

./datasette.com -m datasette

Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "Lib/site-packages/datasette/__main__.py", line 1, in <module>
  File "Lib/site-packages/datasette/cli.py", line 17, in <module>
  File "Lib/site-packages/datasette/app.py", line 14, in <module>
  File "Lib/site-packages/pkg_resources/__init__.py", line 1031, in <module>
  File "Lib/site-packages/pkg_resources/__init__.py", line 1035, in Environment
  File "Lib/site-packages/pkg_resources/__init__.py", line 195, in get_supported_platform
  File "Lib/site-packages/pkg_resources/__init__.py", line 438, in get_build_platform
  File "Lib/sysconfig.py", line 793, in get_platform
  File "Lib/sysconfig.py", line 670, in get_config_vars
  File "Lib/sysconfig.py", line 531, in _init_posix
ModuleNotFoundError: No module named '_sysconfigdata__linux_aarch64-linux-gnu'

[ahgamut]

By unzip -vl datasette.com, we see that the _sysconfigdata__linux_aarch64-linux-gnu is not present in the ZIP store of datasette.com, which is why the module import is failing.

I think this is what's happening: I build the aarch64 first, and then make clean before building x86_64, which means that sysconfigdata file for aarch64 gets deleted.

I've pushed a fix and started another build on GA.

[ahgamut]

Okay, the latest release now should import from the right libraries. @ingenieroariel can you test it with ./datasette.com -v and then import sysconfig in the console?

[ingenieroariel]

@ahgamut sysconfig imported fine, but datasette could not find it's own version.

When I looked inside the zip file, it was full of .pyc files, but I could not find the original version.py

import _frozen_importlib # frozen
import _imp # builtin
import '_thread' # <class '_frozen_importlib.BuiltinImporter'>
import '_warnings' # <class '_frozen_importlib.BuiltinImporter'>
import '_weakref' # <class '_frozen_importlib.BuiltinImporter'>
import '_io' # <class '_frozen_importlib.BuiltinImporter'>
import 'marshal' # <class '_frozen_importlib.BuiltinImporter'>
import 'posix' # <class '_frozen_importlib.BuiltinImporter'>
import '_frozen_importlib_external' # <class '_frozen_importlib.FrozenImporter'>
# installing zipimport hook
import 'time' # <class '_frozen_importlib.BuiltinImporter'>
import 'zipimport' # <class '_frozen_importlib.FrozenImporter'>
# installed zipimport hook
# code object from '/zip/Lib/encodings/__init__.pyc'
import '_codecs' # <class '_frozen_importlib.BuiltinImporter'>
import 'codecs' # <class '_frozen_importlib.FrozenImporter'>
# code object from '/zip/Lib/encodings/aliases.pyc'
import 'encodings.aliases' # <_frozen_importlib_external.SourcelessFileLoader object at 0x1000801d7f50>
import 'encodings' # <_frozen_importlib_external.SourcelessFileLoader object at 0x1000801d4750>
# code object from '/zip/Lib/encodings/utf_8.pyc'
import 'encodings.utf_8' # <_frozen_importlib_external.SourcelessFileLoader object at 0x1000801d6050>
import '_signal' # <class '_frozen_importlib.BuiltinImporter'>
import '_abc' # <class '_frozen_importlib.BuiltinImporter'>
import 'abc' # <class '_frozen_importlib.FrozenImporter'>
import 'io' # <class '_frozen_importlib.FrozenImporter'>
import '_stat' # <class '_frozen_importlib.BuiltinImporter'>
import 'stat' # <class '_frozen_importlib.FrozenImporter'>
import '_collections_abc' # <class '_frozen_importlib.FrozenImporter'>
import 'genericpath' # <class '_frozen_importlib.FrozenImporter'>
import 'posixpath' # <class '_frozen_importlib.FrozenImporter'>
import 'os' # <class '_frozen_importlib.FrozenImporter'>
import '_sitebuiltins' # <class '_frozen_importlib.FrozenImporter'>
Processing user site-packages
Processing global site-packages
import 'site' # <class '_frozen_importlib.FrozenImporter'>
import 'readline' # <class '_frozen_importlib.BuiltinImporter'>
# code object from '/zip/Lib/rlcompleter.pyc'
import 'atexit' # <class '_frozen_importlib.BuiltinImporter'>
# code object from '/zip/Lib/inspect.pyc'
# code object from '/zip/Lib/ast.pyc'
import '_ast' # <class '_frozen_importlib.BuiltinImporter'>
# code object from '/zip/Lib/contextlib.pyc'
# code object from '/zip/Lib/collections/__init__.pyc'
import 'itertools' # <class '_frozen_importlib.BuiltinImporter'>
# code object from '/zip/Lib/keyword.pyc'
import 'keyword' # <_frozen_importlib_external.SourcelessFileLoader object at 0x100080393c90>
# code object from '/zip/Lib/operator.pyc'
import '_operator' # <class '_frozen_importlib.BuiltinImporter'>
import 'operator' # <_frozen_importlib_external.SourcelessFileLoader object at 0x1000803a8210>
# code object from '/zip/Lib/reprlib.pyc'
import 'reprlib' # <_frozen_importlib_external.SourcelessFileLoader object at 0x1000803a8e10>
import '_collections' # <class '_frozen_importlib.BuiltinImporter'>
import 'collections' # <_frozen_importlib_external.SourcelessFileLoader object at 0x100080373b10>
# code object from '/zip/Lib/functools.pyc'
# code object from '/zip/Lib/types.pyc'
import 'types' # <_frozen_importlib_external.SourcelessFileLoader object at 0x1000803bf390>
import '_functools' # <class '_frozen_importlib.BuiltinImporter'>
import 'functools' # <_frozen_importlib_external.SourcelessFileLoader object at 0x1000803810d0>
import 'contextlib' # <_frozen_importlib_external.SourcelessFileLoader object at 0x100080372ad0>
# code object from '/zip/Lib/enum.pyc'
import 'enum' # <_frozen_importlib_external.SourcelessFileLoader object at 0x1000803339d0>
import 'ast' # <_frozen_importlib_external.SourcelessFileLoader object at 0x100080332250>
# code object from '/zip/Lib/dis.pyc'
# code object from '/zip/Lib/opcode.pyc'
import '_opcode' # <class '_frozen_importlib.BuiltinImporter'>
import 'opcode' # <_frozen_importlib_external.SourcelessFileLoader object at 0x10008050de90>
import 'dis' # <_frozen_importlib_external.SourcelessFileLoader object at 0x1000803408d0>
# code object from '/zip/Lib/collections/abc.pyc'
import 'collections.abc' # <_frozen_importlib_external.SourcelessFileLoader object at 0x10008050dd50>
# code object from '/zip/Lib/importlib/__init__.pyc'
# code object from '/zip/Lib/warnings.pyc'
import 'warnings' # <_frozen_importlib_external.SourcelessFileLoader object at 0x10008051d550>
import 'importlib' # <_frozen_importlib_external.SourcelessFileLoader object at 0x1000805154d0>
import 'importlib.machinery' # <class '_frozen_importlib.FrozenImporter'>
# code object from '/zip/Lib/linecache.pyc'
# code object from '/zip/Lib/tokenize.pyc'
# code object from '/zip/Lib/re/__init__.pyc'
# code object from '/zip/Lib/re/_compiler.pyc'
import '_sre' # <class '_frozen_importlib.BuiltinImporter'>
# code object from '/zip/Lib/re/_parser.pyc'
# code object from '/zip/Lib/re/_constants.pyc'
import 're._constants' # <_frozen_importlib_external.SourcelessFileLoader object at 0x10008053e610>
import 're._parser' # <_frozen_importlib_external.SourcelessFileLoader object at 0x10008053b790>
# code object from '/zip/Lib/re/_casefix.pyc'
import 're._casefix' # <_frozen_importlib_external.SourcelessFileLoader object at 0x10008054e050>
import 're._compiler' # <_frozen_importlib_external.SourcelessFileLoader object at 0x100080538490>
# code object from '/zip/Lib/copyreg.pyc'
import 'copyreg' # <_frozen_importlib_external.SourcelessFileLoader object at 0x10008054f290>
import 're' # <_frozen_importlib_external.SourcelessFileLoader object at 0x100080532810>
# code object from '/zip/Lib/token.pyc'
import 'token' # <_frozen_importlib_external.SourcelessFileLoader object at 0x1000805324d0>
import 'tokenize' # <_frozen_importlib_external.SourcelessFileLoader object at 0x10008051e410>
import 'linecache' # <_frozen_importlib_external.SourcelessFileLoader object at 0x100080515810>
import 'inspect' # <_frozen_importlib_external.SourcelessFileLoader object at 0x100080303f10>
import 'rlcompleter' # <_frozen_importlib_external.SourcelessFileLoader object at 0x100080302fd0>
Python 3.11.4 (heads/datasette:dd9dadf, Aug 20 2023, 03:12:45) [GCC 11.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sysconfig
# code object from '/zip/Lib/sysconfig.pyc'
import 'sysconfig' # <_frozen_importlib_external.SourcelessFileLoader object at 0x10008056b490>
>>> import datasette
# code object from '/zip/Lib/site-packages/datasette/__init__.pyc'
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<frozen importlib._bootstrap>", line 1189, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1160, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 940, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "Lib/site-packages/datasette/__init__.py", line 1, in <module>
  File "<frozen importlib._bootstrap>", line 1189, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1153, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'datasette.version'

[ahgamut]

unzip -vl datasette.com | grep "datasette" shows there is Lib/site-packages/datasette/version.pyc within the zip store.

sys.path shows /zip/Lib/site-packages. I wonder why it's not getting imported.

[ingenieroariel]

got a similar behavior on windows

[ahgamut]

confirmed on Linux as well. yeah this is probably a python stdlib/APE zip store thing. Will search for the cause+fix.

[ingenieroariel]

Fixed for me on the latest release!

[ingenieroariel]

Below is a screenshot of datasette.com loading a 30GB sqlite file with a billion records on aarch64-linux.

[ahgamut]

Closing because we've built fat binaries for a while now, that build on both x86_64 and aarch64