Combined issues for running Katrain on 2020 Mac M1 architectures

[kentdorsey]

I worked through several dependency issues during installation, including sdl2* and libavcodec (resolved latter via brew install ffmpeg). However, at runtime, the following error appears:

% katrain -d
[INFO   ] [Logger      ] Record log in /Users/REDACTED/.kivy/logs/kivy_21-01-20_24.txt
[INFO   ] [Kivy        ] v2.0.0
[INFO   ] [Kivy        ] Installed at "/opt/homebrew/lib/python3.9/site-packages/kivy/__init__.py"
[INFO   ] [Python      ] v3.9.1 (default, Jan  8 2021, 12:11:08) 
[Clang 12.0.0 (clang-1200.0.32.28)]
[INFO   ] [Python      ] Interpreter at "/opt/homebrew/opt/python@3.9/bin/python3.9"
[ERROR  ] [Clock       ] Unable to import kivy._clock. Have you perhaps forgotten to compile kivy? Kivy contains Cython code which needs to be compiled. A missing kivy._clock often indicates the Cython code has not been compiled. Please follow the installation instructions and make sure to compile Kivy
 Traceback (most recent call last):
   File "/opt/homebrew/bin/katrain", line 5, in <module>
     from katrain.__main__ import run_app
   File "/opt/homebrew/lib/python3.9/site-packages/katrain/__main__.py", line 48, in <module>
     from kivy.base import ExceptionHandler, ExceptionManager
   File "/opt/homebrew/lib/python3.9/site-packages/kivy/base.py", line 28, in <module>
     from kivy.clock import Clock
   File "/opt/homebrew/lib/python3.9/site-packages/kivy/clock.py", line 466, in <module>
     from kivy._clock import CyClockBase, ClockEvent, FreeClockEvent, \
 ImportError: dlopen(/opt/homebrew/lib/python3.9/site-packages/kivy/_clock.cpython-39-darwin.so, 2): no suitable image found.  Did find:
    /opt/homebrew/lib/python3.9/site-packages/kivy/_clock.cpython-39-darwin.so: mach-o, but wrong architecture
    /opt/homebrew/lib/python3.9/site-packages/kivy/_clock.cpython-39-darwin.so: mach-o, but wrong architecture

[sanderland]

Yes, the gui framework is not always very happy with mac, and in this case it's a new architecture released after the latest version. Have a look at https://github.com/kivy/kivy/issues/7265

[vividfog]

@kentdorsey Kivy's master branch now contains a fix for this. Early access: pip install "kivy[base] @ https://github.com/kivy/kivy/archive/master.zip"

Thanks for mentioning the ffmpeg dependency, I too was struggling to find what the missing piece was.

With these, katrain runs otherwise fine, but it looks like the top bar of the GUI is missing. Topic for another issue perhaps? @sanderland feel free to suggest how to debug or gather data in my end, so I can create an issue for this. As far as I'm concerned, the issue at hand is now fixed upstream and will be part of kivy 2.1.

katrain-log-2021-01-23-1236.txt

Also, the UI is huge. I have to set the M1 Air resolution to max and full screen in order to see all of the UI.

[sanderland]

Good to see it's fixed in kivy. So coordinates are missing, icons at the bottom for navigation, and fonts look a bit out of alignment.

• Is the grey region part of the window, and can it be resized?
• If you set debug_level=1 in general settings and restart, is there anything in the log that points to trouble (weird font provider/window provider, errors, missing files, etc)

[kentdorsey]

Thanks @vividfog I installed the kiva fix. When attempting to run katrain, I received the following:

[INFO   ] [Logger      ] Record log in /Users/kentdorsey/.kivy/logs/kivy_21-01-23_4.txt
[INFO   ] [Kivy        ] v2.1.0.dev0
[INFO   ] [Kivy        ] Installed at "/opt/homebrew/lib/python3.9/site-packages/kivy/__init__.py"
[INFO   ] [Python      ] v3.9.1 (default, Jan  8 2021, 12:11:08) 
[Clang 12.0.0 (clang-1200.0.32.28)]
[INFO   ] [Python      ] Interpreter at "/opt/homebrew/opt/python@3.9/bin/python3.9"
[INFO   ] [Logger      ] Purge log fired. Processing...
[INFO   ] [Logger      ] Purge finished!
[INFO   ] [Factory     ] 187 symbols loaded
[DEBUG  ] [Cache       ] register <kv.resourcefind> with limit=None, timeout=60
[DEBUG  ] [Cache       ] register <kv.lang> with limit=None, timeout=None
[DEBUG  ] [Cache       ] register <kv.image> with limit=None, timeout=60
[DEBUG  ] [Cache       ] register <kv.atlas> with limit=None, timeout=None
[INFO   ] [ImageLoaderFFPy] Using ffpyplayer 4.3.2
[INFO   ] [Image       ] Providers: img_tex, img_imageio, img_dds, img_sdl2, img_pil, img_ffpyplayer 
[DEBUG  ] [Cache       ] register <kv.texture> with limit=1000, timeout=60
[DEBUG  ] [Cache       ] register <kv.shader> with limit=1000, timeout=3600
[INFO   ] [Clipboard   ] Provider: sdl2(['clipboard_nspaste'] ignored)
[INFO   ] [Text        ] Provider: sdl2
[INFO   ] [Window      ] Provider: sdl2
[CRITICAL] [App         ] Unable to get a Window, abort.

However, I was able to have it execute successfully by adding the -f option for full screen, and basic operation appears to function correctly. However, any interaction with the Finder does not work, until cmd-Q is pressed to exit katrain.

[sanderland]

Could someone with a new mac try the package version? Go here https://github.com/sanderland/katrain/pull/332/checks?check_run_id=1753696564 And download the 'artifact' in the top right.

[vividfog]

The Artifact variant, 396 MB app, runs normally but analysis doesn't work. It's just waiting for something, there's not much python CPU/GPU activity in Activity Monitor. And it's an Intel binary. How do I show the logs for this version?

All the buttons show up and the window behaves and resizes very well. Not so much with the native/compiled version.

[vividfog]

The native version is missing menus, so I can't set the debug level with it. Config.json didn't have a mention of it. What and where should I add to config.json to enable more debug info?

Also, the window resizing is pretty broken. Here's a window that isn't full screen. The first native screen I sent was at max resolution and full screen. Only then I was able to see most of the UI, but that dark bar on top then appears. Since it's full screen, I can't resize the window to see how it reacts. But in window mode the bar isn't there, because the board is pushed all the way to the top window bar and it has no room to be there.

[sanderland]

https://github.com/sanderland/katrain/blob/c6af552dbdc8232b87f67eecfa1c5f319bd568d1/katrain/config.json#L18

Setting this to 1 for some output and 2 for a lot.

The Artifact variant, 396 MB app, runs normally but analysis doesn't work. It's just waiting for something, there's not much python CPU/GPU activity in Activity Monitor. And it's an Intel binary. How do I show the logs for this version?

I suppose this means the katago that comes with it doesn't start. This is not too suprising given that it's an x64 binary. Does your brew binary show up in general settings? Also do .app files have the option to start from a command line so you can check the stdout/stderr?

All the buttons show up and the window behaves and resizes very well. Not so much with the native/compiled version.

That's good to hear that it can work, although a bit annoying that the native version doesn't..

[vividfog]

First the native version, here's the log. I'm not sure what to make of it, but maybe there's something that helps with the missing icons. If there's more that I can dig into, let me know. This is debug level 2, errors only. katrain-debug-2021-01-23-2018.log

[vividfog]

As for the bundled version, if I try to run katago-osx from Terminal, I get this:

> ./katago-osx
dyld: Library not loaded: /usr/local/opt/libzip/lib/libzip.5.dylib
  Referenced from: KaTrain.app/Contents/Resources/venv/lib/python3.8/site-packages/katrain/KataGo/./katago-osx
  Reason: image not found

> cd /usr/local/opt
cd: no such file or directory: /usr/local/opt

I found config.json from within the bundle, but I'm not able to launch Katrain.app in such a way that it would show any of its logs. This does not show them:

open Katrain.app

[sanderland]

First the native version, here's the log. I'm not sure what to make of it, but maybe there's something that helps with the missing icons. If there's more that I can dig into, let me know. This is debug level 2, errors only. katrain-debug-2021-01-23-2018.log

So yeah it's using pygame as the backend, rather than sdl2. This means the sdl2 backend wasn't found, probably because you built the release without having installed the dependencies

https://kivy.org/doc/stable/installation/installation-osx.html

[sanderland]

in any case, if the .app starts, simply going to general settings and selecting the katago binary in your path (from brew) should work

[vividfog]

Thanks, the .app now works, I pointed it to a native katago engine.

I'd still like to try to build a native .app bundle if you are able to give me some pointers. Just out of interest, to see if it can be done.

In the meanwhile, I'll check what's wrong with my native installation, or rather with its native dependencies. I remember having trouble installing them a month ago.

[vividfog]

Ok, so about the native version. I removed pygame and made sure that everything was installed. Everything was, after this, and brew now had native versions of them all, unlike a month ago:

brew install pkg-config sdl2 sdl2_image sdl2_ttf sdl2_mixer gstreamer

As far as I understand the kivy docs, that should do it.

Trying to run katrain would complain about missing pygame. I removed katrain too, tried to reinstall it:

> pip3 install katrain
Collecting katrain
  Using cached KaTrain-1.7.2-py3-none-any.whl (58.6 MB)
Requirement already satisfied: wheel in /opt/homebrew/lib/python3.9/site-packages (from katrain) (0.36.2)
Requirement already satisfied: setuptools in /opt/homebrew/lib/python3.9/site-packages (from katrain) (51.1.1)
Requirement already satisfied: kivymd>=0.104.1 in /opt/homebrew/lib/python3.9/site-packages (from katrain) (0.104.1)
Requirement already satisfied: chardet in /opt/homebrew/lib/python3.9/site-packages (from katrain) (4.0.0)
Requirement already satisfied: kivy[full]>=2.0.0 in /opt/homebrew/lib/python3.9/site-packages (from katrain) (2.1.0.dev0)
Requirement already satisfied: urllib3 in /opt/homebrew/lib/python3.9/site-packages (from katrain) (1.26.2)
Requirement already satisfied: docutils in /opt/homebrew/lib/python3.9/site-packages (from kivy[full]>=2.0.0->katrain) (0.16)
Requirement already satisfied: Kivy-Garden>=0.1.4 in /opt/homebrew/lib/python3.9/site-packages (from kivy[full]>=2.0.0->katrain) (0.1.4)
Requirement already satisfied: pygments in /opt/homebrew/lib/python3.9/site-packages (from kivy[full]>=2.0.0->katrain) (2.7.3)
Requirement already satisfied: pillow in /opt/homebrew/lib/python3.9/site-packages (from kivy[full]>=2.0.0->katrain) (8.0.1)
Requirement already satisfied: ffpyplayer in /opt/homebrew/lib/python3.9/site-packages (from kivy[full]>=2.0.0->katrain) (4.3.2)
Requirement already satisfied: requests in /opt/homebrew/lib/python3.9/site-packages (from Kivy-Garden>=0.1.4->kivy[full]>=2.0.0->katrain) (2.25.1)
Collecting pygame
  Using cached pygame-2.0.1-cp39-cp39-macosx_11_0_arm64.whl
Requirement already satisfied: certifi>=2017.4.17 in /opt/homebrew/lib/python3.9/site-packages (from requests->Kivy-Garden>=0.1.4->kivy[full]>=2.0.0->katrain) (2020.12.5)
Requirement already satisfied: idna<3,>=2.5 in /opt/homebrew/lib/python3.9/site-packages (from requests->Kivy-Garden>=0.1.4->kivy[full]>=2.0.0->katrain) (2.10)
Installing collected packages: pygame, katrain
Successfully installed katrain-1.7.2 pygame-2.0.1

You see, this method automatically installs pygame again. The erroneous behavior is the same.

Is there something I can change about the configuration to make it use sdl2 rather than pygame? I understood that pygame isn't needed, if sdl2 works.

Note that I don't really know what sdl2 or pygame are and what they do in this context, so I'm relying on thin speculation and loose interpretation of your responses here. 😅

Log from running katrain after all of the above: katrain-debug-2021-01-23-2100.log

[sanderland]

kivy is a gui framework, which basically forwards the calls to a graphics backend, which is usually sdl2. on some macs however, sdl2 is not available without having users type more brew commands, so I have it install pygame (which is a deprecated alternative but often works). pip3 uninstall pygame ?

[vividfog]

This took a step back. If pygame isn't installed, all I get is this:

> katrain
[CRITICAL] [App         ] Unable to get a Window, abort.

Which is the same as @kentdorsey got. Debug level is at 2, but that's all it says...

I wish there was a straightforward way to test that the kivy framework itself is working OK. There is no make test. They improved the platform check during setup, but that's all that was done for M1 AFAIK. Maybe the upstream has more problems that that, but I don't know how to verify. There were a truckload of warnings during make, but it still went through.

[sanderland]

Well, at least it confirms why pygame is being used, sdl2 doesn't work. I suggest taking this to the kivy devs again

[vividfog]

I found some test app code to run. This is indeed an upstream issue.

> python3 test.py
[INFO   ] [Kivy        ] v2.1.0.dev0
[INFO   ] [Kivy        ] Installed at "/opt/homebrew/lib/python3.9/site-packages/kivy/__init__.py"
[INFO   ] [Python      ] v3.9.1 (default, Jan  8 2021, 12:11:08)
[Clang 12.0.0 (clang-1200.0.32.28)]
[INFO   ] [Python      ] Interpreter at "/opt/homebrew/opt/python@3.9/bin/python3.9"
[INFO   ] [Logger      ] Purge log fired. Processing...
[INFO   ] [Logger      ] Purge finished!
[INFO   ] [Factory     ] 187 symbols loaded
[INFO   ] [ImageLoaderFFPy] Using ffpyplayer 4.3.2
[INFO   ] [Image       ] Providers: img_tex, img_imageio, img_dds, img_sdl2, img_pil, img_ffpyplayer
[INFO   ] [Text        ] Provider: sdl2
[INFO   ] [Window      ] Provider: sdl2
**[CRITICAL] [App         ] Unable to get a Window, abort.**

If you need someone with an M1 Mac to build an all-native .app bundle, I'm still volunteering for that. But maybe it's bound to fail, due to the very issue above.

To sum it up for those arriving late: The x86 .app bundle runs OK under Rosetta 2 and the native katago from homebrew works, making the most intensive computation arm64 native. I guess that's the most important part, and a fair enough outcome for now.

[kentdorsey]

At least for me it can only be run full screen, although that will be the only thing you will be doing on your m1 mac while it runs (good for focus :-). I was only able to take a screen shot using the timed screenshot function. Before engaging with the kivy project, is there any other debugging to be done?

[sanderland]

If you need someone with an M1 Mac to build an all-native .app bundle, I'm still volunteering for that. But maybe it's bound to fail, due to the very issue above.

The app bundle for normal macs is barely a day old, so let's give it some time. A PR to this line detecting M1 and not using the provided exe could be nice though. https://github.com/sanderland/katrain/blob/c6af552dbdc8232b87f67eecfa1c5f319bd568d1/katrain/core/engine.py#L69

[vividfog]

@kentdorsey did you try to download the .app version? For me it works fine in full screen, buttons and all, using Rosetta 2, M1 Air. The only thing it needs is brew install katago (using homebrew/arm64). Then I pointed Katrain.app to /opt/homebrew/bin/katago via settings, via the GUI. All good for the core purpose of playing Go using an arm64 engine.

[kentdorsey]

https://github.com/sanderland/katrain/pull/332/checks?check_run_id=1753696564

This version runs fine for me without issue on a 2020 Mac Mini M1 on macOS Big Sur 11.1. Does the UI look correct?

[sanderland]

Looks good to me!

[sanderland]

@vividfog could you let me know if df27100 would work correctly for m1?

[vividfog]

@sanderland I'm not a developer, I've only got some code tweaking and terminal skills to go with. This is a link to a patch. If you want me to build it, you'd need to send the git spells for how to do that while including this patch :) ... feel free to DM if that's the way to do it.

[sanderland]

@vividfog no worries - this should have a packaged artifact for you in ~30m https://github.com/sanderland/katrain/pull/337/checks?check_run_id=1777914609

[vividfog]

The latest build seems to work fine. It has an arm64 version of katago. The app itself is still Intel.

[sanderland]

It has an arm64 version of katago.

it does not, but it ignores the built-in version on arm and looks for the brewed one. Thanks for testing though, I've added it to the release.

[vividfog]

Oh, that's good too. Now that I remember, I had my own katago configured in the settings from last weekend. If those settings were reused, is it possible that this new code wasn't even run?

I didn't find the configuration file for this .app build. What should I delete to try again, if it really finds the homebrew variant on its own?

[sanderland]

That's possible, the config is persisted in ~/.katrain/config.json Probably the best way to try is to set it back to 'built-in' in general settings, it just looks in your PATH then

[kentdorsey]

Latest version works fine for me too.

Clarification: Latest osxapp build. Have not installed new python package build.

[vividfog]

I set it to built-in by removing all characters from the path to custom katago in GUI settings. Confirm and quit.

Config.json after this:

"engine": {
        "katago": "",
        "altcommand": "",
        "model": "katrain/models/g170e-b15c192-s1672170752-d466197061.bin.gz",
        "config": "katrain/KataGo/analysis_config.cfg",
        "threads": 12,
        "max_visits": 500,
        "fast_visits": 25,
        "max_time": 8.0,
        "wide_root_noise": 0.0,
        "_enable_ownership": true
    },

With this, Analysis doesn't do anything and katago process isn't running. So, something's wrong. Where's a log I can share?

[sanderland]

Hmm, strange. I suppose you could go back to the none-packaged pygame version to get the log despite the glitches.

[vividfog]

You mean download the latest code from github and build/install? Will it contain the patch?

[sanderland]

pip install -U katrain should work too, I thought it all worked after someone else tested it and released 1.7.4

[vividfog]

Ok, I got a wheel for 1.7.4 from the update.

Successfully installed katrain-1.7.4 pygame-2.0.1

The bad news now is that /opt/homebrew/bin/katrain no longer runs, not even with pygame installed. Something about kivy/pygame/sdl interaction is broken on M1 and Kivy developers don't yet know how to get more logs for that, even after we added some raises to kivy init to show what's going on. It fails at some very low level. For the record, an SDL2 test app works, a small game. But here we go again:

[CRITICAL] [App ] Unable to get a Window, abort.

I've got the latest versions, those should be OK.

> pip3 list | grep Kivy
Kivy        2.1.0.dev0
Kivy-Garden 0.1.4

> pip3 list | grep pygame
pygame      2.0.1

This is bogged down by an upstream issue, so to check why homebrew isn't automatically selected, perhaps we need another logging mechanism for that? Is it a lot of work to add a --console option, which would dump the logs to Console.app? Just an idea. You might have better ideas. Or maybe the .app bundle can be opened from Terminal in such a way that it leaves a log trail in the window?

That said, I wonder why you wouldn't choose to bundle an arm64 build of katago binary into the .app itself? I have it installed via homebrew, but how many other M1 owners do, out of a 100? 🤔

[sanderland]

Hmm, I thought this issue started with you being able to start the app, too bad it's failing now. I am not sure how to get output from an .app. As for a binary, I'm quite happy going from 'mac users should use pip and brew and hope for the best' to 'most mac users can click this thing, those with a new architecture also have to type in this single command'. The current binary is created via github actions, which doesn't have an m1 vm yet.

[vividfog]

True, it did initially work. But after I double-checked the dependencies and (I think) fixed SDL2 in the process, then Kivy broke in this new and mysterious way, just like for Kent. But it was not playable with Pygame anyway due to the glitches, so it's not a solution to recommend.

Perhaps it's best to wait until github actions support M1 builds? And until then use Rosetta 2 for the .app, since Kivy binary code seems to work fine with it.

And in the meanwhile something like this to get native performance for katago:

1. Download the .app bundle
2. Install katago with homebrew (x86 or arm64, user choice)
3. Put the path to katago into settings

... unless you can think of a reason, why the binary wasn't found automatically. The current patch assumes that the binary is in $PATH, does it? Mine is. But I'm pretty sure that an .app doesn't read my .zshrc.

which katago
/opt/homebrew/bin/katago

Note that M1 Macs may have Homebrew binaries in two locations, even at the same time. Some users run the x86 version of Homebrew under Rosetta, in which case it's /usr/local/bin. Native Homebrew uses /opt/homebrew/bin. Some few users may have both.

In theory macOS should automatically translate katago-osx upon first launch. I wonder why that didn't happen.

[sanderland]

The previous (pre-patch) behaviour was to look in PATH if only a filename is given. But given that there is no clear error, just a red engine light, it seems to find it, start it, and then fail at some internal level. https://github.com/sanderland/katrain/blob/79470fd13e8b4788b96ec2ea84d19d563d088a11/katrain/core/engine.py#L83

Any help here? https://stackoverflow.com/questions/364564/how-to-get-the-output-of-an-os-x-application-on-the-console-or-to-a-file

[kentdorsey]

I have katrain running natively in a windowed environment; AFTER I uninstalled most of the dependencies in my Python environment, and reinstalled the Kivy master branch again, and all seems to work as expected. I have a few other changes that I made to my environment, which may have impacted the build env, so I will capture that tomorrow, to see if it may help @vividfog, unless @vividfog has already resolved their issues.

[INFO   ] [Logger      ] Record log in /Users/kentdorsey/.kivy/logs/kivy_21-01-27_81.txt
[INFO   ] [Kivy        ] v2.1.0.dev0
[INFO   ] [Kivy        ] Installed at "/opt/homebrew/lib/python3.9/site-packages/kivy/__init__.py"
[INFO   ] [Python      ] v3.9.1 (default, Jan  8 2021, 12:11:08) 
[Clang 12.0.0 (clang-1200.0.32.28)]
[INFO   ] [Python      ] Interpreter at "/opt/homebrew/opt/python@3.9/bin/python3.9"
[INFO   ] [Logger      ] Purge log fired. Processing...
[INFO   ] [Logger      ] Purge finished!
[INFO   ] [Factory     ] 187 symbols loaded
[DEBUG  ] [Cache       ] register <kv.resourcefind> with limit=None, timeout=60
[DEBUG  ] [Cache       ] register <kv.lang> with limit=None, timeout=None
[DEBUG  ] [Cache       ] register <kv.image> with limit=None, timeout=60
[DEBUG  ] [Cache       ] register <kv.atlas> with limit=None, timeout=None
[INFO   ] [ImageLoaderFFPy] Using ffpyplayer 4.3.2
[INFO   ] [Image       ] Providers: img_tex, img_imageio, img_dds, img_sdl2, img_pil, img_ffpyplayer 
[DEBUG  ] [Cache       ] register <kv.texture> with limit=1000, timeout=60
[DEBUG  ] [Cache       ] register <kv.shader> with limit=1000, timeout=3600
[INFO   ] [Clipboard   ] Provider: sdl2(['clipboard_nspaste'] ignored)
[INFO   ] [Text        ] Provider: sdl2
[INFO   ] [Window      ] Provider: sdl2
[INFO   ] [GL          ] Using the "OpenGL ES 2" graphics system
[INFO   ] [GL          ] Backend used <sdl2>
[INFO   ] [GL          ] OpenGL version <b'2.1 Metal - 71.0.7'>
[INFO   ] [GL          ] OpenGL vendor <b'Apple'>
[INFO   ] [GL          ] OpenGL renderer <b'Apple M1'>
[INFO   ] [GL          ] OpenGL parsed version: 2, 1
[INFO   ] [GL          ] Shading version <b'1.20'>
[INFO   ] [GL          ] Texture max size <16384>
[INFO   ] [GL          ] Texture max units <16>
[DEBUG  ] [Shader      ] Fragment compiled successfully
[DEBUG  ] [Shader      ] Vertex compiled successfully
[DEBUG  ] [ImageImageIO] Load </opt/homebrew/lib/python3.9/site-packages/kivy/data/glsl/default.png>
[INFO   ] [Window      ] auto add sdl2 input provider
[INFO   ] [Window      ] virtual keyboard not allowed, single mode, not docked
[INFO   ] [KivyMD      ] v0.104.1
[DEBUG  ] [Atlas       ] Load </opt/homebrew/lib/python3.9/site-packages/kivymd/images/rec_shadow.atlas>
[DEBUG  ] [Atlas       ] Need to load 2 images
[DEBUG  ] [Atlas       ] Load </opt/homebrew/lib/python3.9/site-packages/kivymd/images/rec_shadow-1.png>
[DEBUG  ] [ImageImageIO] Load </opt/homebrew/lib/python3.9/site-packages/kivymd/images/rec_shadow-1.png>
[INFO   ] [GL          ] NPOT texture support is available
[DEBUG  ] [Atlas       ] Load </opt/homebrew/lib/python3.9/site-packages/kivymd/images/rec_shadow-0.png>
[DEBUG  ] [ImageImageIO] Load </opt/homebrew/lib/python3.9/site-packages/kivymd/images/rec_shadow-0.png>
[DEBUG  ] [Atlas       ] Load </opt/homebrew/lib/python3.9/site-packages/kivymd/images/rec_st_shadow.atlas>
[DEBUG  ] [Atlas       ] Need to load 3 images
[DEBUG  ] [Atlas       ] Load </opt/homebrew/lib/python3.9/site-packages/kivymd/images/rec_st_shadow-0.png>
[DEBUG  ] [ImageImageIO] Load </opt/homebrew/lib/python3.9/site-packages/kivymd/images/rec_st_shadow-0.png>
[DEBUG  ] [Atlas       ] Load </opt/homebrew/lib/python3.9/site-packages/kivymd/images/rec_st_shadow-1.png>
[DEBUG  ] [ImageImageIO] Load </opt/homebrew/lib/python3.9/site-packages/kivymd/images/rec_st_shadow-1.png>
[DEBUG  ] [Atlas       ] Load </opt/homebrew/lib/python3.9/site-packages/kivymd/images/rec_st_shadow-2.png>
[DEBUG  ] [ImageImageIO] Load </opt/homebrew/lib/python3.9/site-packages/kivymd/images/rec_st_shadow-2.png>
[DEBUG  ] [Atlas       ] Load </opt/homebrew/lib/python3.9/site-packages/kivymd/images/quad_shadow.atlas>
[DEBUG  ] [Atlas       ] Need to load 3 images
[DEBUG  ] [Atlas       ] Load </opt/homebrew/lib/python3.9/site-packages/kivymd/images/quad_shadow-1.png>
[DEBUG  ] [ImageImageIO] Load </opt/homebrew/lib/python3.9/site-packages/kivymd/images/quad_shadow-1.png>
[DEBUG  ] [Atlas       ] Load </opt/homebrew/lib/python3.9/site-packages/kivymd/images/quad_shadow-0.png>
[DEBUG  ] [ImageImageIO] Load </opt/homebrew/lib/python3.9/site-packages/kivymd/images/quad_shadow-0.png>
[DEBUG  ] [Atlas       ] Load </opt/homebrew/lib/python3.9/site-packages/kivymd/images/quad_shadow-2.png>
[DEBUG  ] [ImageImageIO] Load </opt/homebrew/lib/python3.9/site-packages/kivymd/images/quad_shadow-2.png>
[DEBUG  ] [Atlas       ] Load </opt/homebrew/lib/python3.9/site-packages/kivymd/images/round_shadow.atlas>
[DEBUG  ] [Atlas       ] Need to load 3 images
[DEBUG  ] [Atlas       ] Load </opt/homebrew/lib/python3.9/site-packages/kivymd/images/round_shadow-1.png>
[DEBUG  ] [ImageImageIO] Load </opt/homebrew/lib/python3.9/site-packages/kivymd/images/round_shadow-1.png>
[DEBUG  ] [Atlas       ] Load </opt/homebrew/lib/python3.9/site-packages/kivymd/images/round_shadow-0.png>
[DEBUG  ] [ImageImageIO] Load </opt/homebrew/lib/python3.9/site-packages/kivymd/images/round_shadow-0.png>
[DEBUG  ] [Atlas       ] Load </opt/homebrew/lib/python3.9/site-packages/kivymd/images/round_shadow-2.png>
[DEBUG  ] [ImageImageIO] Load </opt/homebrew/lib/python3.9/site-packages/kivymd/images/round_shadow-2.png>
[DEBUG  ] [Cache       ] register <textinput.label> with limit=None, timeout=60.0
[DEBUG  ] [Cache       ] register <textinput.width> with limit=None, timeout=60.0

Also:

Package     Version
----------- ----------
certifi     2020.12.5
chardet     4.0.0
docutils    0.16
ffpyplayer  4.3.2
gpg         1.15.1
idna        2.10
KaTrain     1.7.4
Kivy        2.1.0.dev0
Kivy-Garden 0.1.4
kivymd      0.104.1
Pillow      8.1.0
pip         21.0
pygame      2.0.1
Pygments    2.7.4
requests    2.25.1
setuptools  52.0.0
urllib3     1.26.2
wheel       0.36.2

[kentdorsey]

I also will see if I can break the install tomorrow by reverting some changes, etc. to determine if anything I did locally addressed the issue.

[vividfog]

@kentdorsey Interesting. Maybe I too should wipe homebrew and python/modules entirely and start over, in case there are leftover issues from my very early adoption of arm64 homebrew. Thanks for posting a confirmation that it worked for you. Curious to hear if you find out more.

[sanderland]

I also will see if I can break the install tomorrow by reverting some changes, etc. to determine if anything I did locally addressed the issue.

Could you set debug level to >= 1, katago choice in settings to built-in / empty and look for the logs around "Starting KataGo with..." ?

[vividfog]

Finally some good news. I ran the Homebrew uninstall script, exterminated the whole /opt/homebrew, checked .zshrc for any remains, finally reinstalled the packages I had previously listed in brews.list.

cat brews.list | xargs brew install

Added kivy:

python3 -m pip install "kivy[base] @ https://github.com/kivy/kivy/archive/master.zip"

Installed katrain:

/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install.sh)"
brew install python3
brew install katago
pip3 install katrain

> katrain

It works. The whole UI scales well. The brew version of katago is found. The GUI shows that I had not manually added a path to it.

I'll update the Kivy thread about this info. This wasn't quite a "have you tried unplugging/restarting" type of a fix, but not far from that either.

Unfortunately the .app says "Engine died unexpectedly...". I'd like to debug that part further, but I can't find the proper binary to run from the .app bundle. There's a binary called yourapp but that doesn't run; python errors. I can see that the whole environment is bundled there, and since this is a python app, it doesn't have a traditional app binary in the spirit of /MyProgram.app/Contents/MacOS/MyProgram to run.

[vividfog]

@sanderland ok, the correct "binary" was ... script 🙈

Now I got some logs from the .app version. It looks like it's trying to open the bundled x86 katago and an x86 version of homebrew's libzip. Which I obviously don't have. Does this help?

[INFO   ] [Base        ] Start application main loop
dyld: Library not loaded: /usr/local/opt/libzip/lib/libzip.5.dylib
Referenced from: /Applications/KaTrain.app/Contents/Resources/venv/lib/python3.8/site-packages/katrain/KataGo/katago-osx
Reason: image not found
ERROR: Engine died unexpectedly without sending output, possibly due to out of memory: status -6
 Exception in thread Thread-1:
ERROR: Engine died unexpectedly without sending output, possibly due to out of memory: status -6
 Traceback (most recent call last):
Message Loop Received update_state: () for Game 2021-01-28 12 24 18
   File "/Applications/KaTrain.app/Contents/Frameworks/Python.framework/Versions/3.8/lib/python3.8/threading.py", line 932, in _bootstrap_inner
Exception in reading stdout Engine died unexpectedly without sending output, possibly due to out of memory: status -6
Message Loop Received update_state: () for Game 2021-01-28 12 24 18
Message Loop Received update_state: () for Game 2021-01-28 12 24 18
     self.run()
   File "/Applications/KaTrain.app/Contents/Frameworks/Python.framework/Versions/3.8/lib/python3.8/threading.py", line 870, in run
     self._target(*self._args, **self._kwargs)
   File "/Applications/KaTrain.app/Contents/Resources/venv/lib/python3.8/site-packages/katrain/core/engine.py", line 204, in _analysis_read_thread
     self.check_alive(exception_if_dead=True)
   File "/Applications/KaTrain.app/Contents/Resources/venv/lib/python3.8/site-packages/katrain/core/engine.py", line 157, in check_alive
     raise EngineDiedException(died_msg)
 katrain.core.engine.EngineDiedException: Engine died unexpectedly without sending output, possibly due to out of memory: status -6
[DEBUG  ] [ImageImageIO] Load </Applications/KaTrain.app/Contents/Resources/venv/lib/python3.8/site-packages/katrain/img/board.png>

[sanderland]

Are your logs back on info? seems a bit sparse.

[vividfog]

@sanderland That was a snippet, here's the whole log at debug level 2. Hope this helps.

> cd ~/.katrain
> cat config.json | grep debug
        "debug_level": 2,

katrain.log

@kentdorsey where have you set your $PATH? Mine is in .zshrc and it works fine in iTerm.app, but I do not think this makes it system-wide in such a way that an .app launched from Finder would execute source .zshrc. But it sounds like your $PATH is set such that even .apps get the benefit of it. Did you manage to run KaTrain.app from Finder in such a way that $PATH was applied too? If yes, that implies you've managed to set $PATH globally and not just inside the Terminal box. And that's not necessarily the default macOS behavior, in which case a random bypasser wouldn't get the same benefit by just downloading the latest KaTrain.app. That was a long deduction, but I hope you see what I'm after. Any color you can add to this?

All that said, if I run the script inside the .app from iTerm, which I did, it should have the $PATH info to go with it. And still KaTrain.app wants to open the -osx version according to logs. Puzzling.

[sanderland]

@vividfog if you run python does "arm" in platform.machine() return true?

oh wait, the real question is what this is within this emulation layer :exploding_head:

[vividfog]

That's a good point! The theory goes that an app that's being run under Rosetta 2 (= all Intel .apps) will be given the strongest possible impression that "yep, definitely an Intel Mac".

And one would hope that this happens even if I launched from Terminal, which is otherwise very much arm64 native.

These together would explain, why the .app and the Python env inside it thinks it's running on Intel, even when run via terminal with

/Applications/KaTrain.app/Contents/Resources/script &> katrain.log

[vividfog]

Since the macOS default Python is both x86 and arm64, I can run it under Rosetta. Here's the result:

> arch -x86_64 python

WARNING: Python 2.7 is not recommended.
This version is included in macOS for compatibility with legacy software.
Future versions of macOS will not include Python 2.7.
Instead, it is recommended that you transition to using 'python3' from within Terminal.

Python 2.7.16 (default, Dec 21 2020, 23:00:36)
[GCC Apple LLVM 12.0.0 (clang-1200.0.30.4) [+internal-os, ptrauth-isa=sign+stri on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import platform
>>> platform.machine()
'x86_64'

So yes, if running via Rosetta 2, Python is convinced that this is not an ARM Mac.