sanderland / katrain

Improve your Baduk skills by training with KataGo!
Other
1.63k stars 227 forks source link

KaTrain doesn't work on MacBook Pro M1 MAX(maxed out) macOS #636

Closed Chess321 closed 2 months ago

Chess321 commented 1 year ago

brew install katrain brew install katago brew install python (pip)

When I look at KaTrain v1.13.0 ERROR: KataGo config file/Applications/KaTrain.app/Contents/Resources/venv/bin does not exist, please check general settings.

When I look at General and Engine Settings, then I see: Path to KataGo executable: Is empty Path to KataGo config file: Is empty Path to KataGo model file: Is filled Override Engine Command: Is empty

I tried: command+shift+. In Finder, press ⌘⇧G. This should open a dialog box. You can enter the folder path /opt/homebrew/Cellar to access the contents of the Cellar.

I tried with Terminal: /opt/homebrew/Cellar/katago/1.13.2/bin/katago ; exit;

Usage: /opt/homebrew/Cellar/katago/1.13.2/bin/katago SUBCOMMAND

---Common subcommands------------------

gtp : Runs GTP engine that can be plugged into any standard Go GUI for play/analysis. benchmark : Test speed with different numbers of search threads. genconfig : User-friendly interface to generate a config with rules and automatic performance tuning.

contribute : Connect to online distributed KataGo training and run perpetually contributing selfplay games.

match : Run self-play match games based on a config, more efficient than gtp due to batching. version : Print version and exit.

analysis : Runs an engine designed to analyze entire games in parallel. tuner : (OpenCL only) Run tuning to find and optimize parameters that work on your GPU.

---Selfplay training subcommands---------

selfplay : Play selfplay games and generate training data. gatekeeper : Poll directory for new nets and match them against the latest net so far.

---Testing/debugging subcommands------------- evalsgf : Utility/debug tool, analyze a single position of a game from an SGF file.

testgpuerror : Print the average error of the neural net between current config and fp32 config.

runtests : Test important board algorithms and datastructures runnnlayertests : Test a few subcomponents of the current neural net backend

runnnontinyboardtest : Run neural net on a tiny board and dump result to stdout runnnsymmetriestest : Run neural net on a hardcoded rectangle board and dump symmetries result runownershiptests : Run neural net search on some hardcoded positions and print avg ownership

runoutputtests : Run a bunch of things and dump details to stdout runsearchtests : Run a bunch of things using a neural net and dump details to stdout runsearchtestsv3 : Run a bunch more things using a neural net and dump details to stdout runsearchtestsv8 : Run a bunch more things using a neural net and dump details to stdout runsearchtestsv9 : Run a bunch more things using a neural net and dump details to stdout runselfplayinittests : Run some tests involving selfplay training init using a neural net and dump details to stdout runsekitrainwritetests : Run some tests involving seki train output

Saving session... ...copying shared history... ...saving history...truncating history files... ...completed.

I tried with Terminal: brew list --verbose katago /opt/homebrew/Cellar/katago/1.13.2/INSTALL_RECEIPT.json /opt/homebrew/Cellar/katago/1.13.2/LICENSE /opt/homebrew/Cellar/katago/1.13.2/bin/katago /opt/homebrew/Cellar/katago/1.13.2/.brew/katago.rb /opt/homebrew/Cellar/katago/1.13.2/README.md /opt/homebrew/Cellar/katago/1.13.2/share/katago/g170-b30c320x2-s4824661760-d1229536699.bin.gz /opt/homebrew/Cellar/katago/1.13.2/share/katago/g170-b40c256x2-s5095420928-d1229425124.bin.gz /opt/homebrew/Cellar/katago/1.13.2/share/katago/g170e-b20c256x2-s5303129600-d1228401921.bin.gz /opt/homebrew/Cellar/katago/1.13.2/share/katago/configs/misc/datamining1.cfg /opt/homebrew/Cellar/katago/1.13.2/share/katago/configs/misc/datamining4.cfg /opt/homebrew/Cellar/katago/1.13.2/share/katago/configs/match_example.cfg /opt/homebrew/Cellar/katago/1.13.2/share/katago/configs/gtp_example.cfg /opt/homebrew/Cellar/katago/1.13.2/share/katago/configs/training/selfplay8a.cfg /opt/homebrew/Cellar/katago/1.13.2/share/katago/configs/training/gatekeeper2bfaster.cfg /opt/homebrew/Cellar/katago/1.13.2/share/katago/configs/training/gatekeeper2a.cfg /opt/homebrew/Cellar/katago/1.13.2/share/katago/configs/training/selfplay8b.cfg /opt/homebrew/Cellar/katago/1.13.2/share/katago/configs/training/gatekeeper2b.cfg /opt/homebrew/Cellar/katago/1.13.2/share/katago/configs/training/gatekeeper2c.cfg /opt/homebrew/Cellar/katago/1.13.2/share/katago/configs/training/gatekeeper1.cfg /opt/homebrew/Cellar/katago/1.13.2/share/katago/configs/training/selfplay1.cfg /opt/homebrew/Cellar/katago/1.13.2/share/katago/configs/training/selfplay2.cfg /opt/homebrew/Cellar/katago/1.13.2/share/katago/configs/training/selfplay8b20.cfg /opt/homebrew/Cellar/katago/1.13.2/share/katago/configs/analysis_example.cfg /opt/homebrew/Cellar/katago/1.13.2/share/katago/configs/contribute_example.cfg /opt/homebrew/Cellar/katago/1.13.2/share/katago/configs/book/genbook7tt.cfg /opt/homebrew/Cellar/katago/1.13.2/share/katago/configs/book/genbook7jp.cfg /opt/homebrew/Cellar/katago/1.13.2/share/katago/configs/task_example.cfg

I tried with Terminal: katago gtp -config $(brew list --verbose katago | grep gtp.cfg) -model $(brew list --verbose katago | grep .gz | head -1) zsh: no matches found: gtp.cfg PARSE ERROR: Argument: /opt/homebrew/Cellar/katago/1.13.2/share/katago/g170-b30c320x2-s4824661760-d1229536699.bin.gz Couldn't find match for argument

Brief USAGE: gtp [-config ] ... [-model ] [...other flags...]

For complete USAGE and HELP type: gtp -help

I tried with KaTrain: Path to KataGo executable: /opt/homebrew/Cellar/katago/1.13.2/bin/katago Path to KataGo config file: /opt/homebrew/Cellar/katago/1.13.2/share/katago/configs/gtp_example.cfg Path to KataGo model file: katrain/models/g170e-b15c192-s1672170752-d466197061.bin.gz -Update Settings And I got: ERROR: Engine died unexpectedly without sending output: status -6

When I look at Finder user me gtp_logs: 20230702-121418-CF2F5369.log 2023-07-02 12:14:18+0200: Running with following config: allowResignation = true homeDataDir = /Users/me/.katrain lagBuffer = 1.0 logAllGTPCommunication = true logDir = gtp_logs logSearchInfo = true logToStderr = false maxTimePondering = 60.0 maxVisits = 500 numSearchThreads = 6 ponderingEnabled = false resignConsecTurns = 3 resignThreshold = -0.90 rules = tromp-taylor searchFactorAfterOnePass = 0.50 searchFactorAfterTwoPass = 0.25 searchFactorWhenWinning = 0.40 searchFactorWhenWinningThreshold = 0.95

2023-07-02 12:14:18+0200: Analysis Engine starting... 2023-07-02 12:14:18+0200: KataGo v1.13.2 2023-07-02 12:14:19+0200: nnRandSeed0 = 8197818535970151932 2023-07-02 12:14:19+0200: After dedups: nnModelFile0 = /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/katrain/models/g170e-b15c192-s1672170752-d466197061.bin.gz useFP16 auto useNHWC auto

https://github.com/lightvector/KataGo#macos https://formulae.brew.sh/formula/katago https://github.com/sanderland/katrain/blob/master/INSTALL.md#MacQuick

Can someone please send me a config file and executable via download for macOS M1? (M1 MAX only if possible)

Any ideas what’s going on and how to make it work?

Chess321 commented 1 year ago

Path to KataGo executable: /opt/homebrew/bin/katago Path to KataGo config file: katrain/KataGo/analysis_config.cfg Path to KataGo model file: katrain/models/g170e-b15c192-s1672170752-d466197061.bin.gz

Now it works.

But I still have some difficulties: I started a new game and analysis. After some moves on the board, KaTrain stops to show playable moves and colors on the board.

Win Rate, Estimated Score and Points Lost also stops to work and are empty.

Katrain shows only: Analyzing move… And MacBook confirms that it is doing an intensive task.

Could it be that KataGo is analyzing all played moves/boards at the same time and that’s to much? I want to run an endless/infinite analysis, which is only analyzing the main position on the board and not the moves that happened before. I played 27 moves. Under the board I see a yellow/orange ring with the number 30.

Please make it possible the change the colors when looking inside Teaching/Analysis Settings

We have two green colors, that’s one to much.

Please make it possible to show the best move in blue color.

Please make more sounds available when placing a stone on the board and when winning stones.

Please improve the color of the stones and board or let us choose different boards and stones. I mean for example Crazy Stone Pro (only HD) from 2013 running on my old iPad Pro 2 (2K resolution) looks much better, than KaTrain on my new MacBook Pro 16-inch M1 MAX, 10 cpu cores, 32 gpu cores, 64 gb ram (4K resolution and much better and brighter colors) https://apps.apple.com/app/crazystone-deeplearning-pro/id1160054151?l=en https://store.steampowered.com/app/479330/Crazy_Stone_Deep_Learning_The_First_Edition/

Please make it possible to show the coordinates on different or all sides.

I changed threads from 12 to 10 because the MacBook Pro M1 MAX has only 10 cpu cores.

{ "engine": { "katago": "/opt/homebrew/bin/katago", "altcommand": "", "model": "katrain/models/g170e-b15c192-s1672170752-d466197061.bin.gz", "config": "katrain/KataGo/analysis_config.cfg", "threads": 10, "max_visits": 9999999999, "fast_visits": 9999999999, "max_time": 28800.0, "wide_root_noise": 0.0, "_enable_ownership": true }, "contribute": { "katago": "", "config": "katrain/KataGo/contribute_config.cfg", "ownership": false, "maxgames": 6, "movespeed": 2, "username": "", "password": "", "savepath": "./dist_sgf/", "savesgf": false }, "general": { "sgf_load": "~/Downloads", "sgf_save": "./sgfout", "anim_pv_time": 0.5, "debug_level": 1, "lang": "en", "version": "1.13.0", "load_fast_analysis": false, "load_sgf_rewind": true }, "timer": { "byo_length": 30, "byo_periods": 5, "minimal_use": 0, "main_time": 0, "sound": true }, "game": { "size": "19", "komi": 6.5, "handicap": 0, "rules": "japanese", "clear_cache": false, "setup_move": 100, "setup_advantage": 20 }, "trainer": { "theme": "theme:normal", "num_undo_prompts": [ 1, 1, 1, 0.5, 0, 0 ], "eval_thresholds": [ 0.4, 0.3, 0.2, 0.1, 0.01, 0 ], "save_feedback": [ true, true, true, true, true, true ], "show_dots": [ true, true, true, true, true, true ], "extra_precision": true, "save_analysis": true, "save_marks": false, "low_visits": 25, "eval_on_show_last": 3, "top_moves_show": "top_move_score", "top_moves_show_secondary": "top_move_visits", "eval_show_ai": true, "lock_ai": false }, "ai": { "ai:default": {}, "ai:antimirror": {}, "ai:handicap": { "automatic": true, "pda": 0 }, "ai:jigo": { "target_score": 0.5 }, "ai:scoreloss": { "strength": 0.2 }, "ai:policy": { "opening_moves": 22.0 }, "ai:simple": { "max_points_lost": 1.75, "settled_weight": 1.0, "opponent_fac": 0.5, "min_visits": 3, "attach_penalty": 1, "tenuki_penalty": 0.5 }, "ai:p:weighted": { "weaken_fac": 1.25, "pick_override": 1.0, "lower_bound": 0.001 }, "ai:p:pick": { "pick_override": 0.95, "pick_n": 5, "pick_frac": 0.35 }, "ai:p:local": { "pick_override": 0.95, "stddev": 1.5, "pick_n": 15, "pick_frac": 0.0, "endgame": 0.5 }, "ai:p:tenuki": { "pick_override": 0.85, "stddev": 7.5, "pick_n": 5, "pick_frac": 0.4, "endgame": 0.45 }, "ai:p:influence": { "pick_override": 0.95, "pick_n": 5, "pick_frac": 0.3, "threshold": 3.5, "line_weight": 10, "endgame": 0.4 }, "ai:p:territory": { "pick_override": 0.95, "pick_n": 5, "pick_frac": 0.3, "threshold": 3.5, "line_weight": 2, "endgame": 0.4 }, "ai:p:rank": { "kyu_rank": 4.0 } }, "ui_state": { "restoresize": true, "size": [ 1171, 850 ], "play": { "analysis_controls": { "show_children": true, "eval": false, "hints": false, "policy": false, "ownership": false }, "panels": { "graph_panel": [ "open", { "score": true, "winrate": true } ], "stats_panel": [ "open", { "score": true, "winrate": true, "points": true } ], "notes_panel": [ "open", { "info": true, "info-details": true, "notes": true } ] } }, "analyze": { "analysis_controls": { "show_children": true, "eval": true, "hints": true, "policy": false, "ownership": false }, "panels": { "graph_panel": [ "open", { "score": true, "winrate": true } ], "stats_panel": [ "open", { "score": true, "winrate": true, "points": true } ], "notes_panel": [ "open", { "info": true, "info-details": true, "notes": true } ] } }, "top": 182, "left": 178 }, "dist_models": {} }

Chess321 commented 1 year ago

It looks like there is an "Analyzing move..." BUG.

Engine Settings: "max_visits": 9999999999, "fast_visits": 9999999999, "max_time": 28800.0,

When I start a new game, then analysis works fine for some moves. But then I get always: Analyzing move… and nothing happens.

When I start the analysis in one position, it can easily go above 1.5m visits after some minutes. But when I do some more moves, I get: Analyzing move… and nothing happens. The most RAM is empty so it can't be a RAM problem unless KataGo or KaTrain is limited to something like 5 GB RAM. The BUG happens in most cases between move 10 and 20. I don't know why it can go above 1.5m visits in one position without problems but after 20k in another game I get: Analyzing move… Could it have something to do with how fast the moves are played on the board? I decreased the number of cpu cores but nothing changed. I also decreased a lot: "max_visits": 9999999999, "fast_visits": 9999999999, "max_time": 28800.0, But the BUG is still there. The only difference is that Analyzing move… happens most of the time between move 50 to 80. When I start the analysis, it takes 1 second to fill 10 MB of RAM. Could it be that other settings cause this trouble?

github-actions[bot] commented 2 months ago

This issue is stale because it has been open 90 days with no activity. Remove stale label or comment or this will be closed in 10 days.

Chess321 commented 2 months ago

Up