How to set up on render?

[kylebakerio]

Instructions are out of date for heroku, which no longer offers a free tier. Would be nice to see advice for getting this up on render, or an equivalent small free basic service.

I've been trying, and it's close, but I'm getting this when I try to ping the server:

Unfortunately, don't see a way to ssh into the box in render for free accounts right now. :/

Feb 2 07:51:07 PM  ==> Starting service with 'gunicorn katago_server_eigen:app -w 1'
Feb 2 07:51:10 PM  [2023-02-03 01:51:10 +0000] [52] [INFO] Starting gunicorn 20.0.4
Feb 2 07:51:10 PM  [2023-02-03 01:51:10 +0000] [52] [INFO] Listening at: http://0.0.0.0:10000 (52)
Feb 2 07:51:10 PM  [2023-02-03 01:51:10 +0000] [52] [INFO] Using worker: sync
Feb 2 07:51:10 PM  [2023-02-03 01:51:10 +0000] [58] [INFO] Booting worker with pid: 58
Feb 2 07:51:10 PM  [2023-02-03 01:51:10,458] ERROR in app: Exception on /select-move/katago_gtp_bot [POST]
Feb 2 07:51:10 PM  Traceback (most recent call last):
Feb 2 07:51:10 PM    File "/opt/render/project/src/.venv/lib/python3.7/site-packages/flask/app.py", line 2447, in wsgi_app
Feb 2 07:51:10 PM      response = self.full_dispatch_request()
Feb 2 07:51:10 PM    File "/opt/render/project/src/.venv/lib/python3.7/site-packages/flask/app.py", line 1952, in full_dispatch_request
Feb 2 07:51:10 PM      rv = self.handle_user_exception(e)
Feb 2 07:51:10 PM    File "/opt/render/project/src/.venv/lib/python3.7/site-packages/flask/app.py", line 1821, in handle_user_exception
Feb 2 07:51:10 PM      reraise(exc_type, exc_value, tb)
Feb 2 07:51:10 PM    File "/opt/render/project/src/.venv/lib/python3.7/site-packages/flask/_compat.py", line 39, in reraise
Feb 2 07:51:10 PM      raise value
Feb 2 07:51:10 PM    File "/opt/render/project/src/.venv/lib/python3.7/site-packages/flask/app.py", line 1950, in full_dispatch_request
Feb 2 07:51:10 PM      rv = self.dispatch_request()
Feb 2 07:51:10 PM    File "/opt/render/project/src/.venv/lib/python3.7/site-packages/flask/app.py", line 1936, in dispatch_request
Feb 2 07:51:10 PM      return self.view_functions[rule.endpoint](**req.view_args)
Feb 2 07:51:10 PM    File "/opt/render/project/src/get_bot_app.py", line 36, in select_move
Feb 2 07:51:10 PM      bot_move = bot.select_move( content['moves'], config)
Feb 2 07:51:10 PM    File "/opt/render/project/src/katago_gtp_bot.py", line 169, in select_move
Feb 2 07:51:10 PM      self.set_komi( komi)
Feb 2 07:51:10 PM    File "/opt/render/project/src/katago_gtp_bot.py", line 273, in set_komi
Feb 2 07:51:10 PM      self._katagoCmd( 'komi %f' % komi)
Feb 2 07:51:10 PM    File "/opt/render/project/src/katago_gtp_bot.py", line 153, in _katagoCmd
Feb 2 07:51:10 PM      p.stdin.write( cmdstr.encode('utf8'))
Feb 2 07:51:10 PM  BrokenPipeError: [Errno 32] Broken pipe
Feb 2 07:51:10 PM  kata resp: ./katago_eigen: error while loading shared libraries: libzip.so.4: cannot open shared object file: No such file or directory
Feb 2 07:51:10 PM  
Feb 2 07:51:10 PM  Katago died. Resurrecting.
Feb 2 07:51:10 PM  >>> 2023-02-03 01:51:10 select move katago_gtp_bot {}
Feb 2 07:51:10 PM  >>>>>>>>> komi:%f 7.5
Feb 2 07:51:10 PM  sending komi 7.500000
Feb 2 07:51:10 PM  127.0.0.1 - - [03/Feb/2023:01:51:10 +0000] "POST /select-move/katago_gtp_bot HTTP/1.1" 500 290 "-" "curl/7.81.0"

Have also seen some errors indicating that libzip.so.4 is a problem/missing.

[hauensteina]

This means that libzip.so is missing. In Heroku, it is installed by default. I don't know how to do apt installs on render, sth like $ apt-get install libzip katago needs this to unzip network files, I think.

It is true that there is no more free layer on Heroku. But the Eco layer is 5 dollars a month, which is pretty close to free.

Hope this helps,

Andreas

On Thu, Feb 2, 2023 at 6:01 PM Kyle Baker @.***> wrote:

Instructions are out of date for heroku, which no longer offers a free tier. Would be nice to see advice for getting this up on render, or an equivalent small free basic service.

I've been trying, and it's close, but I'm getting this when I try to ping the server:

Unfortunately, don't see a way to ssh into the box in render for free accounts right now. :/

Feb 2 07:51:07 PM ==> Starting service with 'gunicorn katago_server_eigen:app -w 1' Feb 2 07:51:10 PM [2023-02-03 01:51:10 +0000] [52] [INFO] Starting gunicorn 20.0.4 Feb 2 07:51:10 PM [2023-02-03 01:51:10 +0000] [52] [INFO] Listening at: http://0.0.0.0:10000 (52) Feb 2 07:51:10 PM [2023-02-03 01:51:10 +0000] [52] [INFO] Using worker: sync Feb 2 07:51:10 PM [2023-02-03 01:51:10 +0000] [58] [INFO] Booting worker with pid: 58 Feb 2 07:51:10 PM [2023-02-03 01:51:10,458] ERROR in app: Exception on /select-move/katago_gtp_bot [POST] Feb 2 07:51:10 PM Traceback (most recent call last): Feb 2 07:51:10 PM File "/opt/render/project/src/.venv/lib/python3.7/site-packages/flask/app.py", line 2447, in wsgi_app Feb 2 07:51:10 PM response = self.full_dispatch_request() Feb 2 07:51:10 PM File "/opt/render/project/src/.venv/lib/python3.7/site-packages/flask/app.py", line 1952, in full_dispatch_request Feb 2 07:51:10 PM rv = self.handle_user_exception(e) Feb 2 07:51:10 PM File "/opt/render/project/src/.venv/lib/python3.7/site-packages/flask/app.py", line 1821, in handle_user_exception Feb 2 07:51:10 PM reraise(exc_type, exc_value, tb) Feb 2 07:51:10 PM File "/opt/render/project/src/.venv/lib/python3.7/site-packages/flask/_compat.py", line 39, in reraise Feb 2 07:51:10 PM raise value Feb 2 07:51:10 PM File "/opt/render/project/src/.venv/lib/python3.7/site-packages/flask/app.py", line 1950, in full_dispatch_request Feb 2 07:51:10 PM rv = self.dispatch_request() Feb 2 07:51:10 PM File "/opt/render/project/src/.venv/lib/python3.7/site-packages/flask/app.py", line 1936, in dispatch_request Feb 2 07:51:10 PM return self.view_functionsrule.endpoint Feb 2 07:51:10 PM File "/opt/render/project/src/get_bot_app.py", line 36, in select_move Feb 2 07:51:10 PM bot_move = bot.select_move( content['moves'], config) Feb 2 07:51:10 PM File "/opt/render/project/src/katago_gtp_bot.py", line 169, in select_move Feb 2 07:51:10 PM self.set_komi( komi) Feb 2 07:51:10 PM File "/opt/render/project/src/katago_gtp_bot.py", line 273, in set_komi Feb 2 07:51:10 PM self._katagoCmd( 'komi %f' % komi) Feb 2 07:51:10 PM File "/opt/render/project/src/katago_gtp_bot.py", line 153, in _katagoCmd Feb 2 07:51:10 PM p.stdin.write( cmdstr.encode('utf8')) Feb 2 07:51:10 PM BrokenPipeError: [Errno 32] Broken pipe Feb 2 07:51:10 PM kata resp: ./katago_eigen: error while loading shared libraries: libzip.so.4: cannot open shared object file: No such file or directory Feb 2 07:51:10 PM Feb 2 07:51:10 PM Katago died. Resurrecting. Feb 2 07:51:10 PM >>> 2023-02-03 01:51:10 select move katago_gtp_bot {} Feb 2 07:51:10 PM >>>>>>>>> komi:%f 7.5 Feb 2 07:51:10 PM sending komi 7.500000 Feb 2 07:51:10 PM 127.0.0.1 - - [03/Feb/2023:01:51:10 +0000] "POST /select-move/katago_gtp_bot HTTP/1.1" 500 290 "-" "curl/7.81.0"

Have also seen some errors indicating that libzip.so.4 is a problem/missing.

— Reply to this email directly, view it on GitHub https://github.com/hauensteina/katago-server/issues/6, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAH54SBKXBSAZNKFDOHZLHLWVRRIDANCNFSM6AAAAAAUPXFCFM . You are receiving this because you are subscribed to this thread.Message ID: @.***>

[kylebakerio]

thanks. :)

I had actually used your running free heroku server from time to time for an app of mine I rarely spun up, but that is down now of course.

it seems there's no way to apt-get install on render; their current design expects you to use a docker image if you need that kind of feature.

I'm not very familiar with python, been many years since I seriously used it. Also never used docker. May need to go that route though if I want a basic free katago server running... I just try not to have little expenses leaching out if I can avoid it. If the project takes off I'll spend a little to get it running.

I guess the other option is a free google cloud f1 micro, perhaps. More hands on, but should be able to run whatever command I want there.

[kylebakerio]

btw: looks like you later added 9x9 and 13x13 support finally, yes? Will be very excited to try that if so. I built a VR go interface that used this server, planning to rewrite it, just getting the old version up first.

[hauensteina]

Feel free to use my katago server. The API is open and the 10 block network or smaller board sizes don’t use any resources.

On Thursday, February 2, 2023, Kyle Baker @.***> wrote:

btw: looks like you later added 9x9 and 13x13 support finally, yes? Will be very excited to try that if so. I built a VR go interface that used this server, planning to rewrite it, just getting the old version up first.

— Reply to this email directly, view it on GitHub https://github.com/hauensteina/katago-server/issues/6#issuecomment-1414808202, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAH54SDCLOXUQK3WM2XPWN3WVR4IJANCNFSM6AAAAAAUPXFCFM . You are receiving this because you commented.Message ID: @.***>

[kylebakerio]

Thanks! That'll work for now. What's the URL?

[hauensteina]

Try this:

13x13:

$ curl -d '{"board_size":13, "moves":["E5", "E7"]}' -H "Content-Type: application/json" -X POST http://www.ahaux.com:2823/select-move/katago_gtp_bot

9x9:

$ curl -d '{"board_size":9, "moves":["E5", "E7"]}' -H "Content-Type: application/json" -X POST http://www.ahaux.com:2822/select-move/katago_gtp_bot

19x19 (10 block):

$ curl -d '{"board_size":19, "moves":["E5", "E7"]}' -H "Content-Type: application/json" -X POST http://www.ahaux.com:2821/select-move/katago_gtp_bot

LMK if it works.

-Andreas

On Fri, Feb 3, 2023 at 10:43 PM Kyle Baker @.***> wrote:

Thanks! That'll work for now. What's the URL?

— Reply to this email directly, view it on GitHub https://github.com/hauensteina/katago-server/issues/6#issuecomment-1416679122, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAH54SAR2I6MIKQFJSX6PJTWVX27VANCNFSM6AAAAAAUPXFCFM . You are receiving this because you commented.Message ID: @.***>

[kylebakerio]

hey, I was pulling my hair out because I didn't notice that you are using a different port for each size! once I noticed that it all clicked. It seems to all be working smoothly now. I do notice a slight difference in play style; is this one stronger than the prior one that had been thrown up on heroku?

One thought: is it possible to send a param to modify the komi?

And I haven't tested it, but I presume I can get katago to play a handicap game by just sending a board with handicap stones placed, yeah?

[hauensteina]

You can set komi like this:

$ curl -d '{"board_size":19, "moves":["E5", "E7"], "config":{"komi": 6.5 } }' -H "Content-Type: application/json" -X POST http://www.ahaux.com:2821/select-move/katago_gtp_bot

Handicap is given by inserting white passes at the beginning. For two stones:

$ curl -d '{"board_size":19, "moves":["Q16", "pass", "D4"], "config":{"komi": 0.5 } }' -H "Content-Type: application/json" -X POST http://www.ahaux.com:2821/select-move/katago_gtp_bot

Rules are Chinese for odd komi (like 7.5) and automatically switch to Japanese for even komi.

The katago-server for Heroku from the katago-server repo only does 8 playouts. This one does 256 (which still isn't much) and runs on a box with an RTX2080. Nobody I know can beat it. Try taking 4 handicaps. If you can win even one game, you are very strong.

Hope this helps,

-Andres

On Fri, Feb 17, 2023 at 2:58 PM Kyle Baker @.***> wrote:

hey, I was pulling my hair out because I didn't notice that you are using a different port for each size! once I noticed that it all clicked. It seems to all be working smoothly now. I do notice a slight difference in play style; is this one stronger than the prior one that had been thrown up on heroku?

One thought: is it possible to send a param to modify the komi?

And I haven't tested it, but I presume I can get katago to play a handicap game by just sending a board with handicap stones placed, yeah?

— Reply to this email directly, view it on GitHub https://github.com/hauensteina/katago-server/issues/6#issuecomment-1435372840, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAH54SDDQI7XM5CNVKQ3ZW3WX77CDANCNFSM6AAAAAAUPXFCFM . You are receiving this because you commented.Message ID: @.***>

[hauensteina]

URLs have changed:

$ curl -d '{"board_size":9, "moves":["E5", "G5"]}' -H "Content-Type: application/json" -X POST https://katagui.baduk.club/select-move-9/katago_gtp_bot $ curl -d '{"board_size":9, "moves":["E5", "G5"]}' -H "Content-Type: application/json" -X POST https://katagui.baduk.club/select-move-13/katago_gtp_bot $ curl -d '{"board_size":9, "moves":["E5", "G5"]}' -H "Content-Type: application/json" -X POST https://katagui.baduk.club/select-move-guest/katago_gtp_bot

I should have given you these in the first place. Sorry.

-Andreas

On Fri, Feb 17, 2023 at 2:58 PM Kyle Baker @.***> wrote:

hey, I was pulling my hair out because I didn't notice that you are using a different port for each size! once I noticed that it all clicked. It seems to all be working smoothly now. I do notice a slight difference in play style; is this one stronger than the prior one that had been thrown up on heroku?

One thought: is it possible to send a param to modify the komi?

And I haven't tested it, but I presume I can get katago to play a handicap game by just sending a board with handicap stones placed, yeah?

— Reply to this email directly, view it on GitHub https://github.com/hauensteina/katago-server/issues/6#issuecomment-1435372840, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAH54SDDQI7XM5CNVKQ3ZW3WX77CDANCNFSM6AAAAAAUPXFCFM . You are receiving this because you commented.Message ID: @.***>

[kylebakerio]

Thanks so much! interesting having the other data as well.

Are there any other config options worth knowing about, or any other details that might help? Maybe a way to tell it to be weaker or stronger? Didn't see any in readme. (Could start going through code, of course, but that should cover everything for me.)

[kylebakerio]

I presume the 'guest' url is for 19x19?

And... how can I access what was previously the /score/katago_gtp_bot endpoint for various sizes? I guessed maybe /score-13/..., but that did not work.

[kylebakerio]

Would of course be cool if there were just one url and passing in 'size' in the payload worked! I'm sure for some reason this way is easier for you to set up and is just meant to be internal, though, so not really complaining. Having access to a 2080 with 256 playouts like this is great.

I'm sure I'm not strong enough to beat it with 4 handicaps; I will be interested to see how many handicaps I do need to beat it though.

I use the /score endpoint to generate heatmaps, and it was great for that brief moment where the previous URLs were up and I could do that.

[hauensteina]

So you noticed that the board_size parameter gets ignored. Indeed. Board size is determined by the endpoint. What is embarrassing is that the board_size parameter is required even though it is ignored. Oh well. Actually I forget why this was more convenient back then.

To score:

curl -d '{"board_size":19, "moves":["E5", "G5"]}' -H "Content-Type: application/json" -X POST https://katagui.baduk.club/score/katago_gtp_bot

This only works for 19x19. Also, it always uses the strongest net (40 blocks, 1024 playouts). So it might differ in opinion vs what the 10 block thinks while playing. Scoring on small boards is currently unsupported.

I might eventually clean this up, but currently it doesn't look like it.

On Tue, Feb 21, 2023 at 6:00 PM Kyle Baker @.***> wrote:

Would of course be cool if there were just one url and passing in 'size' in the payload worked! I'm sure for some reason this way is easier for you to set up and is just meant to be internal, though, so not really complaining. Having access to a 2080 with 256 playouts like this is great.

I'm sure I'm not strong enough to beat it with 4 handicaps; I will be interested to see how many handicaps I do need to beat it though.

— Reply to this email directly, view it on GitHub https://github.com/hauensteina/katago-server/issues/6#issuecomment-1439330433, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAH54SHV4ZLDB5U7UM2VIUTWYVXLFANCNFSM6AAAAAAUPXFCFM . You are receiving this because you commented.Message ID: @.***>

[kylebakerio]

I see. For a few days there, scoring on the smaller boards was possible with the port-based url api you shared! Ah well, guess I'll just have to accept it as it is unless I want to dig in and contribute and/or put up my own machine. Interesting that the scoring API uses a different playout, I'm sure I would have wondered about that if I noticed it not playing moves that matched the score result.