search

giongto35 / cloud-game

Web-based Cloud Gaming service for Retro Game
https://www.youtube.com/watch?v=GUBrJGAxZZg
Apache License 2.0
2.32k stars 348 forks source link

Test Google Colab #322

Closed ImanCol closed 3 years ago

ImanCol commented 3 years ago

I have managed to get it to work in Google Colab, as far as Cloud-Game is running, but since it stays running, I can't run ngrok to make localhost public.

image

image

Can you help me finish the cell?

https://gist.github.com/ImanCol/2928cc0870efcca8cd0041cd1fa4975f

Is there another way to access localhost?

Google Colab: Distributor ID: Ubuntu Description: Ubuntu 18.04.5 LTS Release: 18.04 Codename: bionic

giongto35 commented 3 years ago

Wow, really nice to know that we can use google collab. It can support GPU as well. This is nice approach that I can take reference from. I remember last time I tried GG Collab I don't need ngrok to access its address :/

Regarding to your script, I think your syntax for ngrok is wrong, so it pointed to port 80 but it should be port 8000 "Tracking URL: NgrokTunnel: "http://13699365c7b6.ngrok.io" -> "http://localhost:80"

I change to "public_url = ngrok.connect(8000, proto="http", options={"bind_tls": True}), so it can map the port correctly. However, the server section I cannot run it successfully, so cannot verify the whole flow work.

sergystepanov commented 3 years ago

Um... apart from serving static HTML/JS with the HTTP/S, it requires opening a bunch of random UDP ports for audio/video/data streaming directly to the users. The easiest way is to open all the ports on your server...

ImanCol commented 3 years ago

Wow, really nice to know that we can use google collab. It can support GPU as well. This is nice approach that I can take reference from. I remember last time I tried GG Collab I don't need ngrok to access its address :/

Regarding to your script, I think your syntax for ngrok is wrong, so it pointed to port 80 but it should be port 8000 "Tracking URL: NgrokTunnel: "http://13699365c7b6.ngrok.io" -> "http://localhost:80"

I change to "public_url = ngrok.connect(8000, proto="http", options={"bind_tls": True}), so it can map the port correctly. However, the server section I cannot run it successfully, so cannot verify the whole flow work.

Can I change the Cloud-Game configuration to use port 80?

ImanCol commented 3 years ago

Um... apart from serving static HTML/JS with the HTTP/S, it requires opening a bunch of random UDP ports for audio/video/data streaming directly to the users. The easiest way is to open all the ports on your server...

I was testing the binary available in "release" in Windows, but when using ngrok or cloudfare, to obtain a public link, only the interface was displayed, but it was not seen that it loaded the emulator or the game in shared links using the url created by ngrok or cloudfare.

ImanCol commented 3 years ago

https://gist.github.com/ImanCol/2928cc0870efcca8cd0041cd1fa4975f

The code is now ready. All that remains is for the emulator to be displayed, and it is something that I cannot fix if I am missing something.

She left them in her hands.

sergystepanov commented 3 years ago

I don't think you can easily run it on Colab (or run it at all). As I said, all server ports should be open, not just one 8000 with Ngrok tunnel. If I understand correctly, Google Colaboratory has a custom firewall, so you can try:

ImanCol commented 3 years ago

I don't think you can easily run it on Colab (or run it at all). As I said, all server ports should be open, not just one 8000 with Ngrok tunnel. If I understand correctly, Google Colaboratory has a custom firewall, so you can try:

  • to open UDP ports from the range 40000-65536, if it can open UDP ports
  • or to open several UDP ports specified as I wrote here #257
  • or (your best chance) try to add some TURN server into the app's config file (TURN server should essentially do for WebRTC what Ngrok does for Web server, it works as a proxy). Some guide for TURN, after creating it, just add something like this into your config:
# ...
webrtc:
  # a list of STUN/TURN servers for the client
  iceServers:
    - url: stun:stun.l.google.com:19302
    -
      url: turn:somethin-somethin-your-server-address
      username: your-username
      credential: your-password
# ...

Work image Cloud-Game-Open-Web.txt

I0623 01:37:16.661849    7009 handlers.go:151] Coordinator: A user is connecting...
I0623 01:37:16.661956    7009 browser.go:33] Browser 278f3ab0-62c6-401b-9914-bf1f048d9956] Generated worker ID
I0623 01:37:16.661997    7009 browser.go:29] Browser 278f3ab0-62c6-401b-9914-bf1f048d9956] Get Room  Zone  From URL /ws?room_id=&zone=
I0623 01:37:16.662018    7009 browser.go:33] Browser 278f3ab0-62c6-401b-9914-bf1f048d9956] Send sync[http://localhost:9000/echo]http://localhost:9000/echo
I0623 01:37:18.837344    7009 browser.go:33] Browser 278f3ab0-62c6-401b-9914-bf1f048d9956] Latency mapmap[0xc0000ab7a0:9999]
I0623 01:37:19.020334    7009 browser.go:29] Browser 278f3ab0-62c6-401b-9914-bf1f048d9956] Received init_webrtc request -> relay to worker: b6600ad0-af48-4af8-8070-a313431e12c9
I0623 01:37:19.020607    7010 internalhandlers.go:44] Received a request to createOffer from browser via coordinator
I0623 01:37:19.020769    7010 webrtc.go:119] === StartClient ===
I0623 01:37:19.021533    7010 webrtc.go:137] Add video track
I0623 01:37:19.022253    7010 webrtc.go:230] Created Offer
I0623 01:37:19.022427    7010 internalhandlers.go:72] Start peerconnection 278f3ab0-62c6-401b-9914-bf1f048d9956
I0623 01:37:19.023368    7010 webrtc.go:191] OnIceCandidate: candidate:2126570329 1 udp 2130706431 172.28.0.2 52286 typ host
I0623 01:37:19.023694    7009 browser.go:33] Browser 278f3ab0-62c6-401b-9914-bf1f048d9956] Received SDP from worker -> sending back to browser
I0623 01:37:19.024029    7009 worker.go:38] Worker b6600ad0-af48-4af8-8070-a313431e12c9] Received IceCandidate from worker -> relay to browser
I0623 01:37:19.028784    7010 webrtc.go:191] OnIceCandidate: candidate:1746682845 1 udp 1694498815 35.237.26.42 54405 typ srflx raddr 0.0.0.0 rport 54405
I0623 01:37:19.029125    7009 worker.go:38] Worker b6600ad0-af48-4af8-8070-a313431e12c9] Received IceCandidate from worker -> relay to browser
I0623 01:37:19.029174    7009 worker.go:38] Worker b6600ad0-af48-4af8-8070-a313431e12c9] Received IceCandidate from worker -> relay to browser
I0623 01:37:19.279477    7009 browser.go:33] Browser 278f3ab0-62c6-401b-9914-bf1f048d9956] Received browser answered SDP -> relay to worker
I0623 01:37:19.279852    7010 internalhandlers.go:88] Received answer SDP from browser
I0623 01:37:19.280241    7010 webrtc.go:274] Set Remote Description
I0623 01:37:19.280413    7010 webrtc.go:175] ICE Connection State has changed: checking
I0623 01:37:19.343189    7009 browser.go:33] Browser 278f3ab0-62c6-401b-9914-bf1f048d9956] Received IceCandidate from browser -> relay to worker
I0623 01:37:19.343390    7010 internalhandlers.go:105] Received remote Ice Candidate from browser
I0623 01:37:19.343456    7010 webrtc.go:285] Decoded Ice: candidate:694132342 1 udp 2113937151 99331516-4642-41e3-8f04-28485dfa1766.local 58813 typ host generation 0 ufrag PHsG network-cost 999
I0623 01:37:19.343493    7010 webrtc.go:293] Add Ice Candidate: candidate:694132342 1 udp 2113937151 99331516-4642-41e3-8f04-28485dfa1766.local 58813 typ host generation 0 ufrag PHsG network-cost 999
I0623 01:37:19.516897    7009 browser.go:33] Browser 278f3ab0-62c6-401b-9914-bf1f048d9956] Received IceCandidate from browser -> relay to worker
I0623 01:37:19.517362    7010 internalhandlers.go:105] Received remote Ice Candidate from browser
I0623 01:37:19.517424    7010 webrtc.go:285] Decoded Ice: candidate:842163049 1 udp 1677729535 181.129.184.237 58813 typ srflx raddr 0.0.0.0 rport 0 generation 0 ufrag PHsG network-cost 999
I0623 01:37:19.517481    7010 webrtc.go:293] Add Ice Candidate: candidate:842163049 1 udp 1677729535 181.129.184.237 58813 typ srflx raddr 0.0.0.0 rport 0 generation 0 ufrag PHsG network-cost 999
I0623 01:37:20.084612    7010 webrtc.go:175] ICE Connection State has changed: connected
I0623 01:37:20.084648    7010 webrtc.go:179] ConnectionStateConnected
I0623 01:37:20.084655    7010 webrtc.go:325] Start streaming
I0623 01:37:20.708994    7010 webrtc.go:159] Data channel 'game-input'-'824638113628' open.

But when I try to start a game image Cloud-Game-Open-Gametxt.txt

I0623 01:38:42.392804    7009 browser.go:33] Browser 278f3ab0-62c6-401b-9914-bf1f048d9956] Received start request from a browser -> relay to worker
I0623 01:38:42.393084    7010 internalhandlers.go:125] Received a start request from coordinator
I0623 01:38:42.393122    7010 internalhandlers.go:260] Loading game: Super Mario All-Stars (USA)
I0623 01:38:42.393216    7010 internalhandlers.go:266] Got Room from local  <nil>  ID:  
I0623 01:38:42.393295    7010 room.go:126] New room:  4587ced56027909b___Super Mario All-Stars (USA) { Super Mario All-Stars (USA) sfc /content/cloud-game/assets/games/Super Mario All-Stars (USA).sfc}
I0623 01:38:42.393334    7010 room.go:241] Updated player Index to:  0
I0623 01:38:42.393355    7010 internalhandlers.go:281] Is PC in room false
I0623 01:38:42.393363    7010 handlers.go:143] [worker] closing peer connection
I0623 01:38:42.393411    7010 room.go:152] Check 4587ced56027909b___Super Mario All-Stars (USA) on online storage: false
I0623 01:38:42.393566    7009 browser.go:33] Browser 278f3ab0-62c6-401b-9914-bf1f048d9956] Received room response from browser: 4587ced56027909b___Super Mario All-Stars (USA)
I0623 01:38:42.393627    7009 internalhandlers.go:29] Coordinator: Received registerRoom room 4587ced56027909b___Super Mario All-Stars (USA) from worker b6600ad0-af48-4af8-8070-a313431e12c9
I0623 01:38:42.393653    7009 internalhandlers.go:31] Coordinator: Current room list is: map[4587ced56027909b___Super Mario All-Stars (USA):b6600ad0-af48-4af8-8070-a313431e12c9]
I0623 01:38:42.393421    7010 room.go:384] Check if game is on cloud storage
I0623 01:38:42.393896    7010 room.go:252] Start WebRTC session
I0623 01:38:44.394894    7010 room.go:154] Warn: Room 4587ced56027909b___Super Mario All-Stars (USA) is not in online storage, error Get "https://storage.googleapis.com/game-save/4587ced56027909b___Super%20Mario%20All-Stars%20%28USA%29": Get "http://169.254.169.254/computeMetadata/v1/instance/service-accounts/default/token?scopes=https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fdevstorage.full_control": dial tcp 169.254.169.254:80: i/o timeout
I0623 01:38:44.394926    7010 room.go:158] Room 4587ced56027909b___Super Mario All-Stars (USA) started. GameName: Super Mario All-Stars (USA), WithGame: false
I0623 01:38:44.395917    7010 nanoarch.go:239] [Log] [Libretro] Initialization...
I0623 01:38:44.403917    7010 nanoarch.go:462] Libretro API version: 1
I0623 01:38:44.404004    7010 nanoarch.go:493] ROM size: 2097152
I0623 01:38:44.404023    7010 nanoarch.go:504]   library_name: Snes9x
I0623 01:38:44.404031    7010 nanoarch.go:505]   library_version: 1.60 937c177
I0623 01:38:44.404043    7010 nanoarch.go:506]   valid_extensions: smc|sfc|swc|fig|bs|st
I0623 01:38:44.404067    7010 nanoarch.go:507]   need_fullpath: false
I0623 01:38:44.404078    7010 nanoarch.go:508]   block_extract: false
I0623 01:38:44.411096    7010 nanoarch.go:239] [Log] [Libretro] Loading the game...
Map_LoROMMap
I0623 01:38:44.435586    7010 nanoarch.go:239] [Log] "SUPER MARIO ALL_STARS" [checksum ok] LoROM, 16Mbits, ROM+RAM+BAT, NTSC, SRAM:64Kbits, ID:____, CRC32:925637C7
I0623 01:38:44.435969    7010 nanoarch.go:543] -----------------------------------
I0623 01:38:44.435985    7010 nanoarch.go:544] ---  Core audio and video info  ---
I0623 01:38:44.435996    7010 nanoarch.go:545] -----------------------------------
I0623 01:38:44.436014    7010 nanoarch.go:546]   Frame: 256x224 (604x478)
I0623 01:38:44.436030    7010 nanoarch.go:549]   AR:    1.3333333730697632
I0623 01:38:44.436042    7010 nanoarch.go:550]   FPS:   60.098811862348406
I0623 01:38:44.436060    7010 nanoarch.go:551]   Audio: 32040Hz
I0623 01:38:44.436070    7010 nanoarch.go:552] -----------------------------------
I0623 01:38:44.436241    7010 room.go:192] Viewport custom size is disabled, base size will be used instead 256x224
I0623 01:38:44.436506    7010 media.go:76] Video codec: h264
I0623 01:38:44.436527    7010 x264.go:31] x264: warning, installed version of libx264 152 is older than minimally supported v160, expect bugs
x264 [error]: I0623 01:38:44.436630    7010 media.go:53] OPUS: libopus 1.1.2, Bitrate: 192000 bps, Complexity: 10, DTX: false, FEC: false, Max bandwidth: *1105, Loss%: 0, Rate: 48000 Hz
invalid level_idc: 8
error create new encoder x264: cannot open the encoder

Use only CPU mode, because I couldn't get quota for GPU. When I access it (If I am lucky) I will see if the codec does not give a problem.

For my part, IT WORKS ON GOOGLE COLAB!.

sergystepanov commented 3 years ago

Cool. Then stun server is enough. @ImanCol, try to set vpx codec in the config, your error says that x264 system library is outdated and won't work, probably, due to an old Linux distro version on the server. Either download and install manually x264 lib version 159+ from somewhere for Colab Linux, i.e. from Debian bullseye or sid (if Google Colab uses Debian-based OS under the hood, check it, I think it uses Ubuntu), or change h264 codec to vpx in the app's config (the exact line is bellow). Can you do it with Colab syntax or edit and upload config file like you upload your game ROMs? I'm not familiar with Colab at all. https://github.com/giongto35/cloud-game/blob/c3c3351febdd8f7b85a584a6cf8b8c5f4dacc98c/configs/config.yaml#L153-L156

giongto35 commented 3 years ago

Congrat, Look good to me. When you can see the game menu screen, it indicates your WebRTC is successful. According to log, it seems like Codec issue as @sergystepanov said. In Collab, It seems like you can ssh to the server and modify the Config file.

ImanCol commented 3 years ago

Cool. Then stun server is enough. @ImanCol, try to set vpx codec in the config, your error says that x264 system library is outdated and won't work, probably, due to an old Linux distro version on the server. Either download and install manually x264 lib version 159+ from somewhere for Colab Linux, i.e. from Debian bullseye or sid (if Google Colab uses Debian-based OS under the hood, check it, I think it uses Ubuntu), or change h264 codec to vpx in the app's config (the exact line is bellow). Can you do it with Colab syntax or edit and upload config file like you upload your game ROMs? I'm not familiar with Colab at all. https://github.com/giongto35/cloud-game/blob/c3c3351febdd8f7b85a584a6cf8b8c5f4dacc98c/configs/config.yaml#L153-L156

The first thing, I already did. I get an error that the package is broken.

"Sudo apt install x264_0.160.3011 + gitcde9a93-2build1_amd64.deb"

The second thing, change the codec to "vpx" and it transmits. image

I0623 15:36:05.287628    6649 media.go:76] Video codec: vpx
I0623 15:36:05.288042    6649 media.go:53] OPUS: libopus 1.1.2, Bitrate: 192000 bps, Complexity: 10, DTX: false, FEC: false, Max bandwidth: *1105, Loss%: 0, Rate: 48000 Hz

I guess apart from the drawbacks presented here https://github.com/giongto35/cloud-game/pull/275 using "vpx", it is very very slow to use only CPU in Google Colab. Besides, forgive me for the redundancy, of a noise in the video. I have not had a chance to test the GPU, so it is a benefit for Google Colab Pro.

sergystepanov commented 3 years ago

So it works. Nice job. Speaking of CPU, I can, probably, optimize a/v pipe with x2, x3 performance increase even without GPU utilization. That would be the limit of the hybrid C/Go app, I think. I recommend you try to install Debian packages that have fewer dependencies, i.e. libx264-160. Yeah, GPU acceleration comes in handy when we'd run OpenGL cores like N64 (I didn't look up Vulkan emulation).

ImanCol commented 3 years ago

So it works. Nice job. Speaking of CPU, I can, probably, optimize a/v pipe with x2, x3 performance increase even without GPU utilization. That would be the limit of the hybrid C/Go app, I think. I recommend you try to install Debian packages that have fewer dependencies, i.e. libx264-160. Yeah, GPU acceleration comes in handy when we'd run OpenGL cores like N64 (I didn't look up Vulkan emulation).

This is the only compatible package and it is the one you download. https://packages.ubuntu.com/bionic/amd64/libx264-152/download

/content# sudo apt-get -f install libx264-dev
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following NEW packages will be installed:
  libx264-dev
0 upgraded, 1 newly installed, 0 to remove and 70 not upgraded.
Need to get 461 kB of archives.
After this operation, 1,766 kB of additional disk space will be used.
Get:1 http://archive.ubuntu.com/ubuntu bionic/universe amd64 libx264-dev amd64 2:0.152.2854+gite9a5903-2 [461 kB]
Fetched 461 kB in 1s (618 kB/s)
debconf: unable to initialize frontend: Dialog
debconf: (No usable dialog-like program is installed, so the dialog based frontend cannot be used. at /usr/share/perl5/Debconf/FrontEnd/Dialog.pm line 76, <> line 1.)
debconf: falling back to frontend: Readline
debconf: unable to initialize frontend: Readline
debconf: (This frontend requires a controlling tty.)
debconf: falling back to frontend: Teletype
dpkg-preconfigure: unable to re-open stdin: 
Selecting previously unselected package libx264-dev:amd64.
(Reading database ... 171711 files and directories currently installed.)
Preparing to unpack .../libx264-dev_2%3a0.152.2854+gite9a5903-2_amd64.deb ...
Unpacking libx264-dev:amd64 (2:0.152.2854+gite9a5903-2) ...
Setting up libx264-dev:amd64 (2:0.152.2854+gite9a5903-2) ...
Reading package lists... Done
Building dependency tree       
Reading state information... Done
Note, selecting 'libx264-160' instead of '/content/libx264-160_0.160.3011+gitcde9a93-2_amd64.deb'
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:

The following packages have unmet dependencies:
 libx264-160 : Depends: libc6 (>= 2.29) but 2.27-3ubuntu1.4 is to be installed
E: Unable to correct problems, you have held broken packages.

I think I'm getting a long way to get it to work, when the main problem is the distro version that Google Colab uses (Ubuntu4). Most of the packages that I try to install tell me that they are broken or that their dependencies are out of date, of which they also ask for other dependencies.

PD: I do not know if I am surprised that it worked with my zero knowledge that I have, or that they will congratulate me when you are the ones who have done everything to make this work.

Thanks for the help, I am going to try other environments, maybe some VPS, and I would like to know which one you would recommend or which one is running https://cloudretro.io/

sergystepanov commented 3 years ago

Ow, I didn't expect that it used Ubuntu 18.04 which has an incompatible libc6 version and I don't recommend trying to install version 2.29 because it'll break the system. And Ubuntu upgrade to 20.04 probably is not going to work on Colab (sudo do-release-upgrade). cloud-retro is being hosted on a bunch of DigitalOcean droplets. Personally, I prefer Amazon EC2 or Oracle Cloud with a basic forever free VPS.

giongto35 commented 3 years ago

@sergystepanov oh, :'( I tried to cut cost with DO but forgot Micro tier of EC2 is free. Thanks for reminding me about that. Let me migrate the infrastructure to AWS EC2

sergystepanov commented 3 years ago

@giongto35, it's free for 1yr, then you have to register another account (: As far as I know, only Oracle provides unconditional free slow machines, but I didn't research this field thoroughly.

ImanCol commented 3 years ago

I just wanted to say that since I have access to the GPU, I tried to use the same cells, but I get this error when CloudRetro is being built

go: downloading golang.org/x/oauth2 v0.0.0-20210402161424-2e8d93401602
go: downloading github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da

 github.com/go-gl/gl/v2.1/gl
/usr/lib/gcc/x86_64-linux-gnu/7/../../../x86_64-linux-gnu/libGL.so: undefined reference to `_glapi_tls_Current'
collect2: error: ld returned 1 exit status
Makefile:72: recipe for target 'dev.build-local' failed
make: *** [dev.build-local] Error 2

Google Colab: Distributor ID: Ubuntu Description: Ubuntu 18.04.5 LTS Release: 18.04 Codename: bionic

sergystepanov commented 3 years ago

Seems that Colab (NVIDIA) requires some additional GL lib linking, maybe something like this.

ImanCol commented 3 years ago

Ow, I didn't expect that it used Ubuntu 18.04 which has an incompatible libc6 version and I don't recommend trying to install version 2.29 because it'll break the system. And Ubuntu upgrade to 20.04 probably is not going to work on Colab (sudo do-release-upgrade). cloud-retro is being hosted on a bunch of DigitalOcean droplets. Personally, I prefer Amazon EC2 or Oracle Cloud with a basic forever free VPS.

Which AMI do you recommend I use to build this project on EC2?

sergystepanov commented 3 years ago

If you want to run it natively on some OS in the cloud, then I recommend you some latest Debian-based image, either Ubuntu 21 or Debian (11) Bullseye. If you want to run it with Docker, then it will run pretty much on any mainstream not-so-old OS, so choose whichever you like. Personally, I prefer Debian 11 in both cases. When I have time, I'll try to simplify and optimize the installation process.