Closed toby181 closed 1 year ago
Hello, most probably you are using kubernetes nodes not based on Ubuntu OS, in that case chrome requires --no-sandbox argument, you can add it via capabilities.
пт, 12 мая 2023 г., 16:10 toby181 @.***>:
Hi together,
We've deployed moon2 in version 2.5 to k8s but we're having issues with manually creating a session if we select chrome or edge via the UI, firefox and opera are working.
The deployment configuration and the logs can be found below. Chrome basically terminates with:
- x-server XIO: fatal IO error 2 (No such file or directory) on X server ":0"
- session failed: driver response code: 500: Internal Server Error
helmrelease values:
values: quota: moon: namespace: namespace1 ingress: enabled: true ... configs: default: sessionTimeout: 10m containers: browser: resources: limits: cpu: "1" memory: 1Gi requests: cpu: 500m memory: 500Mi ca-certs: repository: quay.io/aerokube/ca-certs resources: limits: cpu: 100m memory: 32Mi requests: cpu: 50m memory: 16Mi proxy: repository: quay.io/aerokube/proxy resources: limits: cpu: 100m memory: 32Mi requests: cpu: 50m memory: 16Mi defender: repository: quay.io/aerokube/defender resources: limits: cpu: 100m memory: 128Mi requests: cpu: 50m memory: 32Mi video-recorder: repository: quay.io/aerokube/video-recorder resources: limits: cpu: 1000m memory: 2Gi requests: cpu: 250m memory: 1000Mi vnc-server: repository: quay.io/aerokube/vnc-server resources: limits: cpu: 100m memory: 64Mi requests: cpu: 50m memory: 32Mi x-server: repository: quay.io/aerokube/xvfb-server resources: limits: cpu: 100m memory: 128Mi requests: cpu: 50m memory: 64Mi
https://aerokube.com/moon/latest/#video-recording
storage: bucket: "moon" endpoint: "https:s3.example.com" filename: "" pattern: "" secretRef: accessKey: "accessKey" secretKey: "secretKey" name: secretName1 deployment: replicas: 1 securityContext: runAsUser: 1000 experimentalUI: true moonCPULimit: 150m moonCPURequest: 50m moonMemoryLimit: 128Mi moonMemoryRequest: 64Mi moonConfCPULimit: 100m moonConfCPURequest: 25m moonConfMemoryLimit: 64Mi moonConfMemoryRequest: 32Mi moonUICPULimit: 200m moonUICPURequest: 100m moonUIMemoryLimit: 64Mi moonUIMemoryRequest: 32Mi browsers: default: selenium: MicrosoftEdge: repository: quay.io/browser/microsoft-edge-stable #default: 112.0.1722.58-5 chrome: repository: quay.io/browser/google-chrome-stable default: "112.0" firefox: repository: quay.io/browser/firefox-mozilla-build #default: 112.0.2-4 opera: repository: quay.io/browser/opera-stable #default: 98.0.4759.15-6 safari: repository: quay.io/browser/safari #default: 15.0-0 cypress: electron: repository: quay.io/browsers/cypress-electron chromium: repository: quay.io/browsers/cypress-chromium chrome: repository: quay.io/browsers/cypress-chrome firefox: repository: quay.io/browsers/cypress-firefox edge: repository: quay.io/browsers/cypress-edge playwright: chrome: repository: quay.io/browser/playwright-chrome chromium: repository: quay.io/browser/playwright-chromium firefox: repository: quay.io/browser/playwright-firefox webkit: repository: quay.io/browser/playwright-webkit
chrome pod logs:
browser Starting ChromeDriver 112.0.5615.49 (bd2a7bcb881c11e8cfe3078709382934e3916914-refs/branch-heads/5615@{#936}) on port 4444 browser Only local connections are allowed. vnc-server 12/05/2023 13:04:24 passing arg to libvncserver: -passwd vnc-server 12/05/2023 13:04:24 passing arg to libvncserver: -rfbport browser Please see https://chromedriver.chromium.org/security-considerations for suggestions on keeping ChromeDriver safe. browser ChromeDriver was started successfully. x-server _XSERVTransmkdir: ERROR: euid != 0,directory /tmp/.X11-unix will not be created. x-server Unable to revert mtime: /usr/share/fonts defender 2023/05/12 13:04:23 headless mode: false defender 2023/05/12 13:04:23 listening on :4545 defender 2023/05/12 13:04:23 waiting for webdriver defender 2023/05/12 13:04:25 webdriver started in 1.520595007s defender 2023/05/12 13:04:25 proxy POST to http://127.0.0.1:4444/session] vnc-server 12/05/2023 13:04:24 passing arg to libvncserver: 5900 vnc-server 12/05/2023 13:04:24 x11vnc version: 0.9.16 lastmod: 2019-01-05 pid: 11 vnc-server 12/05/2023 13:04:24 vnc-server 12/05/2023 13:04:24 wait_for_client: WAIT:0 vnc-server 12/05/2023 13:04:24 vnc-server 12/05/2023 13:04:24 initialize_screen: fb_depth/fb_bpp/fb_Bpl 24/32/2560 vnc-server 12/05/2023 13:04:24 vnc-server 12/05/2023 13:04:24 Listening for VNC connections on TCP port 5900 vnc-server 12/05/2023 13:04:24 Listening for VNC connections on TCP6 port 5900 vnc-server 12/05/2023 13:04:24 listen6: bind: Address already in use vnc-server 12/05/2023 13:04:24 Not listening on IPv6 interface. vnc-server 12/05/2023 13:04:24 vnc-server vnc-server The VNC desktop is: chrome-112-0-f5a286a4-69ff-45a4-b79c-79174521e7dc:0 vnc-server PORT=5900 Stream closed EOF for ... (ca-certs) Stream closed EOF for ... (x-server-init) defender 2023/05/12 13:04:27 received terminated (15) signal defender 2023/05/12 13:04:27 graceful shutdown in 30s defender 2023/05/12 13:04:27 http: Server closed defender 2023/05/12 13:04:27 exited vnc-server caught signal: 15 Stream closed EOF for ... (browser) Stream closed EOF for ... (defender) Stream closed EOF for ... (vnc-server) x-server XIO: fatal IO error 2 (No such file or directory) on X server ":0" x-server after 297 requests (297 known processed) with 7 events remaining. Stream closed EOF for ... (x-server)
moon2 logs
moon 2023/05/12 13:04:14 moon2-77b45b8775-vs56d: browserset controller: browsers "default": updated moon 2023/05/12 13:04:19 moon2-77b45b8775-vs56d: moon: chrome-112-0-f5a286a4-69ff-45a4-b79c-79174521e7dc: starting new session moon 2023/05/12 13:04:19 moon2-77b45b8775-vs56d: moon: chrome-112-0-f5a286a4-69ff-45a4-b79c-79174521e7dc: creating config map moon 2023/05/12 13:04:19 moon2-77b45b8775-vs56d: moon: chrome-112-0-f5a286a4-69ff-45a4-b79c-79174521e7dc: config map created moon 2023/05/12 13:04:19 moon2-77b45b8775-vs56d: moon: chrome-112-0-f5a286a4-69ff-45a4-b79c-79174521e7dc: creating pod moon 2023/05/12 13:04:19 moon2-77b45b8775-vs56d: moon: chrome-112-0-f5a286a4-69ff-45a4-b79c-79174521e7dc: pod created moon 2023/05/12 13:04:19 moon2-77b45b8775-vs56d: moon: chrome-112-0-f5a286a4-69ff-45a4-b79c-79174521e7dc: waiting pod moon 2023/05/12 13:04:25 moon2-77b45b8775-vs56d: moon: chrome-112-0-f5a286a4-69ff-45a4-b79c-79174521e7dc: pod is up moon 2023/05/12 13:04:25 moon2-77b45b8775-vs56d: moon: chrome-112-0-f5a286a4-69ff-45a4-b79c-79174521e7dc: waiting driver moon 2023/05/12 13:04:25 moon2-77b45b8775-vs56d: moon: chrome-112-0-f5a286a4-69ff-45a4-b79c-79174521e7dc: driver is up moon 2023/05/12 13:04:25 moon2-77b45b8775-vs56d: moon: chrome-112-0-f5a286a4-69ff-45a4-b79c-79174521e7dc: session failed: driver response code: 500: Internal Server Error moon 2023/05/12 13:04:25 moon2-77b45b8775-vs56d: moon: chrome-112-0-f5a286a4-69ff-45a4-b79c-79174521e7dc: deleting pod moon 2023/05/12 13:04:25 moon2-77b45b8775-vs56d: moon: chrome-112-0-f5a286a4-69ff-45a4-b79c-79174521e7dc: pod deleted moon 2023/05/12 13:04:25 moon2-77b45b8775-vs56d: moon: chrome-112-0-f5a286a4-69ff-45a4-b79c-79174521e7dc: deleting config map moon 2023/05/12 13:04:25 moon2-77b45b8775-vs56d: moon: chrome-112-0-f5a286a4-69ff-45a4-b79c-79174521e7dc: config map deleted
Any help is appreciated! Thanks in advance.
— Reply to this email directly, view it on GitHub https://github.com/aerokube/moon/issues/377, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAKY23K5U2VQW4S6VWH57PTXFYZDXANCNFSM6AAAAAAX7QMMPU . You are receiving this because you are subscribed to this thread.Message ID: @.***>
hey @aandryashin We actually got Ubuntu nodes. When I add the argument to the capabilites, it's still not working. Same error. Capabilities:
{
"capabilities": {
"google:chromeOptions": {
"args": ["--no-sandbox"]
},
"alwaysMatch": {
"goog:chromeOptions": {
"excludeSwitches": [
"enable-automation"
]
}
}
}
}
Hello! goog:chromeOptions should be under alaysMatch.
пн, 15 мая 2023 г., 10:52 toby181 @.***>:
hey @aandryashin https://github.com/aandryashin We actually got Ubuntu nodes. When I add the argument to the capabilites, it's still not working. Same error. Capabilities:
{ "capabilities": { "google:chromeOptions": { "args": ["--no-sandbox"] }, "alwaysMatch": { "goog:chromeOptions": { "excludeSwitches": [ "enable-automation" ] } } } }
— Reply to this email directly, view it on GitHub https://github.com/aerokube/moon/issues/377#issuecomment-1547361365, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAKY23OP2TQBHXMVLBSTUMLXGHODBANCNFSM6AAAAAAX7QMMPU . You are receiving this because you were mentioned.Message ID: @.***>
Hey @aandryashin , Sorry, somehow I missed your response. It works with the correct syntax, as you mentioned:
{
"capabilities": {
"alwaysMatch": {
"goog:chromeOptions": {
"excludeSwitches": [
"enable-automation"
],
"args": ["--no-sandbox"]
}
}
}
}
Thanks for that!
What I did not figure out yet is how to add this argument by default on kubernetes in the browser custom resource.
My HelmRelease looks like this
...
browsers:
default:
selenium:
chrome:
repository: quay.io/browser/google-chrome-stable
default: "112.0"
env:
- name: foo
value: "bar"
capabilities:
add:
- --no-sandbox
but whenenver I try to start a manual chrome session, it immediately fails and the moon2 pod says:
moon 2023/05/22 11:31:49 moon2-77b45b8775-wc6vt: moon: chrome-112-0-09c26b74-44fc-41b1-861a-6ff682739a78: starting new session moon 2023/05/22 11:31:49 http: panic serving 192.168.47.164:57212: runtime error: invalid memory address or nil pointer dereference moon goroutine 1114 [running]: moon net/http.(conn).serve.func1() moon /opt/hostedtoolcache/go/1.20.3/x64/src/net/http/server.go:1854 +0xbf moon panic({0x13b1220, 0x2167050}) moon /opt/hostedtoolcache/go/1.20.3/x64/src/runtime/panic.go:890 +0x263 moon main.createSession.func1.3(0xc000162100, 0xc000276c90, 0xc000615d10, 0xc00046e9a0, 0xc000012288, 0xc000389d10, {0xc112f2894ee4815e?, 0x122e5f8af67?, 0x2187b60?}, {0x17ac388, ...}) moon /home/runner/work/moon2/moon2/cmd/moon/session.go:921 +0x5dd7 moon main.createSession.func1({0x17aba70, 0xc0004f6fc0}, 0xc000162100) moon /home/runner/work/moon2/moon2/cmd/moon/session.go:1287 +0x605 moon net/http.HandlerFunc.ServeHTTP(0xc00046e9a0?, {0x17aba70?, 0xc0004f6fc0?}, 0x25?) moon /opt/hostedtoolcache/go/1.20.3/x64/src/net/http/server.go:2122 +0x2f moon main.withQuota.func1({0x17aba70, 0xc0004f6fc0}, 0xc000162100) moon /home/runner/work/moon2/moon2/cmd/moon/handlers.go:91 +0x15b moon net/http.HandlerFunc.ServeHTTP(0x0?, {0x17aba70?, 0xc0004f6fc0?}, 0x0?) moon /opt/hostedtoolcache/go/1.20.3/x64/src/net/http/server.go:2122 +0x2f moon main.post.func1({0x17aba70, 0xc0004f6fc0}, 0xffffffffffffffff?) moon /home/runner/work/moon2/moon2/cmd/moon/handlers.go:101 +0x7b moon net/http.HandlerFunc.ServeHTTP(0xc0004f6fc0?, {0x17aba70?, 0xc0004f6fc0?}, 0xc00004bbc6?) moon /opt/hostedtoolcache/go/1.20.3/x64/src/net/http/server.go:2122 +0x2f moon net/http.(ServeMux).ServeHTTP(0x0?, {0x17aba70, 0xc0004f6fc0}, 0xc000162100) moon /opt/hostedtoolcache/go/1.20.3/x64/src/net/http/server.go:2500 +0x149 moon net/http.serverHandler.ServeHTTP({0x179fc88?}, {0x17aba70, 0xc0004f6fc0}, 0xc000162100) moon /opt/hostedtoolcache/go/1.20.3/x64/src/net/http/server.go:2936 +0x316 moon net/http.(conn).serve(0xc000731680, {0x17ac3c0, 0xc00048d170}) moon /opt/hostedtoolcache/go/1.20.3/x64/src/net/http/server.go:1995 +0x612 moon created by net/http.(Server).Serve moon /opt/hostedtoolcache/go/1.20.3/x64/src/net/http/server.go:3089 +0x5ed
Something like that fails as well
capabilities:
add:
- '"alwaysMatch": {"goog:chromeOptions": {"excludeSwitches": ["enable-automation"],"args": ["--no-sandbox"] }}'
@toby181 this is not supported in Moon settings.
hi @vania-pooh the browsersets CRD offers a configuration option for capabilities, see https://github.com/aerokube/charts/blob/2.5.0/moon2/crds/moon.aerokube.com_browsersets.yaml#L2328. Is that something different then?
Hello, that is not selenium capabilities, that is kubernetes container security context capabilities. You can easily add chrome arguments on test sie. We have decided to not add selenoum caps on the fly to avoid "magic" behaviour. Everything should be under test control.
пн, 22 мая 2023 г., 16:43 toby181 @.***>:
hi @vania-pooh https://github.com/vania-pooh the browsersets CRD offers a configuration option for capabilities, see https://github.com/aerokube/charts/blob/2.5.0/moon2/crds/moon.aerokube.com_browsersets.yaml#L2328 . Is that something different then?
— Reply to this email directly, view it on GitHub https://github.com/aerokube/moon/issues/377#issuecomment-1557246322, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAKY23L5DUL7K7L6GG6EPK3XHNUPFANCNFSM6AAAAAAX7QMMPU . You are receiving this because you were mentioned.Message ID: @.***>
got it, thanks for your support!
Hi together,
We've deployed moon2 in version 2.5 to k8s but we're having issues with manually creating a session if we select chrome or edge via the UI, firefox and opera are working.
The deployment configuration and the logs can be found below. Chrome basically terminates with:
helmrelease values:
chrome pod logs:
moon2 logs
Any help is appreciated! Thanks in advance.