search

zalando / zalenium

A flexible and scalable container based Selenium Grid with video recording, live preview, basic auth & dashboard.
https://opensource.zalando.com/zalenium/
Other
2.38k stars 575 forks source link

Zalenium spins up lots of containers #211

Closed mojsha closed 7 years ago

mojsha commented 7 years ago

I have tried using Zalenium with Minishift and noticed that it seems to spin up alot of containers.

I then get a "Gateway timeout" error from my client.

pearj commented 7 years ago

Can you attached the zalenium container logs?

mojsha commented 7 years ago

Ok, I don't know what happened, but now whenever I start it up I am getting an error:

 | 12:10:29.699 DEBUG - Env. variable BUILD_URL value is not set, falling back to default:   | 12:10:29.722 DEBUG - Env. variable BUILD_URL value is not set, falling back to default:   | Exception in thread "Thread-12" io.fabric8.kubernetes.client.KubernetesClientException: Failure executing: POST at: https://kubernetes.default.svc/api/v1/namespaces/myproject/pods. Message: Pod in version "v1" cannot be handled as a Pod: [pos 123]: json: expect char '"' but got char 'n'. Received status: Status(apiVersion=v1, code=400, details=null, kind=Status, message=Pod in version "v1" cannot be handled as a Pod: [pos 123]: json: expect char '"' but got char 'n', metadata=ListMeta(resourceVersion=null, selfLink=null, additionalProperties={}), reason=BadRequest, status=Failure, additionalProperties={}).   | at io.fabric8.kubernetes.client.dsl.base.OperationSupport.requestFailure(OperationSupport.java:470)   | at io.fabric8.kubernetes.client.dsl.base.OperationSupport.assertResponseCode(OperationSupport.java:409)   | at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleResponse(OperationSupport.java:379)   | at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleResponse(OperationSupport.java:343)   | at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleCreate(OperationSupport.java:226)   | at io.fabric8.kubernetes.client.dsl.base.BaseOperation.handleCreate(BaseOperation.java:741)   | at io.fabric8.kubernetes.client.dsl.base.BaseOperation.create(BaseOperation.java:334)   | at io.fabric8.kubernetes.client.dsl.base.BaseOperation$1.apply(BaseOperation.java:351)   | at io.fabric8.kubernetes.api.model.DoneablePod.done(DoneablePod.java:26)   | at de.zalando.ep.zalenium.container.kubernetes.KubernetesContainerClient.createContainer(KubernetesContainerClient.java:331)   | at de.zalando.ep.zalenium.proxy.DockerSeleniumStarterRemoteProxy.startDockerSeleniumContainer(DockerSeleniumStarterRemoteProxy.java:428)   | at de.zalando.ep.zalenium.proxy.DockerSeleniumStarterRemoteProxy.lambda$createContainersOnStartup$5(DockerSeleniumStarterRemoteProxy.java:531)   | at java.lang.Thread.run(Thread.java:748)

Now it's no longer working.

I am starting it using the below: oc run zalenium --image=dosel/zalenium --env="ZALENIUM_KUBERNETES_CPU_REQUEST=250m" --env="ZALENIUM_KUBERNETES_CPU_LIMIT=500m" --env="ZALENIUM_KUBERNETES_MEMORY_REQUEST=1Gi" --overrides="{\"spec\": {\"template\": {\"spec\": {\"serviceAccount\": \"zalenium\"}}}}" --port=4444 -- start --firefoxContainers 0 --chromeContainers 2 --debugEnabled true --seleniumImageName docker.io/elgalu/selenium:latest

pearj commented 7 years ago

Are you running openshift 3.6 (as minishift)? I haven't tried that version yet. Although when I was writing the initial k8s doco I did try minikube which had k8s 1.6 I thought. So I think zalenium runs ok on 1.6

pearj commented 7 years ago

I just started zalenium in minishift 3.6.0 and it started up ok, so that's a bit weird what you're seeing

mojsha commented 7 years ago

Ok, I have now downloaded the latest Minishift client and then run it and get this (as was the case initially):

Data Processing Agreement

  | 09:06:45.455 DEBUG - Handling: GET /status   | 09:06:45.654 DEBUG - Executing: org.openqa.selenium.remote.server.handler.Status@78c3a438)   | 09:06:45.710 DEBUG - Done: org.openqa.selenium.remote.server.handler.Status@78c3a438   | 09:06:45.711 DEBUG - Finished: GET /status   | 09:06:57.449 DEBUG - Handling: GET /status   | 09:06:57.449 DEBUG - Executing: org.openqa.selenium.remote.server.handler.Status@54d43e5f)   | 09:06:57.449 DEBUG - Done: org.openqa.selenium.remote.server.handler.Status@54d43e5f   | 09:06:57.450 DEBUG - Finished: GET /status   | 09:07:17.282 INFO - Got a request to create a new session: Capabilities [{acceptInsecureCerts=true, browserName=firefox, version=, platform=ANY}]   | 09:07:17.283 DEBUG - Validating {acceptInsecureCerts=true, browserName=firefox, version=, platform=ANY} in node with capabilities {browserName=firefox, maxInstances=1, platform=LINUX}   | 09:07:17.289 DEBUG - Available nodes: [http://172.17.0.7:30000]   | 09:07:17.289 DEBUG - Validating {acceptInsecureCerts=true, browserName=firefox, version=, platform=ANY} in node with capabilities {browserName=firefox, maxInstances=1, platform=LINUX}   | 09:07:17.289 INFO - [DS] Starting new node for {tz=Europe/Berlin, acceptInsecureCerts=true, browserName=firefox, screenResolution=1900x1880, version=, platform=ANY}.   | 09:07:17.305 DEBUG - [DS] 2 docker-selenium containers running   | 09:07:17.305 DEBUG - Env. variable BUILD_URL value is not set, falling back to default:   | 09:07:22.290 DEBUG - Available nodes: [http://172.17.0.7:30000]   | 09:07:22.291 DEBUG - Validating {waitingFor_FIREFOX_Node=1, tz=Europe/Berlin, acceptInsecureCerts=true, browserName=firefox, screenResolution=1900x1880, version=, platform=ANY} in node with capabilities {browserName=firefox, maxInstances=1, platform=LINUX}   | 09:07:22.305 DEBUG - [DS] 3 docker-selenium containers running   | 09:07:22.305 DEBUG - Env. variable BUILD_URL value is not set, falling back to default:   | 09:07:27.299 DEBUG - Available nodes: [http://172.17.0.7:30000]   | 09:07:27.299 DEBUG - Validating {waitingFor_FIREFOX_Node=2, tz=Europe/Berlin, acceptInsecureCerts=true, browserName=firefox, screenResolution=1900x1880, version=, platform=ANY} in node with capabilities {browserName=firefox, maxInstances=1, platform=LINUX}   | 09:07:27.329 DEBUG - [DS] 4 docker-selenium containers running   | 09:07:27.330 DEBUG - Env. variable BUILD_URL value is not set, falling back to default:   | 09:07:32.301 DEBUG - Available nodes: [http://172.17.0.7:30000]   | 09:07:32.302 DEBUG - Validating {waitingFor_FIREFOX_Node=3, tz=Europe/Berlin, acceptInsecureCerts=true, browserName=firefox, screenResolution=1900x1880, version=, platform=ANY} in node with capabilities {browserName=firefox, maxInstances=1, platform=LINUX}   | 09:07:32.381 DEBUG - [DS] 5 docker-selenium containers running   | 09:07:32.382 DEBUG - Env. variable BUILD_URL value is not set, falling back to default:   | 09:07:37.304 DEBUG - Available nodes: [http://172.17.0.7:30000]   | 09:07:37.314 DEBUG - Validating {waitingFor_FIREFOX_Node=4, tz=Europe/Berlin, acceptInsecureCerts=true, browserName=firefox, screenResolution=1900x1880, version=, platform=ANY} in node with capabilities {browserName=firefox, maxInstances=1, platform=LINUX}   | 09:07:40.464 DEBUG - [DS] 6 docker-selenium containers running   | 09:07:40.604 DEBUG - Env. variable BUILD_URL value is not set, falling back to default:   | 09:07:42.942 DEBUG - Available nodes: [http://172.17.0.7:30000]   | 09:07:43.018 DEBUG - Validating {waitingFor_FIREFOX_Node=5, tz=Europe/Berlin, acceptInsecureCerts=true, browserName=firefox, screenResolution=1900x1880, version=, platform=ANY} in node with capabilities {browserName=firefox, maxInstances=1, platform=LINUX}   | 09:07:43.226 DEBUG - [DS] 7 docker-selenium containers running   | 09:07:43.227 DEBUG - Env. variable BUILD_URL value is not set, falling back to default:   | 09:07:43.725 DEBUG - getting the following registration request : {"class":"org.openqa.grid.common.RegistrationRequest","name":null,"description":null,"configuration":{"hubHost":"172.17.0.7","hubPort":4445,"id":"http://172.17.0.8:40001","capabilities":[{"chrome_binary":"/usr/bin/google-chrome-stable","seleniumProtocol":"WebDriver","acceptSslCerts":true,"screen-resolution":"1900x1880","tz":"Europe/Berlin","browserName":"chrome","maxInstances":1,"screenResolution":"1900x1880","resolution":"1900x1880","version":"60.0.3112.101","platform":"LINUX"}],"downPollingLimit":2,"hub":"http://172.17.0.7:4445/grid/register","nodePolling":99000,"nodeStatusCheckTimeout":5000,"proxy":"de.zalando.ep.zalenium.proxy.DockerSeleniumRemoteProxy","register":true,"registerCycle":0,"unregisterIfStillDownAfter":2500,"cleanUpCycle":5000,"custom":{},"host":"172.17.0.8","maxSession":1,"servlets":[],"withoutServlets":[],"browserTimeout":0,"debug":false,"jettyMaxThreads":null,"log":null,"port":40001,"role":"node","timeout":1800}}   | 09:07:45.529 DEBUG - Using class de.zalando.ep.zalenium.proxy.DockerSeleniumRemoteProxy   | 09:07:45.905 DEBUG - getting the following registration request : {"class":"org.openqa.grid.common.RegistrationRequest","name":null,"description":null,"configuration":{"hubHost":"172.17.0.7","hubPort":4445,"id":"http://172.17.0.6:40000","capabilities":[{"chrome_binary":"/usr/bin/google-chrome-stable","seleniumProtocol":"WebDriver","acceptSslCerts":true,"screen-resolution":"1900x1880","tz":"Europe/Berlin","browserName":"chrome","maxInstances":1,"screenResolution":"1900x1880","resolution":"1900x1880","version":"60.0.3112.101","platform":"LINUX"}],"downPollingLimit":2,"hub":"http://172.17.0.7:4445/grid/register","nodePolling":113000,"nodeStatusCheckTimeout":5000,"proxy":"de.zalando.ep.zalenium.proxy.DockerSeleniumRemoteProxy","register":true,"registerCycle":0,"unregisterIfStillDownAfter":2500,"cleanUpCycle":5000,"custom":{},"host":"172.17.0.6","maxSession":1,"servlets":[],"withoutServlets":[],"browserTimeout":0,"debug":false,"jettyMaxThreads":null,"log":null,"port":40000,"role":"node","timeout":1800}}   | 09:07:45.916 DEBUG - Using class de.zalando.ep.zalenium.proxy.DockerSeleniumRemoteProxy   | 09:07:46.791 DEBUG - getting the following registration request : {"class":"org.openqa.grid.common.RegistrationRequest","name":null,"description":null,"configuration":{"hubHost":"172.17.0.7","hubPort":4445,"id":"http://172.17.0.9:40002","capabilities":[{"firefox_binary":"/home/seluser/firefox-for-sel-3/firefox","seleniumProtocol":"WebDriver","acceptSslCerts":true,"screen-resolution":"1900x1880","tz":"Europe/Berlin","browserName":"firefox","maxInstances":1,"screenResolution":"1900x1880","resolution":"1900x1880","version":"52.0.2","platform":"LINUX"}],"downPollingLimit":2,"hub":"http://172.17.0.7:4445/grid/register","nodePolling":103000,"nodeStatusCheckTimeout":5000,"proxy":"de.zalando.ep.zalenium.proxy.DockerSeleniumRemoteProxy","register":true,"registerCycle":0,"unregisterIfStillDownAfter":2500,"cleanUpCycle":5000,"custom":{},"host":"172.17.0.9","maxSession":1,"servlets":[],"withoutServlets":[],"browserTimeout":0,"debug":false,"jettyMaxThreads":null,"log":null,"port":40002,"role":"node","timeout":1800}}   | 09:07:47.603 DEBUG - Using class de.zalando.ep.zalenium.proxy.DockerSeleniumRemoteProxy   | 09:07:48.764 DEBUG - Available nodes: [http://172.17.0.7:30000]   | 09:07:48.780 DEBUG - Validating {waitingFor_FIREFOX_Node=6, tz=Europe/Berlin, acceptInsecureCerts=true, browserName=firefox, screenResolution=1900x1880, version=, platform=ANY} in node with capabilities {browserName=firefox, maxInstances=1, platform=LINUX}   | 09:07:49.586 INFO - Got a request to create a new session: Capabilities [{browserName=chrome, version=, platform=ANY}]   | 09:07:49.586 DEBUG - Validating {browserName=chrome, version=, platform=ANY} in node with capabilities {browserName=firefox, maxInstances=1, platform=LINUX}   | 09:07:49.617 DEBUG - Validating {browserName=chrome, version=, platform=ANY} in node with capabilities {browserName=chrome, maxInstances=1, platform=LINUX}   | 09:07:49.617 DEBUG - Available nodes: [http://172.17.0.7:30000]   | 09:07:49.617 DEBUG - Validating {waitingFor_FIREFOX_Node=7, tz=Europe/Berlin, acceptInsecureCerts=true, browserName=firefox, screenResolution=1900x1880, version=, platform=ANY} in node with capabilities {browserName=firefox, maxInstances=1, platform=LINUX}   | 09:07:49.617 DEBUG - Available nodes: [http://172.17.0.7:30000]   | 09:07:49.617 DEBUG - Validating {browserName=chrome, version=, platform=ANY} in node with capabilities {browserName=firefox, maxInstances=1, platform=LINUX}   | 09:07:49.617 DEBUG - Validating {browserName=chrome, version=, platform=ANY} in node with capabilities {browserName=chrome, maxInstances=1, platform=LINUX}   | 09:07:49.618 INFO - [DS] Starting new node for {tz=Europe/Berlin, browserName=chrome, screenResolution=1900x1880, version=, platform=ANY}.   | 09:07:49.733 DEBUG - starting cleanup thread   | 09:07:49.756 DEBUG - getting the following registration request : {"class":"org.openqa.grid.common.RegistrationRequest","name":null,"description":null,"configuration":{"hubHost":"172.17.0.7","hubPort":4445,"id":"http://172.17.0.10:40003","capabilities":[{"firefox_binary":"/home/seluser/firefox-for-sel-3/firefox","seleniumProtocol":"WebDriver","acceptSslCerts":true,"screen-resolution":"1900x1880","tz":"Europe/Berlin","browserName":"firefox","maxInstances":1,"screenResolution":"1900x1880","resolution":"1900x1880","version":"52.0.2","platform":"LINUX"}],"downPollingLimit":2,"hub":"http://172.17.0.7:4445/grid/register","nodePolling":97000,"nodeStatusCheckTimeout":5000,"proxy":"de.zalando.ep.zalenium.proxy.DockerSeleniumRemoteProxy","register":true,"registerCycle":0,"unregisterIfStillDownAfter":2500,"cleanUpCycle":5000,"custom":{},"host":"172.17.0.10","maxSession":1,"servlets":[],"withoutServlets":[],"browserTimeout":0,"debug":false,"jettyMaxThreads":null,"log":null,"port":40003,"role":"node","timeout":1800}}   | 09:07:50.262 DEBUG - [DS] 8 docker-selenium containers running   | 09:07:50.283 DEBUG - Using class de.zalando.ep.zalenium.proxy.DockerSeleniumRemoteProxy   | 09:07:50.301 DEBUG - Env. variable BUILD_URL value is not set, falling back to default:   | 09:07:50.323 DEBUG - starting cleanup thread   | 09:07:50.627 DEBUG - [DS] 8 docker-selenium containers running   | 09:07:50.628 DEBUG - Env. variable BUILD_URL value is not set, falling back to default:   | 09:07:50.772 DEBUG - cleanup thread starting...   | 09:07:51.107 INFO - Registered a node http://172.17.0.8:40001   | 09:07:51.167 DEBUG - proxy added http://172.17.0.8:40001   | 09:07:51.182 DEBUG - cleanup thread starting...   | 09:07:51.184 DEBUG - Available nodes: [http://172.17.0.8:40001, http://172.17.0.7:30000]   | 09:07:51.184 DEBUG - Validating {waitingFor_FIREFOX_Node=8, tz=Europe/Berlin, acceptInsecureCerts=true, browserName=firefox, screenResolution=1900x1880, version=, platform=ANY} in node with capabilities {chrome_binary=/usr/bin/google-chrome-stable, seleniumProtocol=WebDriver, acceptSslCerts=true, screen-resolution=1900x1880, tz=Europe/Berlin, browserName=chrome, maxInstances=1, screenResolution=1900x1880, resolution=1900x1880, version=60.0.3112.101, platform=LINUX}   | 09:07:51.184 DEBUG - Validating {waitingFor_FIREFOX_Node=8, tz=Europe/Berlin, acceptInsecureCerts=true, browserName=firefox, screenResolution=1900x1880, version=, platform=ANY} in node with capabilities {browserName=firefox, maxInstances=1, platform=LINUX}   | 09:07:51.324 DEBUG - starting cleanup thread   | 09:07:51.404 DEBUG - Available nodes: [http://172.17.0.8:40001, http://172.17.0.7:30000]   | 09:07:51.404 DEBUG - Validating {tz=Europe/Berlin, browserName=chrome, waitingFor_CHROME_Node=1, screenResolution=1900x1880, version=, platform=ANY} in node with capabilities {chrome_binary=/usr/bin/google-chrome-stable, seleniumProtocol=WebDriver, acceptSslCerts=true, screen-resolution=1900x1880, tz=Europe/Berlin, browserName=chrome, maxInstances=1, screenResolution=1900x1880, resolution=1900x1880, version=60.0.3112.101, platform=LINUX}   | 09:07:51.453 DEBUG - Trying to create a new session on node http://172.17.0.8:40001   | 09:07:51.463 DEBUG - Validating {tz=Europe/Berlin, browserName=chrome, waitingFor_CHROME_Node=1, screenResolution=1900x1880, version=, platform=ANY} in node with capabilities {chrome_binary=/usr/bin/google-chrome-stable, seleniumProtocol=WebDriver, acceptSslCerts=true, screen-resolution=1900x1880, tz=Europe/Berlin, browserName=chrome, maxInstances=1, screenResolution=1900x1880, resolution=1900x1880, version=60.0.3112.101, platform=LINUX}   | 09:07:51.478 DEBUG - Validating {tz=Europe/Berlin, browserName=chrome, waitingFor_CHROME_Node=1, screenResolution=1900x1880, version=, platform=ANY} in node with capabilities {chrome_binary=/usr/bin/google-chrome-stable, seleniumProtocol=WebDriver, acceptSslCerts=true, screen-resolution=1900x1880, tz=Europe/Berlin, browserName=chrome, maxInstances=1, screenResolution=1900x1880, resolution=1900x1880, version=60.0.3112.101, platform=LINUX}   | 09:07:51.789 DEBUG - starting cleanup thread   | 09:07:51.813 INFO - Registered a node http://172.17.0.10:40003   | 09:07:51.826 DEBUG - cleanup thread starting...   | 09:07:51.912 WARN - [DS] Max. number of docker-selenium containers has been reached, no more will be created until the number decreases below 10.   | 09:07:51.962 INFO - [DS] No container was created, will try again in a moment...   | 09:07:51.672 INFO - Trying to create a new session on test slot {chrome_binary=/usr/bin/google-chrome-stable, seleniumProtocol=WebDriver, acceptSslCerts=true, screen-resolution=1900x1880, tz=Europe/Berlin, browserName=chrome, maxInstances=1, screenResolution=1900x1880, resolution=1900x1880, version=60.0.3112.101, platform=LINUX}   | 09:07:52.315 DEBUG - cleanup thread starting...   | 09:07:52.319 DEBUG - http://172.17.0.8:40001 Creating session for: {tz=Europe/Berlin, browserName=chrome, waitingFor_CHROME_Node=1, screenResolution=1900x1880, version=, platform=ANY}   | 09:07:52.390 DEBUG - http://172.17.0.8:40001 lastCommand: POST - /session executing...   | 09:07:52.428 INFO - Registered a node http://172.17.0.6:40000   | 09:07:52.497 DEBUG - proxy added http://172.17.0.6:40000   | 09:07:52.503 DEBUG - proxy added http://172.17.0.10:40003   | 09:07:52.504 DEBUG - Available nodes: [http://172.17.0.6:40000, http://172.17.0.10:40003, http://172.17.0.7:30000, http://172.17.0.8:40001]   | 09:07:52.515 INFO - Registered a node http://172.17.0.9:40002   | 09:07:52.526 DEBUG - Validating {waitingFor_FIREFOX_Node=9, tz=Europe/Berlin, acceptInsecureCerts=true, browserName=firefox, screenResolution=1900x1880, version=, platform=ANY} in node with capabilities {chrome_binary=/usr/bin/google-chrome-stable, seleniumProtocol=WebDriver, acceptSslCerts=true, screen-resolution=1900x1880, tz=Europe/Berlin, browserName=chrome, maxInstances=1, screenResolution=1900x1880, resolution=1900x1880, version=60.0.3112.101, platform=LINUX}   | 09:07:52.527 DEBUG - Validating {waitingFor_FIREFOX_Node=9, tz=Europe/Berlin, acceptInsecureCerts=true, browserName=firefox, screenResolution=1900x1880, version=, platform=ANY} in node with capabilities {firefox_binary=/home/seluser/firefox-for-sel-3/firefox, seleniumProtocol=WebDriver, acceptSslCerts=true, screen-resolution=1900x1880, tz=Europe/Berlin, browserName=firefox, maxInstances=1, screenResolution=1900x1880, resolution=1900x1880, version=52.0.2, platform=LINUX}   | 09:07:52.538 DEBUG - Trying to create a new session on node http://172.17.0.10:40003   | 09:07:52.539 DEBUG - Validating {waitingFor_FIREFOX_Node=9, tz=Europe/Berlin, acceptInsecureCerts=true, browserName=firefox, screenResolution=1900x1880, version=, platform=ANY} in node with capabilities {firefox_binary=/home/seluser/firefox-for-sel-3/firefox, seleniumProtocol=WebDriver, acceptSslCerts=true, screen-resolution=1900x1880, tz=Europe/Berlin, browserName=firefox, maxInstances=1, screenResolution=1900x1880, resolution=1900x1880, version=52.0.2, platform=LINUX}   | 09:07:52.539 DEBUG - Validating {waitingFor_FIREFOX_Node=9, tz=Europe/Berlin, acceptInsecureCerts=true, browserName=firefox, screenResolution=1900x1880, version=, platform=ANY} in node with capabilities {firefox_binary=/home/seluser/firefox-for-sel-3/firefox, seleniumProtocol=WebDriver, acceptSslCerts=true, screen-resolution=1900x1880, tz=Europe/Berlin, browserName=firefox, maxInstances=1, screenResolution=1900x1880, resolution=1900x1880, version=52.0.2, platform=LINUX}   | 09:07:52.540 INFO - Trying to create a new session on test slot {firefox_binary=/home/seluser/firefox-for-sel-3/firefox, seleniumProtocol=WebDriver, acceptSslCerts=true, screen-resolution=1900x1880, tz=Europe/Berlin, browserName=firefox, maxInstances=1, screenResolution=1900x1880, resolution=1900x1880, version=52.0.2, platform=LINUX}   | 09:07:52.541 DEBUG - http://172.17.0.10:40003 Creating session for: {waitingFor_FIREFOX_Node=9, tz=Europe/Berlin, acceptInsecureCerts=true, browserName=firefox, screenResolution=1900x1880, version=, platform=ANY}   | 09:07:52.542 DEBUG - http://172.17.0.10:40003 lastCommand: POST - /session executing...   | 09:07:52.569 DEBUG - proxy added http://172.17.0.9:40002   | 09:07:55.798 DEBUG - http://172.17.0.8:40001 lastCommand: POST - /session executed.   | 09:07:55.800 INFO - null-40001-lr4xk [bash, -c, start-video]   | 09:07:56.508 DEBUG - http://172.17.0.8:40001 lastCommand: POST - /session/3194bf9c-b9e8-447c-a066-ba7c156ea42f/url executing...   | 09:07:57.583 INFO - null-40001-lr4xk video-rec: started   | Waiting for ffmpeg video recording to start...   | ..Videos at /home/seluser/videos/vid_chrome_40001.mp4 started to be recorded! (wait-video-rec.sh)   |     | 09:08:00.039 DEBUG - http://172.17.0.8:40001 lastCommand: POST - /session/3194bf9c-b9e8-447c-a066-ba7c156ea42f/url executed.   | 09:08:00.231 DEBUG - http://172.17.0.8:40001 lastCommand: GET - /session/3194bf9c-b9e8-447c-a066-ba7c156ea42f/title executing...   | 09:08:00.399 DEBUG - http://172.17.0.8:40001 lastCommand: GET - /session/3194bf9c-b9e8-447c-a066-ba7c156ea42f/title executed.   | 09:08:00.493 DEBUG - http://172.17.0.8:40001 lastCommand: DELETE - /session/3194bf9c-b9e8-447c-a066-ba7c156ea42f executing...   | 09:08:00.568 DEBUG - http://172.17.0.10:40003 lastCommand: POST - /session executed.   | 09:08:00.568 INFO - null-40003-r3qhw [bash, -c, start-video]   | 09:08:00.802 DEBUG - http://172.17.0.8:40001 lastCommand: DELETE - /session/3194bf9c-b9e8-447c-a066-ba7c156ea42f executed.   | 09:08:01.006 INFO - http://172.17.0.8:40001 AFTER_SESSION command received. Node should shutdown soon...   | 09:08:01.012 INFO - null-40001-lr4xk [bash, -c, stop-video]   | 09:08:02.519 INFO - null-40003-r3qhw video-rec: started   | Waiting for ffmpeg video recording to start...   | ..Videos at /home/seluser/videos/vid_firefox_40003.mp4 started to be recorded! (wait-video-rec.sh)   |     | 09:08:12.806 INFO - null-40001-lr4xk video-rec: stopped   | Waiting for video to stop recording...   | video-rec STOPPED Aug 18 09:08 AM   | Done waiting for video recording to stop.   | Video recording stopped   | --LOG 09:08:12:792914917 -- DEBUG: video-rec-stdout.log ----   | Input #0, x11grab, from ':8.0+0,0':   | Duration: N/A, start: 1503040077.375412, bitrate: N/A   | Stream #0:0: Video: rawvideo (BGR[0] / 0x524742), bgr0, 1900x1880, 10 fps, 10 tbr, 1000k tbn, 10 tbc   | No pixel format specified, yuv444p for H.264 encoding chosen.   | Use -pix_fmt yuv420p for compatibility with outdated media players.   | [libx264 @ 0x16bc0a0] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX LZCNT   | [libx264 @ 0x16bc0a0] profile High 4:4:4 Predictive, level 5.0, 4:4:4 8-bit   | [libx264 @ 0x16bc0a0] 264 - core 148 r2643 5c65704 - H.264/MPEG-4 AVC codec - Copyleft 2003-2015 - http://www.videolan.org/x264.html - options: cabac=0 ref=1 deblock=0:0:0 analyse=0:0 me=dia subme=0 psy=0 mixed_ref=0 me_range=16 chroma_me=1 trellis=0 8x8dct=0 cqm=0 deadzone=21,11 fast_pskip=0 chroma_qp_offset=0 threads=3 lookahead_threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=0 weightp=0 keyint=250 keyint_min=10 scenecut=0 intra_refresh=0 rc=cqp mbtree=0 qp=0   | Output #0, matroska, to '/home/seluser/videos/vid_chrome_40001.mkv':   | Metadata:   | encoder : Lavf56.40.101   | Stream #0:0: Video: h264 (libx264) (H264 / 0x34363248), yuv444p, 1900x1880, q=-1--1, 10 fps, 1k tbn, 10 tbc   | Metadata:   | encoder : Lavc56.60.100 libx264   | Stream mapping:   | Stream #0:0 -> #0:0 (rawvideo (native) -> h264 (libx264))   | Press [q] to stop, [?] for help   | frame= 4 fps=0.0 q=0.0 size= 1kB time=00:00:00.00 bitrate=N/A frame= 6 fps=4.0 q=0.0 size= 72kB time=00:00:00.40 bitrate=1466.7kbits/s frame= 7 fps=3.2 q=0.0 size= 91kB time=00:00:00.60 bitrate=1238.0kbits/s frame= 10 fps=3.5 q=0.0 size= 171kB time=00:00:01.70 bitrate= 823.4kbits/s frame= 14 fps=4.0 q=0.0 size= 255kB time=00:00:03.00 bitrate= 695.5kbits/s frame= 18 fps=4.4 q=0.0 size= 347kB time=00:00:03.70 bitrate= 767.8kbits/s frame= 20 fps=4.3 q=0.0 size= 379kB time=00:00:04.00 bitrate= 777.2kbits/s --LOG 09:08:02:969151171 Trapped SIGTERM or SIGINT so shutting down ffmpeg gracefully...   | --LOG 09:08:02:978688809 Will kill VID_TOOL_PID=885 ...   | frame= 23 fps=4.3 q=0.0 size= 428kB time=00:00:04.40 bitrate= 797.4kbits/s frame= 23 fps=3.8 q=-1.0 Lsize= 494kB time=00:00:05.50 bitrate= 735.2kbits/s   | video:493kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.185926%   | [libx264 @ 0x16bc0a0] frame I:1 Avg QP: 0.00 size: 43663   | [libx264 @ 0x16bc0a0] frame P:22 Avg QP: 0.00 size: 20924   | [libx264 @ 0x16bc0a0] mb I I16..4: 100.0% 0.0% 0.0%   | [libx264 @ 0x16bc0a0] mb P I16..4: 93.7% 0.0% 0.0% P16..4: 0.0% 0.0% 0.0% 0.0% 0.0% skip: 6.3%   | [libx264 @ 0x16bc0a0] coded y,u,v intra: 0.2% 0.1% 0.1% inter: 0.2% 0.1% 0.1%   | [libx264 @ 0x16bc0a0] i16 v,h,dc,p: 100% 0% 0% 0%   | [libx264 @ 0x16bc0a0] kb/s:719.97   | Exiting normally, received signal 2.   | --LOG 09:08:04:046852089 Tried to kill -SIGTERM VID_TOOL_PID=885   | --LOG 09:08:04:051086636 Waiting up to 6s for VID_TOOL_PID=885 to end with SIGTERM...   | --LOG 09:08:04:170944889 wait_pid successfully managed to SIGTERM:VID_TOOL_PID=885 within less than 6s   | --LOG 09:08:04:175530667 Will try to fix the videos...   | --LOG 09:08:04:763393642 Fixing perms for /home/seluser/videos/vid_chrome_40001.mkv*   | --LOG 09:08:04:943219795 Changing video encoding from mkv to mp4...   | --LOG 09:08:11:864222159 Conversion from mkv to mp4 succeeded!   | --LOG 09:08:11:932932804 Cleaning up /home/seluser/videos/vid_chrome_40001.mkv ...   | --LOG 09:08:11:963822728 Optimizing /home/seluser/videos/vid_chrome_40001.mp4 for HTTP streaming...   | --LOG 09:08:12:074552306 Succeeded to mp4box_retry.sh within 8s   | --LOG 09:08:12:161688992 ffmpeg shutdown complete.   | --LOG 09:08:12:799774372 -- DEBUG: video-rec-stderr.log ----   | libavformat 56. 40.101 / 56. 40.101   | libavdevice 56. 4.100 / 56. 4.100   | libavfilter 5. 40.101 / 5. 40.101   | libavresample 2. 1. 0 / 2. 1. 0   | libswscale 3. 1.101 / 3. 1.101   | libswresample 1. 2.101 / 1. 2.101   | libpostproc 53. 3.100 / 53. 3.100   | Input #0, matroska,webm, from '/home/seluser/videos/vid_chrome_40001.mkv':   | Metadata:   | ENCODER : Lavf56.40.101   | Duration: 00:00:05.50, start: 0.000000, bitrate: 735 kb/s   | Stream #0:0: Video: h264 (High 4:4:4 Predictive), yuv444p, 1900x1880, SAR 1:1 DAR 95:94, 10 fps, 10 tbr, 1k tbn, 20 tbc (default)   | Metadata:   | ENCODER : Lavc56.60.100 libx264   | DURATION : 00:00:05.500000000   | No pixel format specified, yuv444p for H.264 encoding chosen.   | Use -pix_fmt yuv420p for compatibility with outdated media players.   | [libx264 @ 0xadb640] using SAR=1/1   | [libx264 @ 0xadb640] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX LZCNT   | [libx264 @ 0xadb640] profile High 4:4:4 Predictive, level 5.0, 4:4:4 8-bit   | [libx264 @ 0xadb640] 264 - core 148 r2643 5c65704 - H.264/MPEG-4 AVC codec - Copyleft 2003-2015 - http://www.videolan.org/x264.html - options: cabac=0 ref=1 deblock=0:0:0 analyse=0:0 me=dia subme=0 psy=0 mixed_ref=0 me_range=16 chroma_me=1 trellis=0 8x8dct=0 cqm=0 deadzone=21,11 fast_pskip=0 chroma_qp_offset=0 threads=3 lookahead_threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=0 weightp=0 keyint=250 keyint_min=10 scenecut=0 intra_refresh=0 rc=cqp mbtree=0 qp=0   | Output #0, mp4, to '/home/seluser/videos/vid_chrome_40001.mp4':   | Metadata:   | encoder : Lavf56.40.101   | Stream #0:0: Video: h264 (libx264) ([33][0][0][0] / 0x0021), yuv444p, 1900x1880 [SAR 1:1 DAR 95:94], q=-1--1, 10 fps, 10240 tbn, 10 tbc (default)   | Metadata:   | DURATION : 00:00:05.500000000   | encoder : Lavc56.60.100 libx264   | Stream mapping:   | Stream #0:0 -> #0:0 (h264 (native) -> h264 (libx264))   | Press [q] to stop, [?] for help   | frame= 8 fps=0.0 q=0.0 size= 104kB time=00:00:00.40 bitrate=2120.0kbits/s dup=4 drop=0 frame= 24 fps= 15 q=0.0 size= 394kB time=00:00:02.00 bitrate=1614.8kbits/s dup=17 drop=0 frame= 25 fps= 11 q=0.0 size= 410kB time=00:00:02.10 bitrate=1599.7kbits/s dup=17 drop=0 frame= 31 fps= 11 q=0.0 size= 526kB time=00:00:02.70 bitrate=1595.3kbits/s dup=20 drop=0 frame= 37 fps=9.7 q=0.0 size= 620kB time=00:00:03.30 bitrate=1539.6kbits/s dup=23 drop=0 frame= 40 fps=9.3 q=0.0 size= 697kB time=00:00:03.60 bitrate=1585.3kbits/s dup=24 drop=0 frame= 48 fps=9.8 q=0.0 size= 827kB time=00:00:04.40 bitrate=1539.6kbits/s dup=28 drop=0 frame= 52 fps=9.4 q=0.0 size= 892kB time=00:00:04.80 bitrate=1522.6kbits/s dup=31 drop=0 frame= 55 fps=9.0 q=0.0 size= 941kB time=00:00:05.10 bitrate=1511.4kbits/s dup=32 drop=0 frame= 55 fps=8.7 q=-1.0 Lsize= 1007kB time=00:00:05.50 bitrate=1499.8kbits/s dup=32 drop=0   | video:1006kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.101061%   | [libx264 @ 0xadb640] frame I:1 Avg QP: 0.00 size: 43663   | [libx264 @ 0xadb640] frame P:54 Avg QP: 0.00 size: 18257   | [libx264 @ 0xadb640] mb I I16..4: 100.0% 0.0% 0.0%   | [libx264 @ 0xadb640] mb P I16..4: 93.3% 0.0% 0.0% P16..4: 0.0% 0.0% 0.0% 0.0% 0.0% skip: 6.6%   | [libx264 @ 0xadb640] coded y,u,v intra: 0.1% 0.0% 0.0% inter: 0.1% 0.0% 0.0%   | [libx264 @ 0xadb640] i16 v,h,dc,p: 100% 0% 0% 0%   | [libx264 @ 0xadb640] kb/s:1497.53   |     | 09:08:13.162 INFO - http://172.17.0.8:40001 Video file copied to: /home/seluser/videos/zalenium_76afe063-037d-4b22-8178-9313fe5353b5_chrome_LINUX_20170818090813.mp4   | 09:08:14.166 INFO - null-40001-lr4xk [bash, -c, transfer-logs.sh]   | 09:08:14.719 INFO - null-40001-lr4xk   | 09:08:15.887 DEBUG - Handling: GET /status   | 09:08:15.894 DEBUG - Executing: org.openqa.selenium.remote.server.handler.Status@65d00db4)   | 09:08:15.914 DEBUG - Done: org.openqa.selenium.remote.server.handler.Status@65d00db4   | 09:08:15.914 DEBUG - Finished: GET /status   | 09:08:16.906 INFO - http://172.17.0.8:40001 Logs copied to: /home/seluser/videos/logs/zalenium_76afe063-037d-4b22-8178-9313fe5353b5_chrome_LINUX_20170818090813   | 09:08:16.974 DEBUG - Test count: 1   | 09:08:16.974 DEBUG - Test count with video: 1   | 09:08:17.218 INFO - http://172.17.0.8:40001 Marking the node as down because it was stopped after 1 tests.   | 09:08:17.219 INFO - http://172.17.0.8:40001 Marking the node as down because it was stopped after 1 tests.   | 09:08:17.220 WARN - Cleaning up stale test sessions on the unregistered node http://172.17.0.8:40001   | 09:08:17.228 DEBUG - http://172.17.0.8:40001 Error while sleeping the thread, stopping thread execution.   | java.lang.InterruptedException: sleep interrupted   | at java.lang.Thread.sleep(Native Method)   | at de.zalando.ep.zalenium.proxy.DockerSeleniumRemoteProxy$DockerSeleniumNodePoller.run(DockerSeleniumRemoteProxy.java:468)   | 09:08:45.640 DEBUG - getting the following registration request : {"class":"org.openqa.grid.common.RegistrationRequest","name":null,"description":null,"configuration":{"hubHost":"172.17.0.7","hubPort":4445,"id":"http://172.17.0.8:40004","capabilities":[{"firefox_binary":"/home/seluser/firefox-for-sel-3/firefox","seleniumProtocol":"WebDriver","acceptSslCerts":true,"screen-resolution":"1900x1880","tz":"Europe/Berlin","browserName":"firefox","maxInstances":1,"screenResolution":"1900x1880","resolution":"1900x1880","version":"52.0.2","platform":"LINUX"}],"downPollingLimit":2,"hub":"http://172.17.0.7:4445/grid/register","nodePolling":97000,"nodeStatusCheckTimeout":5000,"proxy":"de.zalando.ep.zalenium.proxy.DockerSeleniumRemoteProxy","register":true,"registerCycle":0,"unregisterIfStillDownAfter":2500,"cleanUpCycle":5000,"custom":{},"host":"172.17.0.8","maxSession":1,"servlets":[],"withoutServlets":[],"browserTimeout":0,"debug":false,"jettyMaxThreads":null,"log":null,"port":40004,"role":"node","timeout":1800}}   | 09:08:45.675 DEBUG - Using class de.zalando.ep.zalenium.proxy.DockerSeleniumRemoteProxy   | 09:08:46.032 DEBUG - starting cleanup thread   | 09:08:46.162 DEBUG - cleanup thread starting...   | 09:08:46.789 INFO - Registered a node http://172.17.0.8:40004   | 09:08:46.796 DEBUG - proxy added http://172.17.0.8:40004

mojsha commented 7 years ago

And this is how many nodes it created:

image

pearj commented 7 years ago

Oh null doesn't look right. That should be the name of the app label on the zalenium pod. Can you paste the yaml for the zalenium deployment config?

Did you actually run a test to get that many containers? Or did it start that many just from the beginning?

pearj commented 7 years ago

Can you paste the oc run command you used to start zalenium as well?

Also, does your test open and close the browser as it goes through the test? Because each time you do that, it starts a new container to do that. #135 is going to reuse containers instead of start new ones, but I haven't worked on that issue for a while.

mojsha commented 7 years ago

YAML:

apiVersion: v1 kind: DeploymentConfig metadata: creationTimestamp: '2017-08-24T08:31:53Z' generation: 2 labels: app: zalenium name: zalenium namespace: myproject resourceVersion: '2010' selfLink: /oapi/v1/namespaces/myproject/deploymentconfigs/zalenium uid: ae6c48c3-88a6-11e7-89c0-32bc28538335 spec: replicas: 1 selector: app: zalenium strategy: activeDeadlineSeconds: 21600 resources: {} rollingParams: intervalSeconds: 1 maxSurge: 25% maxUnavailable: 25% timeoutSeconds: 600 updatePeriodSeconds: 1 type: Rolling template: metadata: creationTimestamp: null labels: app: zalenium spec: containers:

I run a test that should start two containers - it ends up opening tons of containers.

This is the OC command:

oc run zalenium --image=dosel/zalenium --env="ZALENIUM_KUBERNETES_CPU_REQUEST=250m" --env="ZALENIUM_KUBERNETES_CPU_LIMIT=500m" --env="ZALENIUM_KUBERNETES_MEMORY_REQUEST=1Gi" --overrides="{\"spec\": {\"template\": {\"spec\": {\"serviceAccount\": \"zalenium\"}}}}" -l app=zalenium --port=4444 -- start --firefoxContainers 0 --chromeContainers 2

I can make it work once - right after installing Minishift although even then it spins up alot of containers.

Afterwards, it gives me errors like this:

T E S T S

Running selenium.SeleniumStandaloneServerTest aug. 24, 2017 10:33:39 AM org.openqa.selenium.remote.ProtocolHandshake createSession INFO: Detected dialect: OSS Tests run: 2, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 66.843 sec <<< FAILURE! executeChrome(eu.nets.selenium.SeleniumStandaloneServerTest) Time elapsed: 30.011 sec <<< ERROR! org.openqa.selenium.WebDriverException: Unable to parse remote response:

504 Gateway Time-out

The server didn't respond in time.

Build info: version: '3.4.0', revision: 'unknown', time: 'unknown' System info: host: '', ip: '192.168.99.1', os.name: 'Windows 10', os.arch: 'amd64', os.version: '10.0', java.version: '1.8.0_121' Driver info: driver.version: RemoteWebDriver

Java code:

public class SeleniumStandaloneServerTest {

@Test
public void executeFirefoxDriver() throws MalformedURLException {
    this.execute(DesiredCapabilities.firefox());
}

@Test
public void executeChrome() throws MalformedURLException {
    this.execute(DesiredCapabilities.chrome());
}

private void execute(final DesiredCapabilities capability) throws MalformedURLException {
    WebDriver driver = new RemoteWebDriver(
            new URL("http://zalenium-myproject.192.168.99.100.nip.io/wd/hub"), capability
    );
    driver.get("https://www.google.com/");
    assertThat(
            driver.getTitle(),
            is("Google")
    );

driver.close(); driver.quit(); } }

mojsha commented 7 years ago

Any ideas on why this might be? @diemol @pearj

diemol commented 7 years ago

There is some code @pearj pushed for #135 and I see that it might fix the issue. I need to test it. We plan to release latest next Monday-Tuesday.

diemol commented 7 years ago

I was just able to reproduce it and found the reason, I'll make some changes to the code and release this next week.

mojsha commented 7 years ago

Sounds really good! Looking forward to that...

diemol commented 7 years ago

This fix has been released https://github.com/zalando/zalenium/releases/tag/3.3.1s

pearj commented 7 years ago

Nice pick up @diemol. Looks like I needed to check the outstanding // TODO's I had. I'll refactor getContainerIp it a bit tonight and push up a PR. Because there may be some circumstances where it could hit the wrong container.

diemol commented 7 years ago

Cool, thanks for checking @pearj!