umihico / docker-selenium-lambda

The simplest demo of chrome automation by python and selenium in AWS Lambda
MIT License
504 stars 119 forks source link

[ERROR] WebDriverException #177

Closed Turkmen1Mehmet closed 1 year ago

Turkmen1Mehmet commented 1 year ago

I'm working on the Mac m1 contract via your dockerfile. The error I get is as follows:

22 Jun 2023 16:02:07,721 [INFO] (rapid) exec '/var/runtime/bootstrap' (cwd=/var/task, handler=) START RequestId: 66664e2d-dc2f-46df-baad-476fbd5a843f Version: $LATEST 22 Jun 2023 16:02:13,932 [INFO] (rapid) extensionsDisabledByLayer(/opt/disable-extensions-jwigqn8j) -> stat /opt/disable-extensions-jwigqn8j: no such file or directory 22 Jun 2023 16:02:13,933 [INFO] (rapid) Configuring and starting Operator Domain 22 Jun 2023 16:02:13,933 [INFO] (rapid) Starting runtime domain 22 Jun 2023 16:02:13,933 [WARNING] (rapid) Cannot list external agents error=open /opt/extensions: no such file or directory 22 Jun 2023 16:02:13,933 [INFO] (rapid) Starting runtime without AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_SESSION_TOKEN , Expected?: false Chromedriver version: None Chromedriver path: /opt/chromedriver/chromedriver [WARNING] 2023-06-22T16:02:16.833Z 81d9a614-0b57-4a69-b2d8-d2422513ada0 The version of chrome cannot be detected. Trying with latest driver version [ERROR] WebDriverException: Message: Service /root/.cache/selenium/chromedriver/linux64/114.0.5735.90/chromedriver unexpectedly exited. Status code was: 255 File "/var/lang/lib/python3.10/site-packages/selenium/webdriver/chrome/webdriv File "/var/lang/lib/python3.10/site-packages/selenium/webdriver/chromium/webdr File " /var/lang/lib/python3.10/site-packages/selenium/webdriver/common/service File "/var/lang/lib/python3.10/site-packages/selenium/webdriver/common/service Raise WebDriverException(f "Service {self._path} unexpectedly exited. Status code was: {return_code}") END RequestId: 81d9a614-0b57-4a69-b2d8-d2422513ada0 REPORT RequestId: 81d9a614-0b57-4a69-b2d8-d2422513ada0 Init Duration: 0.30 ms Duration: 3412.10 ms Billed Duration: 3413 ms Memory Size: 3008 MB Max Memory Used: 3008 MB

I changed the version links in mac but I get the same error. What could be the reason for this?

umihico commented 1 year ago

@Turkmen1Mehmet

[ERROR] WebDriverException: Message: Service /root/.cache/selenium/chromedriver/linux64/114.0.5735.90/chromedriver unexpectedly exited. Status code was: 255

It seems you executed this code locally? If then, please note how you executed exactly. I need to reproduce by pasting the commands. I also use M1 Mac, but not tried locally yet.

Turkmen1Mehmet commented 1 year ago

@umihico

First of all, I connected to my own user with AWS Access key and AWS Secret access key via terminal and gave all necessary permissions for AWS. Then I edited the code I wanted to run through your main.py. Then I used the following code to rebuild the Docker image in my terminal in the relevant directory and deploy it to AWS Lambda.

docker build -t my-selenium-lambda .

Convert the Docker image to a Docker container and I used the following code to test it.

docker run -p 8080:8080 my-selenium-lambda

I ran the code using a trigger the trigger code

curl -XPOST "http://localhost:8080/2015-03-31/functions/function/invocations" -d '{}'

I'm doing all of these on the terminal and the error I get is the error I mentioned. Thank you so much for your immediate return.

umihico commented 1 year ago

@Turkmen1Mehmet

I could reproduce.

You can tell docker engine that you prefer to use amd64 arch, by something like

export DOCKER_DEFAULT_PLATFORM=linux/amd64

either

FROM --platform=linux/amd64 public.ecr.aws/lambda/python@sha256:70b056503bffcc49eefe22897782393a984efa04be670dfd7aec1351bea6717d as build

Then your curl takes time, and the scraping will be done. The curl command returned empty string for me, so I inserted raise Exception(chrome.find_element(by=By.XPATH, value="//html").text) to make sure that the process was properly executed inside.

Turkmen1Mehmet commented 1 year ago

@umihico

FROM --platform=linux/amd64 public.ecr.aws/lambda/python@sha256:70b056503bffcc49eefe22897782393a984efa04be670dfd7aec1351bea6717d as build

ENV DOCKER_DEFAULT_PLATFORM=linux/amd64

I added them to the dockerfile and finally got rid of this error, thank you very much, this time I encountered a different error, even getting a different error pleased me :) My new mistake: 23 Jun 2023 13:37:38,475 [WARNING] (rapid) Reset initiated: Timeout 23 Jun 2023 13:37:38,537 [INFO] (rapid) Sending SIGKILL to runtime-1(18). 23 Jun 2023 13:37:38,625 [INFO] (rapid) Stopping runtime domain 23 Jun 2023 13:37:38,634 [INFO] (rapid) Waiting for runtime domain processes termination 23 Jun 2023 13:37:38,641 [INFO] (rapid) Stopping operator domain 23 Jun 2023 13:37:38,644 [INFO] (rapid) Starting runtime domain END RequestId: 907401af-490c-4fe7-8b8c-84c8f699562b REPORT RequestId: 907401af-490c-4fe7-8b8c-84c8f699562b Init Duration: 4.43 ms Duration: 300000.00 ms Billed Duration: 300000 ms Memory Size: 3008 MB Max Memory Used: 3008 MB

I'll think about it a little

umihico commented 1 year ago

It seems no errors, or timeout? Anyway, I'll close this issue to keep one topic per one issue. Please post another issue when you want.