ConnectionError upon using docker compose

[devinat1]

On Ubuntu 22.04.4 LTS running Docker version 26.1.2, build 211e74b, I am getting the following error upon opening the frontend web app through http://localhost:8501/ with command docker compose up -d and with skyvern version 0.1.1 - 2024-03-17:

ConnectionError: HTTPConnectionPool(host='127.0.0.1', port=8000): Max retries exceeded with url: /api/v1/internal/tasks?page=1&page_size=15 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f4c0147f590>: Failed to establish a new connection: [Errno 111] Connection refused'))
Traceback:

File "/usr/local/lib/python3.11/site-packages/streamlit/runtime/scriptrunner/script_runner.py", line 584, in _run_script
    exec(code, module.__dict__)
File "/app/streamlit_app/visualizer/streamlit.py", line 258, in <module>
    tasks_response = repository.get_tasks(task_page_number)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/streamlit_app/visualizer/repository.py", line 15, in get_tasks
    return self.client.get_agent_tasks(page=page, page_size=page_size)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/streamlit_app/visualizer/api.py", line 59, in get_agent_tasks
    response = requests.get(url, params=params, headers=headers)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/requests/api.py", line 73, in get
    return request("get", url, params=params, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/requests/api.py", line 59, in request
    return session.request(method=method, url=url, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/requests/sessions.py", line 589, in request
    resp = self.send(prep, **send_kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/requests/sessions.py", line 703, in send
    r = adapter.send(request, **kwargs)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/requests/adapters.py", line 519, in send
    raise ConnectionError(e, request=request)

The only change I made to the docker compose file was to place in an api key.
Running works locally through ./setup.sh, but not through docker compose.

[suchintan]

Can you run docker ps? What services are online?

[devinat1]

CONTAINER ID   IMAGE                                    COMMAND                  CREATED          STATUS                    PORTS                    NAMES
b2e64e472661   public.ecr.aws/t6d4b5t4/skyvern:latest   "/bin/bash entrypoin…"   16 seconds ago   Up 10 seconds             0.0.0.0:8501->8501/tcp   skyvern-streamlit-1
b469b2ab6678   public.ecr.aws/t6d4b5t4/skyvern:latest   "/bin/bash /app/entr…"   16 seconds ago   Up 10 seconds                                      skyvern-skyvern-1
6d62fa05a9e2   postgres:14-alpine                       "docker-entrypoint.s…"   17 seconds ago   Up 16 seconds (healthy)   5432/tcp                 skyvern-postgres-1
1205de006e32   postgres:14                              "docker-entrypoint.s…"   5 hours ago      Up 5 hours                0.0.0.0:5432->5432/tcp   postgresql-container

[suchintan]

It may be because Skyvern isn't exposed on port 8000 cc @LawyZheng

[LawyZheng]

Yes, we don't expose the 8000 port(Skyvern service) to the local network in docker-compose by default. In this case, you have two solutions:

1. expose the 8000 port in the docker compose. https://github.com/Skyvern-AI/skyvern/blob/f2f9122799a659ade26b0c4e02b7602045ab9d64/docker-compose.yml#L25-L27
2. change 127.0.0.1:8000 to skyvern:8000 in the .streamlit/secrets.toml

BTW: If you want to use docker Skyvern and local Skyvern together, you'd better confirm you're using the same DB for these two services. Because I saw you had 2 DBs running.

[suchintan]

@devinat1 LEt me know if that solves your issue or not!

[devinat1]

I closed the docker container for the postgres I was running locally, and I received the same error after docker compose up -d.

[LawyZheng]

I closed the docker container for the postgres I was running locally, and I received the same error after docker compose up -d.

Not the postgres container matters. What matters is that you need to expose the 8000 port in the docker-compose file.

[devinat1]

I tried following the steps for exposing the port but same issue. This is what I see:

CONTAINER ID   IMAGE                                    COMMAND                  CREATED              STATUS                    PORTS                    NAMES
f39e789d3e56   public.ecr.aws/t6d4b5t4/skyvern:latest   "/bin/bash entrypoin…"   About a minute ago   Up About a minute         0.0.0.0:8501->8501/tcp   skyvern-streamlit-1
69b4a5d934d1   public.ecr.aws/t6d4b5t4/skyvern:latest   "/bin/bash /app/entr…"   About a minute ago   Up About a minute         0.0.0.0:8000->8000/tcp   skyvern-skyvern-1
6d62fa05a9e2   postgres:14-alpine                       "docker-entrypoint.s…"   49 minutes ago       Up 49 minutes (healthy)   5432/tcp                 skyvern-postgres-1

[LawyZheng]

Try to restart all the containers, and refresh the page. If the error persists, can you share the the information and the screenshot you got?

[devinat1]

I get the following error when I click execute within the streamlit frontend:

ConnectionError: HTTPConnectionPool(host='skyvern', port=80): Max retries exceeded with url: /api/v1/tasks (Caused by NameResolutionError("<urllib3.connection.HTTPConnection object at 0x7f7c305c32d0>: Failed to resolve 'skyvern' ([Errno -2] Name or service not known)"))
Traceback:

File "/usr/local/lib/python3.11/site-packages/streamlit/runtime/scriptrunner/script_runner.py", line 584, in _run_script
    exec(code, module.__dict__)
File "/app/streamlit_app/visualizer/streamlit.py", line 202, in <module>
    task_id = client.create_task(task_request_body)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/streamlit_app/visualizer/api.py", line 32, in create_task
    response = requests.post(url, headers=headers, data=json.dumps(payload))
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/requests/api.py", line 115, in post
    return request("post", url, data=data, json=json, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/requests/api.py", line 59, in request
    return session.request(method=method, url=url, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/requests/sessions.py", line 589, in request
    resp = self.send(prep, **send_kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/requests/sessions.py", line 703, in send
    r = adapter.send(request, **kwargs)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/requests/adapters.py", line 519, in send
    raise ConnectionError(e, request=request)

The skyvern container keeps restarting in a loop:

This is a snippet of logs within the skyvern docker container:

2024-05-14 23:55:24 INFO  [alembic.runtime.migration] Context impl PostgresqlImpl.
2024-05-14 23:55:24 INFO  [alembic.runtime.migration] Will assume transactional DDL.
2024-05-14 23:55:25 INFO  [alembic.runtime.migration] Context impl PostgresqlImpl.
2024-05-14 23:55:25 INFO  [alembic.runtime.migration] Will assume transactional DDL.
2024-05-14 23:55:26 xvfb-run: error: Xvfb failed to start
2024-05-14 23:55:27 INFO  [alembic.runtime.migration] Context impl PostgresqlImpl.
2024-05-14 23:55:27 INFO  [alembic.runtime.migration] Will assume transactional DDL.
2024-05-14 23:55:28 INFO  [alembic.runtime.migration] Context impl PostgresqlImpl.
2024-05-14 23:55:28 INFO  [alembic.runtime.migration] Will assume transactional DDL.
2024-05-14 23:55:29 xvfb-run: error: Xvfb failed to start
2024-05-14 23:55:31 INFO  [alembic.runtime.migration] Context impl PostgresqlImpl.
2024-05-14 23:55:31 INFO  [alembic.runtime.migration] Will assume transactional DDL.
2024-05-14 23:55:31 INFO  [alembic.runtime.migration] Context impl PostgresqlImpl.
2024-05-14 23:55:31 INFO  [alembic.runtime.migration] Will assume transactional DDL.
2024-05-14 23:55:32 xvfb-run: error: Xvfb failed to start
2024-05-14 23:55:34 INFO  [alembic.runtime.migration] Context impl PostgresqlImpl.
2024-05-14 23:55:34 INFO  [alembic.runtime.migration] Will assume transactional DDL.
2024-05-14 23:55:34 INFO  [alembic.runtime.migration] Context impl PostgresqlImpl.
2024-05-14 23:55:34 INFO  [alembic.runtime.migration] Will assume transactional DDL.
2024-05-14 23:55:35 xvfb-run: error: Xvfb failed to start
2024-05-14 23:38:16 Alembic mode:  online
2024-05-14 23:38:17 Alembic mode:  online
2024-05-14 23:38:17 No new upgrade operations detected.
2024-05-14 23:38:17 2024-05-15T03:38:17.635460Z [info     ] Agent server starting.         host=0.0.0.0 port=8000
2024-05-14 23:38:21 2024-05-15 03:38:21 [info     ] Registering LLM config         llm_key=OPENAI_GPT4_TURBO
2024-05-14 23:38:21 2024-05-15 03:38:21 [info     ] Registering LLM config         llm_key=OPENAI_GPT4V
2024-05-14 23:38:21 2024-05-15T03:38:21.590363Z [info     ] Initializing ForgeAgent        browser_action_timeout_ms=5000 browser_type=chromium-headful debug_mode=False env=local execute_all_steps=True long_running_task_warning_ratio=0.95 max_scraping_retries=0 max_steps_per_run=75 video_path=/data/videos
2024-05-14 23:38:21 2024-05-15T03:38:21.712460Z [info     ] Starting the skyvern scheduler.
2024-05-14 23:38:21 2024-05-15T03:38:21.713404Z [info     ] Server startup complete. Skyvern is now online
2024-05-14 23:55:24 Alembic mode:  online
2024-05-14 23:55:26 Alembic mode:  online
2024-05-14 23:55:26 No new upgrade operations detected.
2024-05-14 23:55:27 Alembic mode:  online

[LawyZheng]

Can you show your docker-compose file and .streamlit/secrets.toml file here? Remember to remove every secret key(OpenAI key, or your local creds) before sharing.

[devinat1]

[skyvern]
configs = [    {"env" = "local", "host" = "http://skyvern/api/v1", "orgs" = [{name="Skyvern", cred="REDACTED"}]}
]

[LawyZheng]

“host” = "http://skyvern:8000/api/v1" as I mentioned above, not "host" = "http://skyvern/api/v1"

[devinat1]

Ok I will take a look at this tomorrow thanks for the quick replies!