deploy app to the cloud

[bbelderbos]

Heroku, fly, render, we can decide what makes most sense ... (this or another app btw)

[vinostroud]

I've used fly.io & have an account if that works. I think the issue I've run into previously is going from environment > Docker > deploy on fly. That may have been because I used pip to install dependencies but poetry to manage the environment.

[bbelderbos]

OK that works. I need to double-check docs if Docker makes most sense. We also have one or two clinics on hosting on fly that will help you here.

[vinostroud]

OK, I'm a little stuck going from Docker -> web deployment. Maybe you can see where the problem is?

When I do Docker run I get these URLs:

  Local URL: http://localhost:8501
  Network URL: http://172.17.0.2:8501
  External URL: http://100.34.135.248:8501

localhost works, but the network and external urls do not. As a consequence, my fly.io is not launching correctly. i've tried to troubleshoot using chatgpt, i've looked at my pdm apps, and am stuck here.

let me share the following: Dockerfile current and docker-compose

Dockerfile

# Use an official Python runtime as a parent image
FROM python:3.12

# Set the working directory in the container
WORKDIR /app

# Copy the current directory contents into the container at /app
COPY . /app/
COPY pyproject.toml pyproject.toml
COPY poetry.lock poetry.lock

# Install Poetry
RUN pip install --no-cache-dir poetry && \
    poetry config virtualenvs.create false && \
    poetry install --no-interaction --no-ansi

# Copy the current directory contents into the container at /app
COPY . /app

# Expose the port that the app runs on
EXPOSE 8501

# Define environment variable
ENV NAME nfl_analytics

# Run the application
CMD ["streamlit", "run", "src/app_fe.py"]

docker-compose.yml

version: '3.8'

services:
  app:
    container_name: nfl_app
    build: .
    command: "streamlit run --server.port 8501 --server.enableCORS false app.py"
    ports:
      - "8501:8501"
    networks:
      - frontend

[vinostroud]

here are my logs from flyctl in case helpful:

(nfl data analysis-py3.12) kevinstrouse@Kevins-Mac-mini nfl_analytics % flyctl logs                           
2024-08-02T13:07:16Z app[9185420c444908] ewr [info] INFO [fly api proxy] listening at /.fly/api
2024-08-02T13:07:16Z app[9185420c444908] ewr [info]2024/08/02 13:07:16 INFO SSH listening listen_address=[fdaa:3:d72d:a7b:95:592f:cf2:2]:22 dns_server=[fdaa::3]:53
2024-08-02T13:07:16Z runner[9185420c444908] ewr [info]Machine started in 859ms
error.message="instance refused connection. is your app listening on 0.0.0.0:8501? make sure it is not only listening on 127.0.0.1 (hint: look at your startup logs, servers often print the address they are listening on)" 2024-08-02T13:07:16Z proxy[683ddd9f6dd6e8] ewr [error]request.method="GET" request.id="01J49K76BP63ZPPJGK4MMQZ2TV-lga" 
2024-08-02T13:07:17Z app[683ddd9f6dd6e8] ewr [info]Usage: streamlit [OPTIONS] COMMAND [ARGS]...
2024-08-02T13:07:17Z app[683ddd9f6dd6e8] ewr [info]Try 'streamlit --help' for help.
2024-08-02T13:07:17Z app[683ddd9f6dd6e8] ewr [info]Error: No such command 'src/app_fe.py'.
2024-08-02T13:07:17Z app[683ddd9f6dd6e8] ewr [info] INFO Main child exited normally with code: 2
2024-08-02T13:07:17Z app[683ddd9f6dd6e8] ewr [info] INFO Starting clean up.
2024-08-02T13:07:17Z app[683ddd9f6dd6e8] ewr [info] WARN could not unmount /rootfs: EINVAL: Invalid argument
2024-08-02T13:07:17Z app[683ddd9f6dd6e8] ewr [info][    2.718232] reboot: Restarting system
2024-08-02T13:07:17Z runner[683ddd9f6dd6e8] ewr [info]machine did not have a restart policy, defaulting to restart
2024-08-02T13:07:18Z app[9185420c444908] ewr [info]Usage: streamlit [OPTIONS] COMMAND [ARGS]...
2024-08-02T13:07:18Z app[9185420c444908] ewr [info]Try 'streamlit --help' for help.
2024-08-02T13:07:18Z app[9185420c444908] ewr [info]Error: No such command 'src/app_fe.py'.
2024-08-02T13:07:18Z app[9185420c444908] ewr [info] INFO Main child exited normally with code: 2
2024-08-02T13:07:18Z app[9185420c444908] ewr [info] INFO Starting clean up.
2024-08-02T13:07:18Z app[9185420c444908] ewr [info] WARN could not unmount /rootfs: EINVAL: Invalid argument
2024-08-02T13:07:18Z app[9185420c444908] ewr [info][    2.669765] reboot: Restarting system
2024-08-02T13:07:18Z runner[9185420c444908] ewr [info]machine did not have a restart policy, defaulting to restart
2024-08-02T13:07:19Z app[9185420c444908] ewr [info]2024-08-02T13:07:19.344323889 [01J44MJD3WDMGH9BYNJ2TEDQB6:main] Running Firecracker v1.7.0
2024-08-02T13:07:19Z proxy[683ddd9f6dd6e8] ewr [info]Starting machine
error.message="failed to change machine state: machine still active, refusing to start" 2024-08-02T13:07:19Z proxy[683ddd9f6dd6e8] ewr [error]request.method="GET" request.id="01J49K76BP63ZPPJGK4MMQZ2TV-lga" 
2024-08-02T13:07:19Z proxy[9185420c444908] ewr [info]Starting machine
error.message="machines API returned an error: "machine still attempting to start"" 2024-08-02T13:07:19Z proxy[9185420c444908] ewr [error]request.method="GET" request.id="01J49K76BP63ZPPJGK4MMQZ2TV-lga" 
2024-08-02T13:07:19Z app[9185420c444908] ewr [info][    0.281737] PCI: Fatal: No config space access function found
2024-08-02T13:07:19Z app[683ddd9f6dd6e8] ewr [info]2024-08-02T13:07:19.922807117 [01J44MJDA5WM30SP86E048ZG4J:main] Running Firecracker v1.7.0
2024-08-02T13:07:20Z app[9185420c444908] ewr [info] INFO Starting init (commit: 3dd5d9e)...
2024-08-02T13:07:20Z app[9185420c444908] ewr [info] INFO Preparing to run: `poetry run streamlit src/app_fe.py` as root
2024-08-02T13:07:20Z app[9185420c444908] ewr [info] INFO [fly api proxy] listening at /.fly/api
2024-08-02T13:07:20Z app[9185420c444908] ewr [info]2024/08/02 13:07:20 INFO SSH listening listen_address=[fdaa:3:d72d:a7b:95:592f:cf2:2]:22 dns_server=[fdaa::3]:53
2024-08-02T13:07:20Z runner[9185420c444908] ewr [info]Machine started in 881ms
2024-08-02T13:07:20Z app[683ddd9f6dd6e8] ewr [info][    0.289256] PCI: Fatal: No config space access function found
2024-08-02T13:07:20Z app[683ddd9f6dd6e8] ewr [info] INFO Starting init (commit: 3dd5d9e)...
2024-08-02T13:07:20Z proxy[683ddd9f6dd6e8] ewr [info]Starting machine
error.message="machines API returned an error: "machine still attempting to start"" 2024-08-02T13:07:20Z proxy[683ddd9f6dd6e8] ewr [error]request.method="GET" request.id="01J49K76BP63ZPPJGK4MMQZ2TV-lga" 
error.message="instance refused connection. is your app listening on 0.0.0.0:8501? make sure it is not only listening on 127.0.0.1 (hint: look at your startup logs, servers often print the address they are listening on)" 2024-08-02T13:07:20Z proxy[9185420c444908] ewr [error]request.method="GET" request.id="01J49K76BP63ZPPJGK4MMQZ2TV-lga" 
2024-08-02T13:07:20Z app[683ddd9f6dd6e8] ewr [info] INFO Preparing to run: `poetry run streamlit src/app_fe.py` as root
2024-08-02T13:07:20Z app[683ddd9f6dd6e8] ewr [info] INFO [fly api proxy] listening at /.fly/api
2024-08-02T13:07:20Z app[683ddd9f6dd6e8] ewr [info]2024/08/02 13:07:20 INFO SSH listening listen_address=[fdaa:3:d72d:a7b:cb:e5c5:a9fd:2]:22 dns_server=[fdaa::3]:53
2024-08-02T13:07:20Z runner[683ddd9f6dd6e8] ewr [info]Machine started in 926ms
error.message="instance refused connection. is your app listening on 0.0.0.0:8501? make sure it is not only listening on 127.0.0.1 (hint: look at your startup logs, servers often print the address they are listening on)" 2024-08-02T13:07:21Z proxy[683ddd9f6dd6e8] ewr [error]request.method="GET" request.id="01J49K76BP63ZPPJGK4MMQZ2TV-lga" 
error.message="instance refused connection. is your app listening on 0.0.0.0:8501? make sure it is not only listening on 127.0.0.1 (hint: look at your startup logs, servers often print the address they are listening on)" 2024-08-02T13:07:21Z proxy[9185420c444908] ewr [error]request.method="GET" request.id="01J49K76BP63ZPPJGK4MMQZ2TV-lga" 
2024-08-02T13:07:21Z app[9185420c444908] ewr [info]Usage: streamlit [OPTIONS] COMMAND [ARGS]...
2024-08-02T13:07:21Z app[9185420c444908] ewr [info]Try 'streamlit --help' for help.
2024-08-02T13:07:21Z app[9185420c444908] ewr [info]Error: No such command 'src/app_fe.py'.
2024-08-02T13:07:22Z app[9185420c444908] ewr [info] INFO Main child exited normally with code: 2
2024-08-02T13:07:22Z app[9185420c444908] ewr [info] INFO Starting clean up.
2024-08-02T13:07:22Z app[9185420c444908] ewr [info] WARN could not unmount /rootfs: EINVAL: Invalid argument
2024-08-02T13:07:22Z app[9185420c444908] ewr [info][    2.720787] reboot: Restarting system
2024-08-02T13:07:22Z runner[9185420c444908] ewr [info]machine did not have a restart policy, defaulting to restart
error.message="instance refused connection. is your app listening on 0.0.0.0:8501? make sure it is not only listening on 127.0.0.1 (hint: look at your startup logs, servers often print the address they are listening on)" 2024-08-02T13:07:22Z proxy[683ddd9f6dd6e8] ewr [error]request.method="GET" request.id="01J49K76BP63ZPPJGK4MMQZ2TV-lga" 
2024-08-02T13:07:22Z app[683ddd9f6dd6e8] ewr [info]Usage: streamlit [OPTIONS] COMMAND [ARGS]...
2024-08-02T13:07:22Z app[683ddd9f6dd6e8] ewr [info]Try 'streamlit --help' for help.
2024-08-02T13:07:22Z app[683ddd9f6dd6e8] ewr [info]Error: No such command 'src/app_fe.py'.
2024-08-02T13:07:23Z app[9185420c444908] ewr [info]2024-08-02T13:07:23.069667537 [01J44MJD3WDMGH9BYNJ2TEDQB6:main] Running Firecracker v1.7.0
2024-08-02T13:07:23Z app[9185420c444908] ewr [info][    0.286464] PCI: Fatal: No config space access function found
2024-08-02T13:07:23Z proxy[9185420c444908] ewr [info]Starting machine
error.message="machines API returned an error: "machine still attempting to start"" 2024-08-02T13:07:23Z proxy[9185420c444908] ewr [error]request.method="GET" request.id="01J49K76BP63ZPPJGK4MMQZ2TV-lga" 
2024-08-02T13:07:23Z app[683ddd9f6dd6e8] ewr [info] INFO Main child exited normally with code: 2
2024-08-02T13:07:23Z app[683ddd9f6dd6e8] ewr [info] INFO Starting clean up.
2024-08-02T13:07:23Z app[683ddd9f6dd6e8] ewr [info] WARN could not unmount /rootfs: EINVAL: Invalid argument
2024-08-02T13:07:23Z app[683ddd9f6dd6e8] ewr [info][    3.728108] reboot: Restarting system
2024-08-02T13:07:23Z app[9185420c444908] ewr [info] INFO Starting init (commit: 3dd5d9e)...
2024-08-02T13:07:23Z app[9185420c444908] ewr [info] INFO Preparing to run: `poetry run streamlit src/app_fe.py` as root
2024-08-02T13:07:23Z app[9185420c444908] ewr [info] INFO [fly api proxy] listening at /.fly/api
2024-08-02T13:07:23Z app[9185420c444908] ewr [info]2024/08/02 13:07:23 INFO SSH listening listen_address=[fdaa:3:d72d:a7b:95:592f:cf2:2]:22 dns_server=[fdaa::3]:53
2024-08-02T13:07:23Z runner[9185420c444908] ewr [info]Machine started in 925ms
2024-08-02T13:07:23Z runner[683ddd9f6dd6e8] ewr [info]machine did not have a restart policy, defaulting to restart
error.message="instance refused connection. is your app listening on 0.0.0.0:8501? make sure it is not only listening on 127.0.0.1 (hint: look at your startup logs, servers often print the address they are listening on)" 2024-08-02T13:07:24Z proxy[9185420c444908] ewr [error]request.method="GET" request.id="01J49K76BP63ZPPJGK4MMQZ2TV-lga" 
2024-08-02T13:07:25Z app[9185420c444908] ewr [info]Usage: streamlit [OPTIONS] COMMAND [ARGS]...
2024-08-02T13:07:25Z app[9185420c444908] ewr [info]Try 'streamlit --help' for help.
2024-08-02T13:07:25Z app[9185420c444908] ewr [info]Error: No such command 'src/app_fe.py'.
error.message="instance refused connection. is your app listening on 0.0.0.0:8501? make sure it is not only listening on 127.0.0.1 (hint: look at your startup logs, servers often print the address they are listening on)" 2024-08-02T13:07:25Z proxy[9185420c444908] ewr [error]request.method="GET" request.id="01J49K76BP63ZPPJGK4MMQZ2TV-lga" 
2024-08-02T13:07:25Z proxy[683ddd9f6dd6e8] ewr [info]Starting machine
error.message="failed to change machine state: machine still active, refusing to start" 2024-08-02T13:07:25Z proxy[683ddd9f6dd6e8] ewr [error]request.method="GET" request.id="01J49K76BP63ZPPJGK4MMQZ2TV-lga" 
2024-08-02T13:07:25Z app[9185420c444908] ewr [info] INFO Main child exited normally with code: 2
2024-08-02T13:07:25Z app[9185420c444908] ewr [info] INFO Starting clean up.
2024-08-02T13:07:25Z app[9185420c444908] ewr [info] WARN could not unmount /rootfs: EINVAL: Invalid argument
2024-08-02T13:07:25Z app[9185420c444908] ewr [info][    2.686046] reboot: Restarting system
2024-08-02T13:07:26Z runner[9185420c444908] ewr [info]machine did not have a restart policy, defaulting to restart
2024-08-02T13:07:26Z proxy[683ddd9f6dd6e8] ewr [info]Starting machine
error.message="failed to change machine state: machine still active, refusing to start" 2024-08-02T13:07:26Z proxy[683ddd9f6dd6e8] ewr [error]request.method="GET" request.id="01J49K76BP63ZPPJGK4MMQZ2TV-lga" 
2024-08-02T13:07:27Z app[9185420c444908] ewr [info]2024-08-02T13:07:27.350969467 [01J44MJD3WDMGH9BYNJ2TEDQB6:main] Running Firecracker v1.7.0
2024-08-02T13:07:27Z proxy[9185420c444908] ewr [info]Starting machine
error.message="machines API returned an error: "machine still attempting to start"" 2024-08-02T13:07:27Z proxy[9185420c444908] ewr [error]request.method="GET" request.id="01J49K76BP63ZPPJGK4MMQZ2TV-lga" 
2024-08-02T13:07:27Z proxy[683ddd9f6dd6e8] ewr [info]Starting machine
error.message="failed to change machine state: machine still active, refusing to start" 2024-08-02T13:07:27Z proxy[683ddd9f6dd6e8] ewr [error]request.method="GET" request.id="01J49K76BP63ZPPJGK4MMQZ2TV-lga" 
2024-08-02T13:07:27Z app[9185420c444908] ewr [info][    0.286343] PCI: Fatal: No config space access function found
2024-08-02T13:07:28Z app[9185420c444908] ewr [info] INFO Starting init (commit: 3dd5d9e)...
2024-08-02T13:07:28Z app[9185420c444908] ewr [info] INFO Preparing to run: `poetry run streamlit src/app_fe.py` as root
2024-08-02T13:07:28Z app[9185420c444908] ewr [info] INFO [fly api proxy] listening at /.fly/api
2024-08-02T13:07:28Z runner[9185420c444908] ewr [info]Machine started in 865ms
2024-08-02T13:07:28Z app[9185420c444908] ewr [info]2024/08/02 13:07:28 INFO SSH listening listen_address=[fdaa:3:d72d:a7b:95:592f:cf2:2]:22 dns_server=[fdaa::3]:53
error.message="instance refused connection. is your app listening on 0.0.0.0:8501? make sure it is not only listening on 127.0.0.1 (hint: look at your startup logs, servers often print the address they are listening on)" 2024-08-02T13:07:28Z proxy[9185420c444908] ewr [error]request.method="GET" request.id="01J49K76BP63ZPPJGK4MMQZ2TV-lga" 
2024-08-02T13:07:28Z proxy[683ddd9f6dd6e8] ewr [info]Starting machine
error.message="failed to change machine state: machine still active, refusing to start" 2024-08-02T13:07:28Z proxy[683ddd9f6dd6e8] ewr [error]request.method="GET" request.id="01J49K76BP63ZPPJGK4MMQZ2TV-lga" 
2024-08-02T13:07:29Z proxy[683ddd9f6dd6e8] ewr [info]Starting machine
error.message="failed to change machine state: machine still active, refusing to start" 2024-08-02T13:07:29Z proxy[683ddd9f6dd6e8] ewr [error]request.method="GET" request.id="01J49K76BP63ZPPJGK4MMQZ2TV-lga" 
2024-08-02T13:07:30Z app[9185420c444908] ewr [info] INFO Starting clean up.
2024-08-02T13:07:30Z app[9185420c444908] ewr [info][    2.711012] reboot: Restarting system
2024-08-02T13:07:30Z runner[9185420c444908] ewr [info]machine did not have a restart policy, defaulting to restart
2024-08-02T13:07:30Z proxy[683ddd9f6dd6e8] ewr [info]Starting machine
error.message="failed to change machine state: machine still active, refusing to start" 2024-08-02T13:07:30Z proxy[683ddd9f6dd6e8] ewr [error]request.method="GET" request.id="01J49K76BP63ZPPJGK4MMQZ2TV-lga" 
2024-08-02T13:07:31Z app[683ddd9f6dd6e8] ewr [info]2024-08-02T13:07:31.259793284 [01J44MJDA5WM30SP86E048ZG4J:main] Running Firecracker v1.7.0
2024-08-02T13:07:31Z app[9185420c444908] ewr [info][    0.277837] PCI: Fatal: No config space access function found
error.message="machines API returned an error: "machine still attempting to start"" 2024-08-02T13:07:31Z proxy[9185420c444908] ewr [error]request.method="GET" request.id="01J49K76BP63ZPPJGK4MMQZ2TV-lga" 
2024-08-02T13:07:31Z app[9185420c444908] ewr [info] INFO Starting init (commit: 3dd5d9e)...
2024-08-02T13:07:31Z app[9185420c444908] ewr [info] INFO Preparing to run: `poetry run streamlit src/app_fe.py` as root
2024-08-02T13:07:31Z app[9185420c444908] ewr [info] INFO [fly api proxy] listening at /.fly/api
2024-08-02T13:07:38Z proxy[683ddd9f6dd6e8] ewr [info]Starting machine
2024-08-02T13:07:38Z proxy[683ddd9f6dd6e8] ewr [error][PM07] failed to change machine state: machine still active, refusing to start
2024-08-02T13:07:38Z proxy[9185420c444908] ewr [info]Starting machine
2024-08-02T13:07:38Z proxy[9185420c444908] ewr [error][PM07] failed to change machine state: machine still active, refusing to start
2024-08-02T13:07:42Z proxy[9185420c444908] ewr [error][PC01] instance refused connection. is your app listening on 0.0.0.0:8501? make sure it is not only listening on 127.0.0.1 (hint: look at your startup logs, servers often print the address they are listening on)
2024-08-02T13:07:42Z proxy[683ddd9f6dd6e8] ewr [info]Starting machine
2024-08-02T13:07:42Z proxy[683ddd9f6dd6e8] ewr [error][PM07] failed to change machine state: machine still active, refusing to start
2024-08-02T13:07:43Z app[9185420c444908] ewr [info]Usage: streamlit [OPTIONS] COMMAND [ARGS]...
2024-08-02T13:07:43Z app[9185420c444908] ewr [info]Try 'streamlit --help' for help.
2024-08-02T13:07:43Z app[9185420c444908] ewr [info]Error: No such command 'src/app_fe.py'.
2024-08-02T13:07:43Z proxy[683ddd9f6dd6e8] ewr [info]Starting machine
2024-08-02T13:07:43Z proxy[683ddd9f6dd6e8] ewr [error][PM07] failed to change machine state: machine still active, refusing to start
2024-08-02T13:07:44Z app[9185420c444908] ewr [info] INFO Main child exited normally with code: 2
2024-08-02T13:07:44Z app[9185420c444908] ewr [info] INFO Starting clean up.
2024-08-02T13:07:44Z app[9185420c444908] ewr [info] WARN could not unmount /rootfs: EINVAL: Invalid argument
2024-08-02T13:07:44Z app[9185420c444908] ewr [info][    3.720559] reboot: Restarting system
2024-08-02T13:07:44Z proxy[683ddd9f6dd6e8] ewr [info]Starting machine
2024-08-02T13:07:44Z proxy[683ddd9f6dd6e8] ewr [error][PM07] failed to change machine state: machine still active, refusing to start
2024-08-02T13:07:44Z runner[9185420c444908] ewr [info]machine did not have a restart policy, defaulting to restart
2024-08-02T13:07:45Z proxy[683ddd9f6dd6e8] ewr [info]Starting machine
2024-08-02T13:07:45Z proxy[683ddd9f6dd6e8] ewr [error][PM07] failed to change machine state: machine still active, refusing to start
2024-08-02T13:07:45Z proxy[9185420c444908] ewr [info]Starting machine
2024-08-02T13:07:45Z proxy[9185420c444908] ewr [error][PM07] failed to change machine state: machine still active, refusing to start
2024-08-02T13:07:46Z proxy[683ddd9f6dd6e8] ewr [info]Starting machine
2024-08-02T13:07:46Z proxy[683ddd9f6dd6e8] ewr [error][PM07] failed to change machine state: machine still active, refusing to start
2024-08-02T13:07:46Z proxy[9185420c444908] ewr [info]Starting machine
2024-08-02T13:07:46Z proxy[9185420c444908] ewr [error][PM07] failed to change machine state: machine still active, refusing to start
2024-08-02T13:07:46Z proxy[9185420c444908] lga [error][PR04] could not find a good candidate within 91 attempts at load balancing
2024-08-02T13:07:57Z app[683ddd9f6dd6e8] ewr [info]2024-08-02T13:07:57.755371431 [01J44MJDA5WM30SP86E048ZG4J:main] Running Firecracker v1.7.0
2024-08-02T13:07:58Z app[683ddd9f6dd6e8] ewr [info][    0.291631] PCI: Fatal: No config space access function found
2024-08-02T13:07:58Z app[683ddd9f6dd6e8] ewr [info] INFO Starting init (commit: 3dd5d9e)...
2024-08-02T13:07:58Z app[683ddd9f6dd6e8] ewr [info] INFO Preparing to run: `poetry run streamlit src/app_fe.py` as root
2024-08-02T13:07:58Z app[683ddd9f6dd6e8] ewr [info] INFO [fly api proxy] listening at /.fly/api
2024-08-02T13:07:58Z app[683ddd9f6dd6e8] ewr [info]2024/08/02 13:07:58 INFO SSH listening listen_address=[fdaa:3:d72d:a7b:cb:e5c5:a9fd:2]:22 dns_server=[fdaa::3]:53
2024-08-02T13:07:58Z runner[683ddd9f6dd6e8] ewr [info]Machine started in 939ms
2024-08-02T13:08:00Z app[683ddd9f6dd6e8] ewr [info]Usage: streamlit [OPTIONS] COMMAND [ARGS]...
2024-08-02T13:08:00Z app[683ddd9f6dd6e8] ewr [info]Try 'streamlit --help' for help.
2024-08-02T13:08:00Z app[683ddd9f6dd6e8] ewr [info]Error: No such command 'src/app_fe.py'.
2024-08-02T13:08:00Z app[683ddd9f6dd6e8] ewr [info] INFO Main child exited normally with code: 2
2024-08-02T13:08:00Z app[683ddd9f6dd6e8] ewr [info] INFO Starting clean up.
2024-08-02T13:08:00Z app[683ddd9f6dd6e8] ewr [info] WARN could not unmount /rootfs: EINVAL: Invalid argument
2024-08-02T13:08:00Z app[683ddd9f6dd6e8] ewr [info][    2.740298] reboot: Restarting system
2024-08-02T13:08:00Z runner[683ddd9f6dd6e8] ewr [info]machine did not have a restart policy, defaulting to restart
2024-08-02T13:08:07Z app[9185420c444908] ewr [info]2024-08-02T13:08:07.003588146 [01J44MJD3WDMGH9BYNJ2TEDQB6:main] Running Firecracker v1.7.0
2024-08-02T13:08:07Z app[9185420c444908] ewr [info][    0.287499] PCI: Fatal: No config space access function found
2024-08-02T13:08:07Z app[9185420c444908] ewr [info] INFO Starting init (commit: 3dd5d9e)...
2024-08-02T13:08:07Z app[9185420c444908] ewr [info] INFO Preparing to run: `poetry run streamlit src/app_fe.py` as root
2024-08-02T13:08:07Z app[9185420c444908] ewr [info] INFO [fly api proxy] listening at /.fly/api
2024-08-02T13:08:07Z runner[9185420c444908] ewr [info]Machine started in 938ms
2024-08-02T13:08:07Z app[9185420c444908] ewr [info]2024/08/02 13:08:07 INFO SSH listening listen_address=[fdaa:3:d72d:a7b:95:592f:cf2:2]:22 dns_server=[fdaa::3]:53
2024-08-02T13:08:09Z app[9185420c444908] ewr [info]Usage: streamlit [OPTIONS] COMMAND [ARGS]...
2024-08-02T13:08:09Z app[9185420c444908] ewr [info]Try 'streamlit --help' for help.
2024-08-02T13:08:09Z app[9185420c444908] ewr [info]Error: No such command 'src/app_fe.py'.
2024-08-02T13:08:10Z app[9185420c444908] ewr [info] INFO Main child exited normally with code: 2
2024-08-02T13:08:10Z app[9185420c444908] ewr [info] INFO Starting clean up.
2024-08-02T13:08:10Z app[9185420c444908] ewr [info] WARN could not unmount /rootfs: EINVAL: Invalid argument
2024-08-02T13:08:10Z app[9185420c444908] ewr [info][    3.783121] reboot: Restarting system
2024-08-02T13:08:11Z runner[9185420c444908] ewr [info]machine did not have a restart policy, defaulting to restart
2024-08-02T13:08:13Z app[9185420c444908] ewr [info]2024-08-02T13:08:13.381943796 [01J44MJD3WDMGH9BYNJ2TEDQB6:main] Running Firecracker v1.7.0
2024-08-02T13:08:13Z app[9185420c444908] ewr [info][    0.285340] PCI: Fatal: No config space access function found
2024-08-02T13:08:14Z app[9185420c444908] ewr [info] INFO Starting init (commit: 3dd5d9e)...
2024-08-02T13:08:14Z app[9185420c444908] ewr [info] INFO Preparing to run: `poetry run streamlit src/app_fe.py` as root
2024-08-02T13:08:14Z app[9185420c444908] ewr [info] INFO [fly api proxy] listening at /.fly/api
2024-08-02T13:08:14Z app[9185420c444908] ewr [info]2024/08/02 13:08:14 INFO SSH listening listen_address=[fdaa:3:d72d:a7b:95:592f:cf2:2]:22 dns_server=[fdaa::3]:53
2024-08-02T13:08:14Z runner[9185420c444908] ewr [info]Machine started in 851ms
2024-08-02T13:08:15Z app[9185420c444908] ewr [info]Usage: streamlit [OPTIONS] COMMAND [ARGS]...
2024-08-02T13:08:15Z app[9185420c444908] ewr [info]Try 'streamlit --help' for help.
2024-08-02T13:08:15Z app[9185420c444908] ewr [info]Error: No such command 'src/app_fe.py'.
2024-08-02T13:08:16Z app[9185420c444908] ewr [info] INFO Main child exited normally with code: 2
2024-08-02T13:08:16Z app[9185420c444908] ewr [info] INFO Starting clean up.
2024-08-02T13:08:16Z app[9185420c444908] ewr [info] WARN could not unmount /rootfs: EINVAL: Invalid argument
2024-08-02T13:08:16Z app[9185420c444908] ewr [info][    2.702715] reboot: Restarting system
2024-08-02T13:08:16Z runner[9185420c444908] ewr [info]machine has reached its max restart count (10)
2024-08-02T13:08:17Z app[683ddd9f6dd6e8] ewr [info]2024-08-02T13:08:17.426510704 [01J44MJDA5WM30SP86E048ZG4J:main] Running Firecracker v1.7.0
2024-08-02T13:08:17Z app[683ddd9f6dd6e8] ewr [info][    0.292344] PCI: Fatal: No config space access function found
2024-08-02T13:08:18Z app[683ddd9f6dd6e8] ewr [info] INFO Starting init (commit: 3dd5d9e)...
2024-08-02T13:08:18Z app[683ddd9f6dd6e8] ewr [info] INFO Preparing to run: `poetry run streamlit src/app_fe.py` as root
2024-08-02T13:08:18Z app[683ddd9f6dd6e8] ewr [info] INFO [fly api proxy] listening at /.fly/api
2024-08-02T13:08:18Z app[683ddd9f6dd6e8] ewr [info]2024/08/02 13:08:18 INFO SSH listening listen_address=[fdaa:3:d72d:a7b:cb:e5c5:a9fd:2]:22 dns_server=[fdaa::3]:53
2024-08-02T13:08:18Z runner[683ddd9f6dd6e8] ewr [info]Machine started in 954ms
2024-08-02T13:08:20Z app[683ddd9f6dd6e8] ewr [info]Usage: streamlit [OPTIONS] COMMAND [ARGS]...
2024-08-02T13:08:20Z app[683ddd9f6dd6e8] ewr [info]Try 'streamlit --help' for help.
2024-08-02T13:08:20Z app[683ddd9f6dd6e8] ewr [info]Error: No such command 'src/app_fe.py'.
2024-08-02T13:08:21Z app[683ddd9f6dd6e8] ewr [info] INFO Main child exited normally with code: 2
2024-08-02T13:08:21Z app[683ddd9f6dd6e8] ewr [info] INFO Starting clean up.
2024-08-02T13:08:21Z app[683ddd9f6dd6e8] ewr [info] WARN could not unmount /rootfs: EINVAL: Invalid argument
2024-08-02T13:08:21Z app[683ddd9f6dd6e8] ewr [info][    3.712218] reboot: Restarting system
2024-08-02T13:08:21Z runner[683ddd9f6dd6e8] ewr [info]machine has reached its max restart count (10)

[bbelderbos]

Thanks I have not used docker + fly, but let me make an account and run through this this week.

[vinostroud]

Ah, OK, maybe hold off on that and I'll work some more on it first. If I really get stuck we can go through that process.

On Wed, Aug 7, 2024 at 7:19 AM Bob Belderbos @.***> wrote:

Thanks I have not used docker + fly, but let me make an account and run through this this week.

— Reply to this email directly, view it on GitHub https://github.com/vinostroud/nfl_analytics/issues/14#issuecomment-2273233075, or unsubscribe https://github.com/notifications/unsubscribe-auth/BADYLRB45KHTWHFV3W6P5JDZQH7EDAVCNFSM6AAAAABGW6SAKWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDENZTGIZTGMBXGU . You are receiving this because you were assigned.Message ID: @.***>

[bbelderbos]

Seems something with the setup, but let me go through it tomorrow or so ...

2024-08-02T13:07:18Z app[9185420c444908] ewr [info]Error: No such command 'src/app_fe.py'.

[bbelderbos]

Starting this today, but asks for cc so will try after adding one later ...

[bbelderbos]

@vinostroud what steps did you take actually? I am not even getting to a deployed app, let a lone working. After installing their cli tool I get this:

$ fly launch
Scanning source code
INFO Detected Poetry project
Detected a Streamlit app
Creating app in /Users/bbelderbos/code/pdm/kevin/nfl_analytics
We're about to launch your Streamlit app on Fly.io. Here's what you're getting:

Organization: Bob                           (fly launch defaults to the personal org)
Name:         nfl-analytics-still-dust-5882 (generated)
Region:       Madrid, Spain                 (this is the fastest region for you)
App Machines: shared-cpu-1x, 1GB RAM        (most apps need about 1GB of RAM)
Postgres:     <none>                        (not requested)
Redis:        <none>                        (not requested)
Tigris:       <none>                        (not requested)

? Do you want to tweak these settings before proceeding? No
Created app 'nfl-analytics-still-dust-5882' in organization 'personal'
Admin URL: https://fly.io/apps/nfl-analytics-still-dust-5882
Hostname: nfl-analytics-still-dust-5882.fly.dev
...
Setting FLY_API_TOKEN secret in GitHub repository settings
Wrote config file fly.toml
Validating /Users/bbelderbos/code/pdm/kevin/nfl_analytics/fly.toml
✓ Configuration is valid
==> Building image
WARN Failed to start remote builder heartbeat: failed to restart VM 148e29e5c22578: failed_precondition: unable to restart machine, not currently started or stopped

==> Building image
WARN Failed to start remote builder heartbeat: failed to restart VM 148e29e5c22578: failed_precondition: unable to restart machine, not currently started or stopped

Error: failed to fetch an image or build from source: failed to restart VM 148e29e5c22578: failed_precondition: unable to restart machine, not currently started or stopped (Request ID: 01J4VDYBM083WJWBC1CN1D17ET-mad) (Trace ID: 287eebdb00da43259204c5ca05cff6ba)

[vinostroud]

https://nfl-analytics-icy-leaf-6591.fly.dev/

Now deployed successfully. Hooray.

Steps to fix:

The issue was in the docker-compose.yml file. Specifically, I needed to update this line:"streamlit run --server.port 8501 --server.enableCORS false src/app_fe.py". to include src/app_fe.py

Then ran docker compose build then ran docker up

The less-good news -- it is not running well and seems to crash constantly. Is this a resourcing issue (not enough ram)? How might I investigate this problem?

Then started over with fly.io. I deleted the fly.toml file, then ran fly launch to create a new file tied up to the update docker image.

[vinostroud]

Ah yes, RAM issue: Hello! Your “nfl-analytics-icy-leaf-6591” application hosted on Fly.io crashed because it ran out of memory. Specifically, the instance d8d70e6fe32028. Adding more RAM to your application might help!

You should know that adding RAM does cost money. Not much, but some. Figure about $5/mo per extra GB. Our docs have complete pricing details.

When you’re ready, add more RAM by running this from the command line:

fly scale memory [memoryMB] -a nfl-analytics-icy-leaf-6591

(replace [memoryMB] with the amount of memory you want to use, in megabytes eg. 2048)

Here’s the raw log message:

Out of memory: Killed process 318 (streamlit) total-vm:2017040kB, anon-rss:874348kB, file-rss:0kB, shmem-rss:0kB, UID:0 pgtables:2560kB oom_score_adj:0

[vinostroud]

OK, so here's a question we can consider: I can brute force this with $$ by buying more RAM.

But I am curious, maybe I can write a small script to better load the data from the csv file that would improve RAM usage. Is this something you have any suggestion on how to approach (I admittedly don't know where to start, ie, how to measure RAM usage, how to search files more efficiently)

[bbelderbos]

That's a good question, let me research. So are you sure it's related to your app and it's not a free tier thing?

[bbelderbos]

And awesome you got it working, check on certification debugging requirement :)

[bbelderbos]

Is there a way to figure out what memory is allowed vs what it's consuming + script execution time it takes vs timeout per tier?

[vinostroud]

I'm know how much ram i am given, but havent been able to track ram usage.

My thought is to try some improved techniques (like chunking during df creation, and maybe give it a retry. I'll also look at any libraries to monitor for ram usage.

On Tue, Aug 13, 2024 at 10:34 AM Bob Belderbos @.***> wrote:

Is there a way to figure out what memory is allowed vs what it's consuming

• script execution time it takes vs timeout per tier? image.png (view on web) https://github.com/user-attachments/assets/d192a949-9431-4819-a31f-2dc0b771cfad

— Reply to this email directly, view it on GitHub https://github.com/vinostroud/nfl_analytics/issues/14#issuecomment-2286414605, or unsubscribe https://github.com/notifications/unsubscribe-auth/BADYLRBYZSNZJJOVUMDAUC3ZRIKQXAVCNFSM6AAAAABGW6SAKWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEOBWGQYTINRQGU . You are receiving this because you were mentioned.Message ID: @.***>

[bbelderbos]

Let me run it again locally but I remember it would take a bit and cloud environments typically allow for 30 seconds per requests max before timing out.

[bbelderbos]

@vinostroud what steps did you take actually? I am not even getting to a deployed app, let a lone working. After installing their cli tool I get this:

$ fly launch
Scanning source code
INFO Detected Poetry project
Detected a Streamlit app
Creating app in /Users/bbelderbos/code/pdm/kevin/nfl_analytics
We're about to launch your Streamlit app on Fly.io. Here's what you're getting:

Organization: Bob                           (fly launch defaults to the personal org)
Name:         nfl-analytics-still-dust-5882 (generated)
Region:       Madrid, Spain                 (this is the fastest region for you)
App Machines: shared-cpu-1x, 1GB RAM        (most apps need about 1GB of RAM)
Postgres:     <none>                        (not requested)
Redis:        <none>                        (not requested)
Tigris:       <none>                        (not requested)

? Do you want to tweak these settings before proceeding? No
Created app 'nfl-analytics-still-dust-5882' in organization 'personal'
Admin URL: https://fly.io/apps/nfl-analytics-still-dust-5882
Hostname: nfl-analytics-still-dust-5882.fly.dev
...
Setting FLY_API_TOKEN secret in GitHub repository settings
Wrote config file fly.toml
Validating /Users/bbelderbos/code/pdm/kevin/nfl_analytics/fly.toml
✓ Configuration is valid
==> Building image
WARN Failed to start remote builder heartbeat: failed to restart VM 148e29e5c22578: failed_precondition: unable to restart machine, not currently started or stopped

==> Building image
WARN Failed to start remote builder heartbeat: failed to restart VM 148e29e5c22578: failed_precondition: unable to restart machine, not currently started or stopped

Error: failed to fetch an image or build from source: failed to restart VM 148e29e5c22578: failed_precondition: unable to restart machine, not currently started or stopped (Request ID: 01J4VDYBM083WJWBC1CN1D17ET-mad) (Trace ID: 287eebdb00da43259204c5ca05cff6ba)

can you let me know the steps you've used?

[bbelderbos]

I'm know how much ram i am given, but havent been able to track ram usage. My thought is to try some improved techniques (like chunking during df creation, and maybe give it a retry. I'll also look at any libraries to monitor for ram usage. … On Tue, Aug 13, 2024 at 10:34 AM Bob Belderbos @.> wrote: Is there a way to figure out what memory is allowed vs what it's consuming + script execution time it takes vs timeout per tier? image.png (view on web) https://github.com/user-attachments/assets/d192a949-9431-4819-a31f-2dc0b771cfad — Reply to this email directly, view it on GitHub <#14 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/BADYLRBYZSNZJJOVUMDAUC3ZRIKQXAVCNFSM6AAAAABGW6SAKWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEOBWGQYTINRQGU . You are receiving this because you were mentioned.Message ID: @.>

What is ram now? https://fly.io/docs/flyctl/scale-vm/

[vinostroud]

Back from vacation. I currently have 1GB of RAM with fly.io.

Are you still having the error deploying on fly.io?

My issues were with my Docker configuration. As I mentioned once I got that working, i deleted fly.toml file and started over.

I simply ran fly launch

fly statusor fly doctor might also give more insights into problems?

[bbelderbos]

I have not further played with this. I did try to host it on Heroku the other day and got issues there too. I will try it again later this week. Did you run fly doctor on your end?

[vinostroud]

No...but i dont need to as its working for me and the website is deployed and publicly reachable.

My issue is frequent ram crashes

On Wed, Aug 28, 2024 at 8:44 AM Bob Belderbos @.***> wrote:

I have not further played with this. I did try to host it on Heroku the other day and got issues there too. I will try it again later this week. Did you run fly doctor on your end?

— Reply to this email directly, view it on GitHub https://github.com/vinostroud/nfl_analytics/issues/14#issuecomment-2315219502, or unsubscribe https://github.com/notifications/unsubscribe-auth/BADYLRAMYCEJWMP6BHMLJ73ZTXAZXAVCNFSM6AAAAABGW6SAKWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGMJVGIYTSNJQGI . You are receiving this because you were mentioned.Message ID: @.***>

[bbelderbos]

That's a win - great job (deployment we can mark as complete now)

Then we just need to profile the app, see where it's consuming too much memory ...

[bbelderbos]

As discussed, size of the data set is very plausible, what is the size?

So the next action is to run the app with a small data set to see if it's stable?

And later on use caching with a db / orm?