search

run-llama / sec-insights

A real world full-stack application using LlamaIndex
https://www.secinsights.ai/
MIT License
2.32k stars 631 forks source link

Error when running make migrate #31

Closed palasolanki closed 11 months ago

palasolanki commented 1 year ago

(llama-app-backend-py3.10) rl13@rl13:~/RL-Work/Code/sec-insights/backend$ make migrate echo "Running migrations." Running migrations. docker compose start db [+] Running 1/1 ⠿ Container backend-db-1 Started 0.4s poetry run python -m alembic upgrade head Traceback (most recent call last): File "/usr/lib/python3.10/runpy.py", line 196, in _run_module_as_main return _run_code(code, main_globals, None, File "/usr/lib/python3.10/runpy.py", line 86, in _run_code exec(code, run_globals) File "/home/rl13/.cache/pypoetry/virtualenvs/llama-app-backend-3I5GPRtC-py3.10/lib/python3.10/site-packages/alembic/main.py", line 4, in main(prog="alembic") File "/home/rl13/.cache/pypoetry/virtualenvs/llama-app-backend-3I5GPRtC-py3.10/lib/python3.10/site-packages/alembic/config.py", line 630, in main CommandLine(prog=prog).main(argv=argv) File "/home/rl13/.cache/pypoetry/virtualenvs/llama-app-backend-3I5GPRtC-py3.10/lib/python3.10/site-packages/alembic/config.py", line 624, in main self.run_cmd(cfg, options) File "/home/rl13/.cache/pypoetry/virtualenvs/llama-app-backend-3I5GPRtC-py3.10/lib/python3.10/site-packages/alembic/config.py", line 601, in run_cmd fn( File "/home/rl13/.cache/pypoetry/virtualenvs/llama-app-backend-3I5GPRtC-py3.10/lib/python3.10/site-packages/alembic/command.py", line 385, in upgrade script.run_env() File "/home/rl13/.cache/pypoetry/virtualenvs/llama-app-backend-3I5GPRtC-py3.10/lib/python3.10/site-packages/alembic/script/base.py", line 578, in run_env util.load_python_file(self.dir, "env.py") File "/home/rl13/.cache/pypoetry/virtualenvs/llama-app-backend-3I5GPRtC-py3.10/lib/python3.10/site-packages/alembic/util/pyfiles.py", line 93, in load_python_file module = load_module_py(module_id, path) File "/home/rl13/.cache/pypoetry/virtualenvs/llama-app-backend-3I5GPRtC-py3.10/lib/python3.10/site-packages/alembic/util/pyfiles.py", line 109, in load_module_py spec.loader.exec_module(module) # type: ignore File "", line 883, in exec_module File "", line 241, in _call_with_frames_removed File "/home/rl13/RL-Work/Code/sec-insights/backend/alembic/env.py", line 9, in from app.core.config import settings File "/home/rl13/RL-Work/Code/sec-insights/backend/./app/core/config.py", line 169, in settings = Settings() File "pydantic/env_settings.py", line 40, in pydantic.env_settings.BaseSettings.init File "pydantic/main.py", line 341, in pydantic.main.BaseModel.init pydantic.error_wrappers.ValidationError: 1 validation error for Settings DATABASE_URL field required (type=value_error.missing) make: *** [Makefile:23: migrate] Error 1

palasolanki commented 1 year ago

(llama-app-backend-py3.10) rl13@rl13:~/RL-Work/Code/sec-insights/backend$ make run_docker echo "Running in local mode with docker." Running in local mode with docker. docker compose up [+] Running 3/0 ⠿ Container backend-db-1 Running 0.0s ⠿ Container backend-llama-app-fastapi-1 Created 0.0s ⠿ Container localstack_main Created 0.0s Attaching to backend-db-1, backend-llama-app-fastapi-1, localstack_main localstack_main | localstack_main | LocalStack version: 2.2.1.dev localstack_main | LocalStack Docker container id: a8dbe36884a6 localstack_main | LocalStack build date: 2023-09-14 localstack_main | LocalStack build git hash: 858bbf28 localstack_main | localstack_main | 2023-09-15T11:33:12.324 INFO --- [-functhread4] hypercorn.error : Running on https://0.0.0.0:4566 (CTRL + C to quit) localstack_main | 2023-09-15T11:33:12.324 INFO --- [-functhread4] hypercorn.error : Running on https://0.0.0.0:4566 (CTRL + C to quit) localstack_main | Ready. backend-llama-app-fastapi-1 | Running in AppEnvironment: local backend-llama-app-fastapi-1 | 2023-09-15 11:33:48,031 [MainThread ] [INFO ] Set up logging with log level 10 backend-llama-app-fastapi-1 | 2023-09-15 11:33:48,031 [MainThread ] [INFO ] Skipping Sentry setup backend-llama-app-fastapi-1 | 2023-09-15 11:33:48,031 [MainThread ] [DEBUG] Skipping migrations backend-llama-app-fastapi-1 | INFO: Will watch for changes in these directories: ['/app'] backend-llama-app-fastapi-1 | INFO: Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit) backend-llama-app-fastapi-1 | INFO: Started reloader process [1] using StatReload backend-llama-app-fastapi-1 | INFO: Started server process [18] backend-llama-app-fastapi-1 | INFO: Waiting for application startup. backend-llama-app-fastapi-1 | Connected to the database on attempt 1. backend-llama-app-fastapi-1 | 2023-09-15 11:33:52,964 INFO sqlalchemy.engine.Engine select pg_catalog.version() backend-llama-app-fastapi-1 | 2023-09-15 11:33:52,964 INFO sqlalchemy.engine.Engine [raw sql] {} backend-llama-app-fastapi-1 | 2023-09-15 11:33:52,965 INFO sqlalchemy.engine.Engine select current_schema() backend-llama-app-fastapi-1 | 2023-09-15 11:33:52,965 INFO sqlalchemy.engine.Engine [raw sql] {} backend-llama-app-fastapi-1 | 2023-09-15 11:33:52,966 INFO sqlalchemy.engine.Engine show standard_conforming_strings backend-llama-app-fastapi-1 | 2023-09-15 11:33:52,966 INFO sqlalchemy.engine.Engine [raw sql] {} backend-llama-app-fastapi-1 | 2023-09-15 11:33:52,967 INFO sqlalchemy.engine.Engine BEGIN (implicit) backend-llama-app-fastapi-1 | 2023-09-15 11:33:52,972 INFO sqlalchemy.engine.Engine SELECT pg_catalog.pg_class.relname backend-llama-app-fastapi-1 | FROM pg_catalog.pg_class JOIN pg_catalog.pg_namespace ON pg_catalog.pg_namespace.oid = pg_catalog.pg_class.relnamespace backend-llama-app-fastapi-1 | WHERE pg_catalog.pg_class.relname = %(table_name)s AND pg_catalog.pg_class.relkind = ANY (ARRAY[%(param_1)s, %(param_2)s, %(param_3)s, %(param_4)s, %(param_5)s]) AND pg_catalog.pg_table_is_visible(pg_catalog.pg_class.oid) AND pg_catalog.pg_namespace.nspname != %(nspname_1)s backend-llama-app-fastapi-1 | 2023-09-15 11:33:52,972 INFO sqlalchemy.engine.Engine [generated in 0.00029s] {'table_name': 'alembic_version', 'param_1': 'r', 'param_2': 'p', 'param_3': 'f', 'param_4': 'v', 'param_5': 'm', 'nspname_1': 'pg_catalog'} backend-llama-app-fastapi-1 | 2023-09-15 11:33:52,984 INFO sqlalchemy.engine.Engine COMMIT backend-llama-app-fastapi-1 | ERROR: Traceback (most recent call last): backend-llama-app-fastapi-1 | File "/root/.cache/pypoetry/virtualenvs/llama-app-backend-9TtSrW0h-py3.11/lib/python3.11/site-packages/starlette/routing.py", line 677, in lifespan backend-llama-app-fastapi-1 | async with self.lifespan_context(app) as maybe_state: backend-llama-app-fastapi-1 | File "/usr/local/lib/python3.11/contextlib.py", line 204, in aenter backend-llama-app-fastapi-1 | return await anext(self.gen) backend-llama-app-fastapi-1 | ^^^^^^^^^^^^^^^^^^^^^ backend-llama-app-fastapi-1 | File "/app/app/main.py", line 77, in lifespan backend-llama-app-fastapi-1 | raise Exception( backend-llama-app-fastapi-1 | Exception: Database is not up to date. Please run poetry run alembic upgrade head backend-llama-app-fastapi-1 | backend-llama-app-fastapi-1 | ERROR: Application startup failed. Exiting. ^CGracefully stopping... (press Ctrl+C again to force) [+] Running 3/3 ⠿ Container localstack_main Stopped 2.1s ⠿ Container backend-llama-app-fastapi-1 Stopped 0.9s ⠿ Container backend-db-1 Stopped 0.3s canceled make: *** [Makefile:15: run_docker] Error 130

sourabhdesai commented 1 year ago

So looks like you initially tried migrating the database with make migrate but that failed, and then tried spinning up the service with make run_docker and that failed because the migrations hadn't been applied, is that right?

You'd need to ensure the migrations have been applied before being able to spin up the service.

Looking at your output of make migrate:

pydantic.error_wrappers.ValidationError: 1 validation error for Settings
DATABASE_URL
field required (type=value_error.missing)

Seems like your DATABASE_URL environment variable hasn't been set. You'll need to make sure you did step 5 in the backend's setup instructions here which mentions how to properly setup and source from your .env file. Let me know if that helps!

palasolanki commented 1 year ago

i try make run_docker and error come as i mention, after Ctrl + C and try make migrate but still error come

and i think you mention DATABASE_URL inside docker-compse.yml right ? services: llama-app-fastapi: build: context: . args: DATABASE_URL: postgresql://user:password@db:5432/llama_app_db

because you mention in make run_docker


    echo "Running in local mode with docker."
    # Check if .env file would set the DATABASE_URL parameter and ask the user to remove it
    # from .env if it would
    # This is needed because the DATABASE_URL parameter is set in docker-compose.yml
    # Also account for the case where DATABASE_URL exists but has been commented out by
    # doing a regex match to see that its at the start of a line
    if [ -f .env ] && grep -q -E '^\s*DATABASE_URL\s*=' .env; then \
        echo "The .env file contains a DATABASE_URL parameter. Please remove it from the .env file and try again."; \
        exit 1; \
    fi```

I follow 5 step set -a and source .env
sourabhdesai commented 1 year ago

Did you run set -a and source .env before running make migrate? You need to have the DATABASE_URL env var set in your environment before running the migrations

sourabhdesai commented 11 months ago

Closing since there aren't any further questions

rohit-1906 commented 10 months ago

poetry run python -m alembic upgrade head /root/.cache/pypoetry/virtualenvs/llama-app-backend--Qk0ygDj-py3.11/bin/python: No module named alembic.main; 'alembic' is a package and cannot be directly executed make: *** [Makefile:15: migrate] Error 1

Getting some error while making migration @sourabhdesai