ibtsamraza commented 4 months ago

The database may be in a dirty state. To resolve this, the Alembic CLI can be used from the src/phoenix/db directory inside the Phoenix project root. From here, revert any partial migrations and run alembic stamp to reset the migration state, then try starting Phoenix again. i am unable to find src/phoenix/db on my local machine i am running my app on a ubuntu server

anticorrelator commented 4 months ago

hi @ibtsamraza! I'm sorry you've run into a migration error. To locate that directory, it's probably easiest to clone our repo and navigate to src/pehoenix/db from there

axiomofjoy commented 4 months ago

@anticorrelator I think it's worth adding more explicit instructions to the message.

mikeldking commented 4 months ago

hey @ibtsamraza are you still blocked by this? Happy to get you unblocked. Let me know. We can schedule some time.

In general we only have one migration right now so it's pretty surprising you hit this error. Any details would be appreciated.

DavideLagano commented 2 months ago

I also have the error:

⚠️⚠️ Phoenix failed to migrate the database to the latest version. ⚠️⚠️

The database may be in a dirty state. To resolve this, the Alembic CLI can be used from the src/phoenix/db directory inside the Phoenix project root. From here, revert any partial migrations and run alembic stamp to reset the migration state, then try starting Phoenix again.

If issues persist, please reach out for support in the Arize community Slack: https://arize-ai.slack.com

You can also refer to the Alembic documentation for more information: https://alembic.sqlalchemy.org/en/latest/tutorial.html

If inside the folder: venv/lib/python3.12/site-packages/phoenix/db

I run: alembic stamp head

It works the first time. Then if I close and open the streamlit app again, I have to run the same code every time.

mikeldking commented 2 months ago

I also have the error:

⚠️⚠️ Phoenix failed to migrate the database to the latest version. ⚠️⚠️

The database may be in a dirty state. To resolve this, the Alembic CLI can be used from the src/phoenix/db directory inside the Phoenix project root. From here, revert any partial migrations and run alembic stamp to reset the migration state, then try starting Phoenix again.

If issues persist, please reach out for support in the Arize community Slack: https://arize-ai.slack.com

You can also refer to the Alembic documentation for more information: https://alembic.sqlalchemy.org/en/latest/tutorial.html

If inside the folder: venv/lib/python3.12/site-packages/phoenix/db

I run: alembic stamp head

It works the first time. Then if I close and open the streamlit app again, I have to run the same code every time.

@DavideLagano Sorry you are facing this issue. Can you help us diagnose this problem a bit more - what version of phoenix are you using and are you using sqlite or postgres?

DavideLagano commented 2 months ago

Last phoenix version (sorry, I'm not in front of the laptop, I'll give you the exact version tomorrow). I'm not using either sqlite or postgres. I'm just running phoenix server so that I can trace langchain.

az-arteria commented 2 months ago

I'm also facing the exact same error message as DavideLagano when trying to use postgres. I'm using the exact docker compose file found here https://docs.arize.com/phoenix/deployment/docker under the postgresql section, and it gives that error.

mikeldking commented 2 months ago

Last phoenix version (sorry, I'm not in front of the laptop, I'll give you the exact version tomorrow). I'm not using either sqlite or postgres. I'm just running phoenix server so that I can trace langchain.

Ah if you didn't specify it's sqlite. It's the default. This is super confusing as the two migrations we have don't have anything that would conflict. Would you be open to live troubleshooting with the team?

mikeldking commented 2 months ago

I'm also facing the exact same error message as DavideLagano when trying to use postgres. I'm using the exact docker compose file found here https://docs.arize.com/phoenix/deployment/docker under the postgresql section, and it gives that error.

Sorry to hear that. This is equally puzzling. Is it correct to say you had docker running for some time and then upgraded? I guess the one thing that might be the case is that the docker images might be pointing to a version that's older? Can you pin the version in the docker compose like so?

# docker-compose.yml
services:
  phoenix:
    image: arizephoenix/phoenix:version-4.10.1
    depends_on:
      - db
    ports:
      - 6006:6006  # PHOENIX_PORT
      - 4317:4317  # PHOENIX_GRPC_PORT
      - 9090:9090  # [Optional] PROMETHEUS PORT IF ENABLED
    environment:
      - PHOENIX_SQL_DATABASE_URL=postgresql://postgres:postgres@db:5432/postgres
  db:
    image: postgres
    restart: always
    environment:
      - POSTGRES_USER=postgres
      - POSTGRES_PASSWORD=postgres
      - POSTGRES_DB=postgres
    ports:
      - 5432
    volumes:
      - database_data:/var/lib/postgresql/data
volumes:
  database_data:
    driver: local

Any chance you can give us the hash of the image you are using?

DavideLagano commented 2 months ago

@mikeldking sure. I am available for live troubleshooting today. How do we organize it?

az-arteria commented 2 months ago

I'm also facing the exact same error message as DavideLagano when trying to use postgres. I'm using the exact docker compose file found here https://docs.arize.com/phoenix/deployment/docker under the postgresql section, and it gives that error.

Sorry to hear that. This is equally puzzling. Is it correct to say you had docker running for some time and then upgraded? I guess the one thing that might be the case is that the docker images might be pointing to a version that's older? Can you pin the version in the docker compose like so?
# docker-compose.yml
services:
  phoenix:
    image: arizephoenix/phoenix:version-4.10.1
    depends_on:
      - db
    ports:
      - 6006:6006  # PHOENIX_PORT
      - 4317:4317  # PHOENIX_GRPC_PORT
      - 9090:9090  # [Optional] PROMETHEUS PORT IF ENABLED
    environment:
      - PHOENIX_SQL_DATABASE_URL=postgresql://postgres:postgres@db:5432/postgres
  db:
    image: postgres
    restart: always
    environment:
      - POSTGRES_USER=postgres
      - POSTGRES_PASSWORD=postgres
      - POSTGRES_DB=postgres
    ports:
      - 5432
    volumes:
      - database_data:/var/lib/postgresql/data
volumes:
  database_data:
    driver: local
Any chance you can give us the hash of the image you are using?

I tried pinning the version of the docker image to 4.10.1, still same error. I don't think I had docker running for a long time and I didn't manually try upgrading anything.

The hash of the image I got from docker inspect is arizephoenix/phoenix@sha256:f303857cef2d2ecece881f718fdaffc4bc5b41e71a4df84a1e83a0f2e272570e

EDIT

Sorry it works now actually! The issue ended up being that my PHOENIX_SQL_DATABASE_URL env variable was accidentally slightly incorrect. I guess the error message wasn't really related to the actual issue.

Thanks for the help!

mikeldking commented 2 months ago

I'm also facing the exact same error message as DavideLagano when trying to use postgres. I'm using the exact docker compose file found here https://docs.arize.com/phoenix/deployment/docker under the postgresql section, and it gives that error.

Sorry to hear that. This is equally puzzling. Is it correct to say you had docker running for some time and then upgraded? I guess the one thing that might be the case is that the docker images might be pointing to a version that's older? Can you pin the version in the docker compose like so?
# docker-compose.yml
services:
  phoenix:
    image: arizephoenix/phoenix:version-4.10.1
    depends_on:
      - db
    ports:
      - 6006:6006  # PHOENIX_PORT
      - 4317:4317  # PHOENIX_GRPC_PORT
      - 9090:9090  # [Optional] PROMETHEUS PORT IF ENABLED
    environment:
      - PHOENIX_SQL_DATABASE_URL=postgresql://postgres:postgres@db:5432/postgres
  db:
    image: postgres
    restart: always
    environment:
      - POSTGRES_USER=postgres
      - POSTGRES_PASSWORD=postgres
      - POSTGRES_DB=postgres
    ports:
      - 5432
    volumes:
      - database_data:/var/lib/postgresql/data
volumes:
  database_data:
    driver: local
Any chance you can give us the hash of the image you are using?
I tried pinning the version of the docker image to 4.10.1, still same error. I don't think I had docker running for a long time and I didn't manually try upgrading anything.

The hash of the image I got from docker inspect is arizephoenix/phoenix@sha256:f303857cef2d2ecece881f718fdaffc4bc5b41e71a4df84a1e83a0f2e272570e

EDIT

Sorry it works now actually! The issue ended up being that my PHOENIX_SQL_DATABASE_URL env variable was accidentally slightly incorrect. I guess the error message wasn't really related to the actual issue.

Thanks for the help!

Love to hear it @az-arteria !

mikeldking commented 2 months ago

@mikeldking sure. I am available for live troubleshooting today. How do we organize it?

@DavideLagano the easiest way would be to join our slack (https://join.slack.com/t/arize-ai/shared_invite/zt-1px8dcmlf-fmThhDFD_V_48oU7ALan4Q) and we can setup a huddle with some of us. If that's not desirable reach out to me via email (mikyo@arize.com) - and we will set something up! Might not be me or today but we'll get you unblocked!

Arize-ai / phoenix

Phoenix failed to migrate the database to the latest version #3246

EDIT

Sorry it works now actually! The issue ended up being that my PHOENIX_SQL_DATABASE_URL env variable was accidentally slightly incorrect. I guess the error message wasn't really related to the actual issue.

EDIT

Sorry it works now actually! The issue ended up being that my PHOENIX_SQL_DATABASE_URL env variable was accidentally slightly incorrect. I guess the error message wasn't really related to the actual issue.