mikeizbicki / cmc-csci143

big data course materials
40 stars 76 forks source link

Issue with persisting my data with volumes in prod #569

Open ains-arch opened 6 months ago

ains-arch commented 6 months ago

I didn't realize I hadn't actually verified this works until now, but if I bring down my production environment with docker-compose -f docker-compose.prod.yml down and then bring it back up with docker-compose -f docker-compose.prod.yml up -d it does seem to delete the database. I'm very confused because i thought I was correctly defining the volumes and mounting them in the big data folder. Here is my docker-compose.prod.yml

version: '3.8'

services:
  web:
    build:
      context: ./services/web
      dockerfile: Dockerfile.prod
    command: gunicorn --bind 0.0.0.0:5000 manage:app
    volumes:
      - static_volume:/home/app/web/project/static
      - media_volume:/home/app/web/project/media
    expose:
      - 5000
    env_file:
      - ./.env.prod
    depends_on:
      - db
  db:
    build:
      context: ./services/postgres
      dockerfile: Dockerfile.prod
    ports:
      - 1467:5432
    volumes:
      - $HOME/bigdata/postgres_data_prod:/var/lib/postgres/data
    env_file:
      - ./.env.prod.db
  nginx:
    build: ./services/nginx
    volumes:
      - static_volume:/home/app/web/project/static
      - media_volume:/home/app/web/project/media
    ports:
      - 1447:80
    depends_on:
      - web

volumes:
  postgres_data_prod:
  static_volume:
  media_volume:

And here's what I get when I run docker volume ls after bringing up the prod containers.

$ docker volume ls
DRIVER    VOLUME NAME
local     95b2d597d9542440aa2264672981f9eb21e18ff46832d50567d4521ca953a2a5
local     ad91d96af345ac0753d5593786614ef96c53718b40f8252f70621092c988bbe3
local     flask-database_media_volume
local     flask-database_postgres_data_prod
local     flask-database_static_volume

I straight up do not know what to do here. It seems like I have everything defined the same way things were in twitter_postgres_indexes which, when I took the containers down, didn't delete the data. Would appreciate any insight at all as I'm very concerned for the well-being of my database.

Additionally, if I fumble this one thing, can I just request we not bring down the containers during the final, so that I can still get credit for the other parts of the assignment?

irajmoradi commented 6 months ago

I don't think I am having issues with my docker compose prod yml file

version: '3.8'

services:
  web:
    build:
      context: ./services/web
      dockerfile: Dockerfile.prod
    command: gunicorn --bind 0.0.0.0:1472 manage:app
    volumes:
      - static_volume:/home/app/web/project/static
      - media_volume:/home/app/web/project/media
    expose:
      - 1472
    env_file:
      - ./.env.prod
    depends_on:
      - db

  db:
    build: services/postgres
    volumes:
      - ./:/tmp/db
      - postgres_data_prod:/var/lib/postgresql/data/
    ports: ["1471:5432"]
    env_file:
      - ./.env.prod.db

  nginx:
    build: ./services/nginx
    volumes:
      - static_volume:/home/app/web/project/static
      - media_volume:/home/app/web/project/media
    ports:
      - 1472:80
    depends_on:
      - web
volumes:
  postgres_data_prod:
  static_volume:
  media_volume:

I think we do volumes significantly different though between the two of us so the problem is probably in that.

irajmoradi commented 6 months ago

I tried doing the same commands you ran and it didn't delete my database.

ains-arch commented 6 months ago

thank you for responding. here's my docker-compose.prod.yml with those edits

ersion: '3.8'

services:
  web:
    build:
      context: ./services/web
      dockerfile: Dockerfile.prod
    command: gunicorn --bind 0.0.0.0:5000 manage:app
    volumes:
      - static_volume:/home/app/web/project/static
      - media_volume:/home/app/web/project/media
    expose:
      - 5000
    env_file:
      - ./.env.prod
    depends_on:
      - db
  db:
    build:
      context: ./services/postgres
    ports:
      - 1467:5432
    volumes:
      - ./:/tmp/db
      - postgres_data_prod:/var/lib/postgres/data
    env_file:
      - ./.env.prod.db
  nginx:
    build: ./services/nginx
    volumes:
      - static_volume:/home/app/web/project/static
      - media_volume:/home/app/web/project/media
    ports:
      - 1447:80
    depends_on:
      - web
volumes:
  postgres_data_prod:
  static_volume:
  media_volume:

but running

docker-compose -f docker-compose.prod.yml down
docker-compose -f docker-compose.prod.yml up -d

still seems to delete my data

justinchiao commented 6 months ago

This worked for me

in your db service volumes:

ains-arch commented 6 months ago

that is super what I had originally which was also not working 😭 but thank you for trying