mikeizbicki / cmc-csci143

big data course materials
40 stars 77 forks source link

tables from schema.sql not loading in #563

Closed myngpog closed 3 weeks ago

myngpog commented 3 weeks ago

hi i was working on my indexes and everything was working fine but i had to rebuild stuff for one final check before moving over to prod (so I'm still on dev).

i ran docker-compose down -v and then for good measure i ran

docker stop $(docker ps -qa)
docker rm $(docker ps -qa)
docker volume prune --all

the results from all of these was that there were no arguments passed into the first two and that docker volume prune --all cleared 0 MB

also I get no output when i do docker volume ls

i brought it back up using this: docker-compose up -d --build adn then i get this as my output:

...
Step 10/10 : COPY schema.sql /docker-entrypoint-initdb.d/01.sql
 ---> Using cache
 ---> 68aa2e17f9f7
Successfully built 68aa2e17f9f7
Successfully tagged cs143-final_db:latest
Building web
DEPRECATED: The legacy builder is deprecated and will be removed in a future release.
            Install the buildx component to build images with BuildKit:
            https://docs.docker.com/go/buildx/
...
Successfully built 91bbf3a917e5
Successfully tagged cs143-final_web:latest
Creating cs143-final_db_1 ... done
Creating cs143-final_web_1 ... done

but then when i go into my psql thing and do /d, none of the tables from my schema.sql showed up, only the table users_b from the flask tutorial:

                      List of relations
  Schema  |           Name           |   Type   |    Owner
----------+--------------------------+----------+-------------
 public   | geography_columns        | view     | hello_flask
 public   | geometry_columns         | view     | hello_flask
 public   | spatial_ref_sys          | table    | hello_flask
 public   | users_b                  | table    | hello_flask
 public   | users_b_id_seq           | sequence | hello_flask
 tiger    | addr                     | table    | hello_flask
 tiger    | addr_gid_seq             | sequence | hello_flask
 tiger    | addrfeat                 | table    | hello_flask
 tiger    | addrfeat_gid_seq         | sequence | hello_flask
 tiger    | bg                       | table    | hello_flask
 tiger    | bg_gid_seq               | sequence | hello_flask
 tiger    | county                   | table    | hello_flask
 tiger    | county_gid_seq           | sequence | hello_flask
 tiger    | county_lookup            | table    | hello_flask
 tiger    | countysub_lookup         | table    | hello_flask

this is my logs:

web_1  | Waiting for postgres...
web_1  | PostgreSQL started
web_1  |  * Serving Flask app 'project/__init__.py'
web_1  |  * Debug mode: on
web_1  | WARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead.
web_1  |  * Running on all addresses (0.0.0.0)
web_1  |  * Running on http://127.0.0.1:5000
web_1  |  * Running on http://172.25.0.3:5000
web_1  | Press CTRL+C to quit
web_1  |  * Restarting with stat
web_1  |  * Debugger is active!
web_1  |  * Debugger PIN: 782-786-396
db_1   | The files belonging to this database system will be owned by user "postgres".
db_1   | This user must also own the server process.
db_1   |
db_1   | The database cluster will be initialized with locale "en_US.utf8".
db_1   | The default database encoding has accordingly been set to "UTF8".
db_1   | The default text search configuration will be set to "english".
db_1   |
db_1   | Data page checksums are disabled.
db_1   |
db_1   | fixing permissions on existing directory /var/lib/postgresql/data ... ok
db_1   | creating subdirectories ... ok
db_1   | selecting dynamic shared memory implementation ... posix
db_1   | selecting default max_connections ... 100
db_1   | selecting default shared_buffers ... 128MB
db_1   | selecting default time zone ... Etc/UTC
db_1   | creating configuration files ... ok
db_1   | running bootstrap script ... ok
db_1   | performing post-bootstrap initialization ... ok
db_1   | syncing data to disk ... ok
db_1   |
db_1   | initdb: warning: enabling "trust" authentication for local connections
db_1   | initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb.
db_1   |
db_1   | Success. You can now start the database server using:
db_1   |
db_1   |     pg_ctl -D /var/lib/postgresql/data -l logfile start
db_1   |
db_1   | waiting for server to start....2024-05-06 04:11:32.637 UTC [48] LOG:  starting PostgreSQL 16.2 (Debian 16.2-1.pgdg110+2) on x86_64-pc-linux-gnu, compiled by gcc (Debian 10.2.1-6) 10.2.1 20210110, 64-bit
db_1   | 2024-05-06 04:11:32.638 UTC [48] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
db_1   | 2024-05-06 04:11:32.642 UTC [51] LOG:  database system was shut down at 2024-05-06 04:11:32 UTC
db_1   | 2024-05-06 04:11:32.650 UTC [48] LOG:  database system is ready to accept connections
db_1   |  done
db_1   | server started
db_1   | CREATE DATABASE
db_1   |
db_1   |
db_1   | /usr/local/bin/docker-entrypoint.sh: running /docker-entrypoint-initdb.d/01.sql
db_1   | BEGIN
db_1   | CREATE TABLE
db_1   | CREATE TABLE
db_1   | CREATE TABLE
db_1   | CREATE TABLE
db_1   | CREATE EXTENSION
db_1   | CREATE INDEX
db_1   | CREATE INDEX
db_1   | CREATE INDEX
db_1   | CREATE INDEX
db_1   | CREATE INDEX
db_1   | CREATE INDEX
db_1   | CREATE INDEX
db_1   | CREATE INDEX
db_1   |
db_1   |
db_1   | /usr/local/bin/docker-entrypoint.sh: sourcing /docker-entrypoint-initdb.d/10_postgis.sh
db_1   | CREATE DATABASE
db_1   | Loading PostGIS extensions into template_postgis
db_1   | CREATE EXTENSION
db_1   | CREATE EXTENSION
db_1   | You are now connected to database "template_postgis" as user "hello_flask".
db_1   | CREATE EXTENSION
db_1   | CREATE EXTENSION
db_1   | Loading PostGIS extensions into hello_flask_dev
db_1   | CREATE EXTENSION
db_1   | CREATE EXTENSION
db_1   | You are now connected to database "hello_flask_dev" as user "hello_flask".
db_1   | CREATE EXTENSION
db_1   | CREATE EXTENSION
db_1   |
db_1   | 2024-05-06 04:11:35.504 UTC [48] LOG:  received fast shutdown request
db_1   | waiting for server to shut down....2024-05-06 04:11:35.505 UTC [48] LOG:  aborting any active transactions
db_1   | 2024-05-06 04:11:35.508 UTC [48] LOG:  background worker "logical replication launcher" (PID 54) exited with exit code 1
db_1   | 2024-05-06 04:11:35.510 UTC [49] LOG:  shutting down
db_1   | 2024-05-06 04:11:35.511 UTC [49] LOG:  checkpoint starting: shutdown immediate
db_1   | 2024-05-06 04:11:35.581 UTC [49] LOG:  checkpoint complete: wrote 4526 buffers (27.6%); 0 WAL file(s) added, 0 removed, 2 recycled; write=0.061 s, sync=0.007 s, total=0.071 s; sync files=992, longest=0.002 s, average=0.001 s; distance=35623 kB, estimate=35623 kB; lsn=0/37B1098, redo lsn=0/37B1098
db_1   | 2024-05-06 04:11:35.603 UTC [48] LOG:  database system is shut down
db_1   |  done
db_1   | server stopped
db_1   |
db_1   | PostgreSQL init process complete; ready for start up.
db_1   |
db_1   | 2024-05-06 04:11:35.635 UTC [1] LOG:  starting PostgreSQL 16.2 (Debian 16.2-1.pgdg110+2) on x86_64-pc-linux-gnu, compiled by gcc (Debian 10.2.1-6) 10.2.1 20210110, 64-bit
db_1   | 2024-05-06 04:11:35.635 UTC [1] LOG:  listening on IPv4 address "0.0.0.0", port 5432
db_1   | 2024-05-06 04:11:35.635 UTC [1] LOG:  listening on IPv6 address "::", port 5432
db_1   | 2024-05-06 04:11:35.636 UTC [1] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
db_1   | 2024-05-06 04:11:35.642 UTC [74] LOG:  database system was shut down at 2024-05-06 04:11:35 UTC
db_1   | 2024-05-06 04:11:35.651 UTC [1] LOG:  database system is ready to accept connections

I recall in office hours you gave someone a command that dealt with how pruning removed 0 MB but i forgot it unfortunately any ideas? thanks!