mikeizbicki / cmc-csci143

big data course materials
40 stars 76 forks source link

unable to remove $PGDATA #506

Closed myngpog closed 7 months ago

myngpog commented 7 months ago

hello, when I run these commands:

$ docker-compose exec pg_normalized_batch bash -c 'rm -rf $PGDATA'
$ docker-compose exec pg_denormalized bash -c 'rm -rf $PGDATA'

in my twitter_postgres_indexes folder, it says: rm: cannot remove '/var/lib/postgresql/data': Device or resource busy

ains-arch commented 7 months ago

I'm not positive but I think that error doesn't mean it didn't delete what it needed to. If you check the size of the files with docker-compose exec [container name] -c 'du -hd0 $PGDATA' i found they were much smaller after I ran the command. I'm not sure if they should still be 40M and 101M with no data in them though.

myngpog commented 7 months ago

hmm that may be the case but when I run ls -l $HOME/bigdata

the output is still:

total 12
drwx------+ 19  4688518 students 4096 Apr  8 15:19 pg_denormalized
drwx------+ 19  4688518 students 4096 Apr  8 15:19 pg_normalized_batch

even after the rm command

ains-arch commented 7 months ago

~Yeah I don't think that rm -rf command is supposed to the delete the directory on the lambda server, just clear the data postgres has in the container~ actually i dont think i know what im talking about here

You can also try running something like this

docker-compose exec pg_normalized_batch bash -c 'ls -l $PGDATA'

to check to see if there's still data in there after you run the rm -rf command, though I didn't do that so I'm not sure it'll change the output.

Also, if that was copied from your terminal and not directly from the instructions (I notice the UIDs match) I'd be concerned that the date is Apr 8 and not Apr 14.

myngpog commented 7 months ago

heyy thanks for the response! this is what i get when running the suggested command and there's a lot of stuff in there:

total 128
drwx------ 6 postgres postgres  4096 Apr 15 05:20 base
drwx------ 2 postgres postgres  4096 Apr 15 05:20 global
drwx------ 2 postgres postgres  4096 Apr 15 05:20 pg_commit_ts
drwx------ 2 postgres postgres  4096 Apr 15 05:20 pg_dynshmem
-rw------- 1 postgres postgres  5743 Apr 15 05:20 pg_hba.conf
-rw------- 1 postgres postgres  2640 Apr 15 05:20 pg_ident.conf
drwx------ 4 postgres postgres  4096 Apr 15 05:20 pg_logical
drwx------ 4 postgres postgres  4096 Apr 15 05:20 pg_multixact
drwx------ 2 postgres postgres  4096 Apr 15 05:20 pg_notify
drwx------ 2 postgres postgres  4096 Apr 15 05:20 pg_replslot
drwx------ 2 postgres postgres  4096 Apr 15 05:20 pg_serial
drwx------ 2 postgres postgres  4096 Apr 15 05:20 pg_snapshots
drwx------ 2 postgres postgres  4096 Apr 15 05:20 pg_stat
drwx------ 2 postgres postgres  4096 Apr 15 05:20 pg_stat_tmp
drwx------ 2 postgres postgres  4096 Apr 15 05:20 pg_subtrans
drwx------ 2 postgres postgres  4096 Apr 15 05:20 pg_tblspc
drwx------ 2 postgres postgres  4096 Apr 15 05:20 pg_twophase
-rw------- 1 postgres postgres     3 Apr 15 05:20 PG_VERSION
drwx------ 3 postgres postgres  4096 Apr 15 05:20 pg_wal
drwx------ 2 postgres postgres  4096 Apr 15 05:20 pg_xact
-rw------- 1 postgres postgres    88 Apr 15 05:20 postgresql.auto.conf
-rw------- 1 postgres postgres 29770 Apr 15 05:20 postgresql.conf
-rw------- 1 postgres postgres    36 Apr 15 05:20 postmaster.opts
-rw------- 1 postgres postgres    94 Apr 15 05:20 postmaster.pid

I'll keep trying the commands in the step 0 from the homework in the intended order and see if anything changes though, but as of now, the output after the ls -l $HOME/bigdata is something like this:

total 8
drwx------ 19 19106438 students 4096 Apr 14 22:20 pg_denormalized
drwx------ 19 19106438 students 4096 Apr 14 22:20 pg_normalized_batch

instead of the intended lrwxrwxrwx 1 root root 35 Jan 3 15:04 /home/csci143example/bigdata -> /data/users_bigdata/csci143example/