mikeizbicki / cmc-csci143

big data course materials
40 stars 76 forks source link

connection refused for load pg_denormalized #519

Closed JTan242 closed 7 months ago

JTan242 commented 7 months ago

I have the ports matching in my load_tweets_parallel file and my load_denormalized.sh file but I keep getting the following error:

psql: could not connect to server: Connection refused
        Is the server running on host "localhost" (127.0.0.1) and accepting
        TCP/IP connections on port 4224?

I am also wondering why I am having another issue where when I run ls -l

I get :

total 12
drwx------+  3 19892870 students 4096 Apr 20 16:33 pg_denormalized
drwx------+ 19 19892870 students 4096 Apr 22 13:37 pg_normalized_batch
drwxrwxr-x+  6 jetanb   students 4096 Apr 22 13:34 twitter_postgres_indexes

Where the expected output is :

drwx------+ 19  4688518 students 4096 Apr  8 15:19 pg_denormalized
drwx------+ 19  4688518 students 4096 Apr  8 15:19 pg_normalized_batch
JTan242 commented 7 months ago

Also when I run docker-compose log I get this error for my denormalized

pg_denormalized_1      | The files belonging to this database system will be owned by user "postgres".
pg_denormalized_1      | This user must also own the server process.
pg_denormalized_1      |
pg_denormalized_1      | The database cluster will be initialized with locale "en_US.utf8".
pg_denormalized_1      | The default database encoding has accordingly been set to "UTF8".
pg_denormalized_1      | The default text search configuration will be set to "english".
pg_denormalized_1      |
pg_denormalized_1      | Data page checksums are disabled.
pg_denormalized_1      | initdb: error: directory "/var/lib/postgresql/data" exists but is not empty
pg_denormalized_1      | If you want to create a new database system, either remove or empty
pg_denormalized_1      | the directory "/var/lib/postgresql/data" or run initdb
pg_denormalized_1      | with an argument other than "/var/lib/postgresql/data".
pg_denormalized_1      |

I tried to remove the data but it seems I'm not authorize to do so.

sudo rm -rf /var/lib/postgresql/data/*
jetanb is not in the sudoers file.  This incident will be reported.
mikeizbicki commented 7 months ago

If you are trying to delete the contents of your database, you need to run the rm command from within the docker container. You got that error message because you were running the command on the lambda server and you do not have root access there. There are instructions in the homework on how to do this, but if those are not working, than let me know and I will delete the database for you.

JTan242 commented 7 months ago

I've stopped my docker and ran those commands and the files still aren't getting removed.

lbielicki commented 7 months ago

Hi, I'm also having this problem, would you mind sharing how you solved it? I'm also getting a similar error although I did reset the containers.

psql: could not connect to server: Connection refused
        Is the server running on host "localhost" (127.0.0.1) and accepting
        TCP/IP connections on port 1991?