search

mikeizbicki / cmc-csci143

big data course materials
40 stars 76 forks source link

sh load_tweets_parallel.sh broken pipe #530

Closed luisgomez214 closed 6 months ago

luisgomez214 commented 7 months ago

Hello,

Today in office hours @mikeizbicki helped me with an issue by deleting and inserting my pg_denormalized and pg_normalized_batch. I have to run 'sh load_tweets_parallel.sh' again but when I do I get the following output:

lambda-server:~/bigdata/twitter_postgres_indexes (master *%=) $ sh load_tweets_parallel.sh
================================================================================
load pg_denormalized
================================================================================
client_loop: send disconnect: Broken pipe

I got this error twice after running the command for over two hours each time. Is this due to the lambda server being full/slow?

abizermamnoon commented 6 months ago

Did you exit out of the lambda server while the file was running?

luisgomez214 commented 6 months ago

I have my terminal opened in one window and I have github on a different window. While this was running, I started to work on the project. Prior to today, I was able to leave the command running and work on other assignments.

abizermamnoon commented 6 months ago

trying running the command in the background

nohup ./load_tweets_parallel.sh &

Maybe something interrupted the process. Having the command run in the background might fix the issue