mikeizbicki / cmc-csci143

big data course materials
40 stars 76 forks source link

Passing test case for normalized_batch on Github but not on server #498

Closed luisgomez214 closed 7 months ago

luisgomez214 commented 7 months ago

Hello,

I have made sure to bring docker down, deleted volumes, and deleted images before bringing up docker again. When I run the test case for parallel, I pass all the normalized and most of the normalized_batch.

lambda-server:~/bigdata/twitter_postgres_parallel (master=) $ docker-compose exec pg_normalized_batch ./run_tests.sh
/home/Luis.Gomez.25/.local/lib/python3.6/site-packages/paramiko/transport.py:32: CryptographyDeprecationWarning: Python 3.6 is no longer supported by the Python core team. Therefore, support for it is deprecated in cryptography. The next release of cryptography will remove support for Python 3.6.
  from cryptography.hazmat.backends import default_backend
sql/complex_01.sql pass
sql/complex_02.sql pass
sql/complex_03.sql fail
sql/complex_04.sql fail
sql/complex_05.sql fail
sql/simple_01.sql pass
sql/simple_02.sql pass
sql/simple_03.sql pass
sql/simple_04.sql fail

but whenever I upload to Github, my test cases are green.

I am able to run

sh load_tweets_parallel.sh

and receive times significantly faster than the sequential.

Also, I have ran docker-compose logs and everything seems correct.

Is anyone else having this issue?

echen4628 commented 7 months ago

This is probably not your issue since you deleted the volumes.

I saw similar test case results when I accidentally ran the ./load_tweets_parallel.sh twice. To see if that's your issue, maybe you can take a look at your results/simple_04.sql and compare with expected/simple_04.sql. For me the count was doubled. (At the time of this happening, my code also passed on GitHub but not server).

If you have any doubts that the docker container is outdated try docker-compose build --no-cache.