search

mikeizbicki / cmc-csci143

big data course materials
40 stars 76 forks source link

twitter_postgres_parallel: passing test cases in server but not in GitHub Actions #503

Closed meghnapamula closed 7 months ago

meghnapamula commented 7 months ago

Hi,

I am working on the twitter_postgres_parallel homework assignment and I am finding that my tests cases are passing in the server but not in GitHub Actions. I have looked through the pinned issue and the issues linked in it, and have incorporated the suggestions there. However, I am still facing this issue. I think perhaps I am getting a different error than the people who posted about it in the pinned issue.

Here is the error I am getting right now in GitHub actions, even though all my test cases are passing:

(Background on this error at: https://sqlalche.me/e/14/e3q8)
2024-04-13 02:36:47.194961 insert_tweets i= 0
2024-04-13 02:36:50.467792 insert_tweets i= 1
/home/runner/work/twitter_postgres_parallel/twitter_postgres_parallel/load_tweets_batch.py:394: RemovedIn20Warning: Deprecated API features detected! These feature(s) are not compatible with SQLAlchemy 2.0. To prevent incompatible upgrades prior to updating applications, ensure requirements files are pinned to "sqlalchemy<2.0". Set environment variable SQLALCHEMY_WARN_20=1 to show all deprecation warnings.  Set environment variable SQLALCHEMY_SILENCE_UBER_WARNING=1 to silence this message. (Background on SQLAlchemy 2.0 at: https://sqlalche.me/e/b8d9)
2024-04-13 02:36:46.531932 data/geoTwitter21-01-10.zip
  with connection.begin() as trans:
2024-04-13 02:36:47.440522 insert_tweets i= 0
2024-04-13 02:36:52.671215 insert_tweets i= 1
2024-04-13 02:36:53.442112 insert_tweets i= 2
2024-04-13 02:36:54.236569 insert_tweets i= 3
2024-04-13 02:36:55.045543 insert_tweets i= 4
2024-04-13 02:36:55.838611 insert_tweets i= 5
2024-04-13 02:36:56.619201 insert_tweets i= 6
2024-04-13 02:36:57.431602 insert_tweets i= 7
2024-04-13 02:36:58.270555 insert_tweets i= 8
2024-04-13 02:36:59.143830 insert_tweets i= 9
Error: Process completed with exit code 8.

I have been reading the links in the error message and am feeling a bit lost. I would appreciate any guidance on this issue.

ains-arch commented 7 months ago

If you're passing locally but failing on the github you may have stuff stored in a volume locally that won't exist when it builds on github. If you haven't already, you should run the tests on a clean container by doing:

docker-compose down
docker ps -a

kill any containers still listed with docker rm [container id]

docker volume ls
docker volume rm [container name, for all three]
docker-compose build
docker-compose up -d

and then run the tests again and see if they're still passing.

Also - which test case are you working on right now, that's passing locally but failing on github?