search

jaesivsm / JARR

JARR is a web news aggregator.
https://app.jarr.info
GNU Affero General Public License v3.0
117 stars 15 forks source link

Cannot get feed news update #221

Open Cannaxuan opened 1 year ago

Cannaxuan commented 1 year ago

Dear François Schmidts,

Thanks for your work on JaRR. I used the instance of JARR run onapp.jarr.info and it works well.

However, after I git clone the project code from the master branch and run it on my local machine, I cannot get my feed news refreshed (see below).

image

Message fromjarr-jarr-server-1

2023-02-12 20:00:43 {"timestamp": "2023-02-12T12:00:43.583061Z", "message": "Starting new HTTPS connection (1): rss.nytimes.com:443", "host": "3ca8cebc6109", "path": "/jarr/.local/share/virtualenvs/jarr-08iWm22F/lib/python3.9/site-packages/urllib3/connectionpool.py", "tags": [], "level": "DEBUG", "logger": "urllib3.connectionpool", "msg": "Starting new HTTPS connection (%d): %s:%s", "stack_info": null}
2023-02-12 20:00:45 {"timestamp": "2023-02-12T12:00:45.446008Z", "message": "https://rss.nytimes.com:443 \"GET /services/xml/rss/nyt/World.xml HTTP/1.1\" 200 23215", "host": "3ca8cebc6109", "path": "/jarr/.local/share/virtualenvs/jarr-08iWm22F/lib/python3.9/site-packages/urllib3/connectionpool.py", "tags": [], "level": "DEBUG", "logger": "urllib3.connectionpool", "msg": "%s://%s:%s \"%s %s %s\" %s %s", "stack_info": null}
2023-02-12 20:00:45 {"timestamp": "2023-02-12T12:00:45.500295Z", "message": "'site_link' already field", "host": "3ca8cebc6109", "path": "/jarr/jarr/controllers/feed_builder.py", "tags": [], "level": "DEBUG", "logger": "jarr.controllers.feed_builder", "msg": "%r already field", "stack_info": null}
2023-02-12 20:00:45 {"timestamp": "2023-02-12T12:00:45.500663Z", "message": "'link' already field", "host": "3ca8cebc6109", "path": "/jarr/jarr/controllers/feed_builder.py", "tags": [], "level": "DEBUG", "logger": "jarr.controllers.feed_builder", "msg": "%r already field", "stack_info": null}
2023-02-12 20:00:45 {"timestamp": "2023-02-12T12:00:45.501529Z", "message": "Starting new HTTPS connection (1): www.nytimes.com:443", "host": "3ca8cebc6109", "path": "/jarr/.local/share/virtualenvs/jarr-08iWm22F/lib/python3.9/site-packages/urllib3/connectionpool.py", "tags": [], "level": "DEBUG", "logger": "urllib3.connectionpool", "msg": "Starting new HTTPS connection (%d): %s:%s", "stack_info": null}
2023-02-12 20:00:45 {"timestamp": "2023-02-12T12:00:45.781576Z", "message": "https://www.nytimes.com:443 \"GET /section/world HTTP/1.1\" 200 132407", "host": "3ca8cebc6109", "path": "/jarr/.local/share/virtualenvs/jarr-08iWm22F/lib/python3.9/site-packages/urllib3/connectionpool.py", "tags": [], "level": "DEBUG", "logger": "urllib3.connectionpool", "msg": "%s://%s:%s \"%s %s %s\" %s %s", "stack_info": null}
2023-02-12 20:00:45 {"timestamp": "2023-02-12T12:00:45.858585Z", "message": "Starting new HTTPS connection (1): www.nytimes.com:443", "host": "3ca8cebc6109", "path": "/jarr/.local/share/virtualenvs/jarr-08iWm22F/lib/python3.9/site-packages/urllib3/connectionpool.py", "tags": [], "level": "DEBUG", "logger": "urllib3.connectionpool", "msg": "Starting new HTTPS connection (%d): %s:%s", "stack_info": null}
2023-02-12 20:00:45 {"timestamp": "2023-02-12T12:00:45.935189Z", "message": "https://www.nytimes.com:443 \"GET /vi-assets/static-assets/favicon-d2483f10ef688e6f89e23806b9700298.ico HTTP/1.1\" 200 1671", "host": "3ca8cebc6109", "path": "/jarr/.local/share/virtualenvs/jarr-08iWm22F/lib/python3.9/site-packages/urllib3/connectionpool.py", "tags": [], "level": "DEBUG", "logger": "urllib3.connectionpool", "msg": "%s://%s:%s \"%s %s %s\" %s %s", "stack_info": null}
2023-02-12 20:00:48 {"timestamp": "2023-02-12T12:00:48.511595Z", "message": "Starting new HTTPS connection (1): www.nytimes.com:443", "host": "3ca8cebc6109", "path": "/jarr/.local/share/virtualenvs/jarr-08iWm22F/lib/python3.9/site-packages/urllib3/connectionpool.py", "tags": [], "level": "DEBUG", "logger": "urllib3.connectionpool", "msg": "Starting new HTTPS connection (%d): %s:%s", "stack_info": null}
2023-02-12 20:00:48 {"timestamp": "2023-02-12T12:00:48.711810Z", "message": "https://www.nytimes.com:443 \"GET /vi-assets/static-assets/favicon-d2483f10ef688e6f89e23806b9700298.ico HTTP/1.1\" 200 1671", "host": "3ca8cebc6109", "path": "/jarr/.local/share/virtualenvs/jarr-08iWm22F/lib/python3.9/site-packages/urllib3/connectionpool.py", "tags": [], "level": "DEBUG", "logger": "urllib3.connectionpool", "msg": "%s://%s:%s \"%s %s %s\" %s %s", "stack_info": null}
2023-02-12 20:01:20 {"timestamp": "2023-02-12T12:01:20.213941Z", "message": "WORKER TIMEOUT (pid:26)", "host": "3ca8cebc6109", "path": "/jarr/.local/share/virtualenvs/jarr-08iWm22F/lib/python3.9/site-packages/gunicorn/glogging.py", "tags": [], "level": "CRITICAL", "logger": "gunicorn.error", "msg": "WORKER TIMEOUT (pid:%s)", "stack_info": null}
2023-02-12 20:01:22 {"timestamp": "2023-02-12T12:01:22.465997Z", "message": "WORKER TIMEOUT (pid:9)", "host": "3ca8cebc6109", "path": "/jarr/.local/share/virtualenvs/jarr-08iWm22F/lib/python3.9/site-packages/gunicorn/glogging.py", "tags": [], "level": "CRITICAL", "logger": "gunicorn.error", "msg": "WORKER TIMEOUT (pid:%s)", "stack_info": null}
2023-02-12 20:08:56 {"timestamp": "2023-02-12T12:08:56.294236Z", "message": "WORKER TIMEOUT (pid:15)", "host": "3ca8cebc6109", "path": "/jarr/.local/share/virtualenvs/jarr-08iWm22F/lib/python3.9/site-packages/gunicorn/glogging.py", "tags": [], "level": "CRITICAL", "logger": "gunicorn.error", "msg": "WORKER TIMEOUT (pid:%s)", "stack_info": null}

Could you help me to solve this issue? The feed url is: https://rss.nytimes.com/services/xml/rss/nyt/World.xml

Many thanks! Canna

jaesivsm commented 1 year ago

jarr-server is the http backend serving the client with data. WORKER TIMEOUT seems to indicate the http worker is taking too long, which, in turns seems to indicate a database issue. But since you seem to be able to connect to JARR, it may also be another kind of error. The feeds themself are refreshed by both jarr-worker and jarr-clusterizer docker process. Are they running ? Could you paste their logs ?

Cannaxuan commented 1 year ago

I tried to check the log injarr-worker, but it's empty image

and seems there is no jarr-clusterizer docker process image

Btw, I used the prod-example.yml for the setup. Do I need to change any configs there? I saw your pipeline jobs in CI/CD also failed.

onestep87 commented 1 month ago

Hey, sorry for bumping old issue, i have a similar issue as previous poster. basically after creating a base image and bringing up env they would start correctly

onestep87@DESKTOP-FF5RBPR:~/JARR$ make start-env COMPOSE_FILE=Dockerfiles/prod-example.yml [+] Running 7/7 ✔ Network jarr_jarr Created 0.0s ✔ Container redis Started 0.1s ✔ Container rabbitmq Started 0.1s ✔ Container postgres Started 0.1s ✔ Container jarr-jarr-worker-1 Started 0.0s ✔ Container jarr-jarr-server-1 Started 0.0s ✔ Container jarr-jarr-front-1 Started

i would see docker containers running in docker desktop, but if i try to see docker containers that are managed by Docker Compose using the config prod_example.yml it would be empty

onestep87@DESKTOP-FF5RBPR:~/JARR$ docker-compose --file Dockerfiles/prod-example.yml ps WARN[0000] /home/onestep87/JARR/Dockerfiles/prod-example.yml: version is obsolete NAME IMAGE COMMAND SERVICE CREATED STATUS PORTS

i would have error logs worker timeout in jarr servers logs

docker logs jarr-jarr-server-1 {"timestamp": "2024-05-14T21:05:44.750055Z", "message": "WORKER TIMEOUT (pid:14)", "host": "3fd128663ae9", "path": "/jarr/.local/share/virtualenvs/jarr-08iWm22F/lib/python3.11/site-packages/gunicorn/glogging.py", "tags": [], "level": "CRITICAL", "logger": "gunicorn.error", "msg": "WORKER TIMEOUT (pid:%s)", "stack_info": null}

and the logs for the worker are empty

And then i won't be able to run a scheduler on jarr-worker, because it's not running

onestep87@DESKTOP-FF5RBPR:~/JARR$ docker-compose --file Dockerfiles/prod-example.yml exec jarr-worker pipenv run python3 WARN[0000] /home/onestep87/JARR/Dockerfiles/prod-example.yml: version is obsolete service "jarr-worker" is not running

Maybe i am understanding something wrong with installation guide?

Many thanks, Daniil