search

kiwix / operations

Kiwix Kubernetes Cluster
http://charts.k8s.kiwix.org/
7 stars 0 forks source link

Zimfarm: `chance` worker is accepting youtube tasks but not whitelisted #217

Closed benoit74 closed 4 months ago

benoit74 commented 4 months ago

chance worker is back to work (if I'm not mistaken it was gone for few months) but it is accepted youtube tasks while its IP is not whitelisted on Youtube.

Consequence is that all youtube tasks have sudenly failed one by one when assigned to this worker.

I've sent a mail to worker owner to solve the situation (either confirm he is OK with processing youtube tasks, confirm its IP and add it to Youtube IP or remove it from its worker configuration): https://kiwix.freshdesk.com/a/tickets/71762

Until situation is solved, youtube tasks have significant chance to fail because the there is significant chances they will be assigned to this worker which is not capable to run youtube task. Error is ValueError: Unable to connect to Youtube API v3. check API_KEY.. Assigning the task manually to a worker capable to run youtube tasks (athena18, badger1, badger2, ondemand, pixelmemory, razor, blade) is a workaround for "urgent" tasks (but do not forget that workers can process only 1 youtube task at a time).

This proves (again) that implementing https://github.com/openzim/zimfarm/issues/909 is urgent so that we can control on Zimfarm level which kind of tasks a given worker is allowed to run.

@RavanJAltaie @Popolechien FYI

benoit74 commented 4 months ago

Issue solved, the worker owner had whitelisted youtube scraper by mistake, this is fixed now.