-
Hi again @wg. It has been brought to our attention that in some of the results from our frameworks benchmarks, Wrk was reporting socket timeouts that we failed to capture in our automatic scraping of…
-
My sidekiq job is to crawl website and process data using nokogiri. When I open 25 threads the sidekiq process's memory usage reach to 50%, about consume 1G memory, and system's free memory left only…
-
I'm using sidekiq on my web crawler, beautiful project this sidekiq. I have 2 kind of workers a BatchWorker and SiteWorker, the BatchWorker is in charge of parsing a csv with 20000 URL and creating Si…
-
On Arborwiki -
Apache is getting stuck from time to time; I've seen it twice. The presenting symptom is a "MaxClients exceeded" error message in the logs, and the server then refuses to respond to mo…
-
I'm using celery + django-celery and running the worker as
```
./manage.py celeryd -v2 --pool=eventlet -c100 -Q io -l debug
```
This worker hangs if CELERY_DISABLE_RATE_LIMITS=False:
```
> ./manage…
kmike updated
12 years ago