search

billyfung / flask-celery

GNU General Public License v3.0
0 stars 0 forks source link

performance testing #1

Open billyfung opened 5 years ago

billyfung commented 5 years ago

have controller assign N jobs to M workers

something like:

  1. hit endpoint /start_job
  2. app starts N*M*100 jobs
  3. see how long it takes as you add more jobs?
billyfung commented 5 years ago

sequential:

100 jobs 5 workers celery: ~1m40s rq: ~1m40s

1000 jobs 5 workers celery: ~9m rq: ~16m

not sure if i have the rq code wrong or not optimal...

billyfung commented 5 years ago

right, it's because celery has concurrency by default , which is why it's approximately 2x faster for more jobs