search

zalando-stups / mint-worker

The secret rotator and distributor for the STUPS ecosystem
http://stups.readthedocs.org/en/latest/components/mint.html
Other
9 stars 3 forks source link

Improve performance #10

Open harti2006 opened 9 years ago

harti2006 commented 9 years ago

Now, with > 900 registered apps / mint configuration, the job runs forever.

We should think about using caches at some places, e.g. when getting the service user list, or maybe when reading the application details from Kio.

sarnowski commented 9 years ago

+1, it should be trivial to parallize as each app is processed in an independent task. Just throw them in a bounded threadpool for execution. This will save us for the next time until we implement distributed locking for the entites in order to scale the amount of worker machines.

sarnowski commented 9 years ago

hint: parallizing (through threads and/or servers) means more calls to our security service which will lead them to block us. find solution with them.

hjacobs commented 9 years ago

+1 to parallelize