Frankkkkk / pykorm

A python 🐍 kubernetes ☸️ ORM πŸš€. Very useful when writing operators for your CRDs with Kopf.
MIT License
73 stars 4 forks source link

Make pykorm faster on large number of objects #22

Closed Frankkkkk closed 2 years ago

Frankkkkk commented 3 years ago

Pykorm is rather slow when dealing with a large number of objects. In my case, at ~850 objects, an .all() query takes around 20s !

>>> a = time.time() ; len(list(MyCRD.query.all())); print(time.time() - a)
851
21.512641668319702

in comparison, kubectl takes only 1s:

$ time kl get MyCRD.p6k.ch -A 
real    0m1.392s
user    0m0.536s
sys 0m0.182s
Frankkkkk commented 2 years ago

Fixed by 55229f3e47f78a65cd90530373f1ccd96642bfdf

Before:

a = time.time(); len(list(CRD.query.all())); print(time.time() - a)
2826
37.06418442726135

Now:

>>> a = time.time(); len(list(CRD.query.all())); print(time.time() - a)
2826
6.689880609512329