create disk is too slow

[swimfish09]

I have about 250 luns. When i use curl to create new lun, it is cost about 51s, even it is about 1g disk. Is there a way to repid this? curl --insecure --user admin:admin -d mode=create -d size=1g -d pool=rbd_gw -X PUT http://10.142.90.34:8778/api/disk/rbd.zhouqi_test03

[swimfish09]

What did ceph-iscsi-api do when i created a new lun?

[dillaman]

It's definitely not optimized for creating high numbers of LUNs -- the expected use-case would be just a handful of LUNs. I believe currently each LUN creation causes a full rescan of LIO's configfs filesystem, which will take exponentially more time as more LUNs are added.

[swimfish09]

@dillaman can you give me some advices to optimize creating high numbers of LUNs?

[dillaman]

You can probably run it under the python profiler to see where it's spending all its time during a LUN addition.

[swimfish09]

from the log processing cost a lot of time.

[swimfish09]

Now processing is serial, Can i change it to parallel？

[dillaman]

I'd imagine the real problem is that it is doing unnecessary work when adding a new disk (e.g. perhaps it's reconfiguring all existing disks when adding a new LUN). It just needs to be investigated. Of course, I also would give far warning that with hundreds of disks, you are running in a "unique" configuration that probably won't see too much testing.

[mikechristie]

@swimfish09

Does every lun creation take about 50 seconds or the later ones?

Were you using a lot of gateways and if so do you have this patch: https://github.com/ceph/ceph-iscsi-cli/commit/61ec575e34273e47ba761d0acc46022b8c40f513

[swimfish09]

@mikechristie the later ones. I use two gateways.

[swimfish09]

@dillaman Now rtslib-fb allow creating more than 256 LUNs per target, hundreds of disks will be common. https://github.com/open-iscsi/rtslib-fb/commit/20a50d9967464add8d33f723f6849a197dbe0c52

[dillaman]

@swimfish09 my point is that just because you can technically do something, it doesn’t mean that it should be considered best or standard practice.

[swimfish09]

@dillaman fine, is there any technical document that i can view？

[mikechristie]

@swimfish09 There is no spec type of document.

[swimfish09]

record from start to create a disk. python -m cProfile -o profile.out rbd-target-api.py result:

[swimfish09]

function map_luns contains four loops, which cost a lot of time when create a disk two gateways will cost double times

[swimfish09]

i created a patch here. https://github.com/ceph/ceph-iscsi-config/pull/100