search

open-iscsi / tcmu-runner

A daemon that handles the userspace side of the LIO TCM-User backstore.
Apache License 2.0
189 stars 147 forks source link

Avoid multi tcmu-runner processes to run at container env #460

Open lxbsz opened 6 years ago

lxbsz commented 6 years ago

@mikechristie @pkalever

Is it necessary to avoid multi tcmu-runner processes to run in LIO/TCMU & tcmu-runner layer here for container case ? For the PR:#418 it won't work for this case.

Though from the container layer we could avoid more than one LIO/TCMU & tcmu-runner container to run at the same time (?) Will this issue also make sense ?

Thanks,

mikechristie commented 6 years ago

I didn't fully understand the question.

I think we cannot have multiple tcmu-runner processes for the container case right now because they see the same /sys/class/uio and /sys/..../target/core space. If you restart one runner process it will think it owns all the devices in uio and core.

How does netlink currently work too btw? It seems really broken to me. I need to look into it some more but I think the kernel netlink code just broadcasts the events to all runner instances so they all end up trying to handle the event.

Does that answer your question or am I talking about something else?

lxbsz commented 6 years ago

Sorry for confusing, in the case without containers, the PR#418 could avoid the muti tcmu-runner processes by checking "/var/run/lock/tcmu.lock" and failing the rest newer processes, but in container case, the tcmu.lock file won't be visible by the other containers and the node machine.

That to say, one kernel could and allow to run more than one tcmu-runner processes in different containers. I was thinking should we detect this situation and fail the rest newer processes with logging out some useful messages ?

Just after the kernel netlink broadcasts the events, which of the tcmu-runner processes will be the first or correct one to handle them ? Do you have any idea about this ? I didn't have a chance to test it yet, and this idea was just from one of our BZ's comment from someone...

Not sure whether this fixing will make sense for the products ?

Thanks,

mikechristie commented 6 years ago

I was thinking should we detect this situation and fail the rest newer processes with logging out some useful messages ?

I get it. Yeah, that sounds useful.

Just after the kernel netlink broadcasts the events, which of the tcmu-runner processes will be the first or correct one to handle them ? Do you have any idea about this ?

I am not sure.

Not sure whether this fixing will make sense for the products ?

At least for ceph we have not seen anyone request to have multiple tcmu-runners running in their own containers, so it is not high priority for us.

lxbsz commented 6 years ago

At least for ceph we have not seen anyone request to have multiple tcmu-runners running in their own containers, so it is not high priority for us.

Yeah, this is not high priority currently. So let's just record it here.

Thanks,