Slow container grid rendering OMV3.0

[subzero79]

This is the log regarding the plugin in OMV 3.0

The panel still renders very slowly in comparison to the image one.

Here is the omv-engined debug log

https://gist.github.com/subzero79/ee18d411b5f000bfa224

[nicjo814]

I thought I had fixed the $Driver warnings in 3.0.5, but I guess I'll have to give it another go. Regarding slow rendering performance I know there are huge issues when the docker service restarts (when you save the settings in the plugin). Then all containers start working at the same time and on my system they consume 100% of the CPU. If I wait until they have finishedtheir "startup work" and CPU load id back on normal levels, the container grid in the plugin loads in just a few seconds. Could you try this out on your end by just "refreshing" the container grid?

[subzero79]

I always use the refresh button once the grid is loaded. The grid actually loads now in comparison to 2.0, but it takes around 30-40 seconds.

This is a screen video

https://dl.dropboxusercontent.com/u/54639/screen1.mp4

if i copy the full curl to terminal it also takes 30-40 seconds

[nicjo814]

That was slow indeed... Is any of the containers using a lot of network ports?

[subzero79]

Nope. Most of them run with one exposed port. Plex and kodi-headless run in host mode.

[subzero79]

This is the list of containers

I've just runned top and pressed refresh, docker goes 90-100%. That might be the issue

[nicjo814]

I'll do some more testing as soon as I can to try and eliminate this issue.

[nicjo814]

I've pushed some changes now. Could you try and build from source? I don't want to release a new version if it doesn't fix the issue...

[subzero79]

I don't see any changes in the repo in the last 12 hours. Are you sure you push them?

[nicjo814]

Argh! The date/time on my VM was of by quite a lot. The latest push was made yesterday, even though it says 15 days ago here on github...

[subzero79]

ok, i've built it anyway, i was on 3.0.5. The errors were silenced, but is still slow rendering

I am trying to setup mitmproxy to monitor the api calls by the plugin. I find it little bit excessive that cpu goes 100% after i press refresh

[subzero79]

Ok, i think i found the issue. The plugin makes an api call to the image endpoint (the same call) for every container, is that suppose to be like that? Also every container gets an individual call, to get more info I guess?

Here is the video of the refresh action and on the side is the mitproxy registering all the calls

https://dl.dropboxusercontent.com/u/54639/screen.mp4

[nicjo814]

Nice! I'll do some rewrite of the container listing. It should fetch all images via api once and then use this data for further processing when building the list of containers.

I definately have to look at mitproxy...

[subzero79]

Is explained here, https://lorinstechblog.wordpress.com/2015/01/13/using-mitmproxy-to-log-docker-api-calls/

If or boot2docker, but for us i have to, change /etc/default/docker to listen on 42004, without using the plugin, then restart docker. This way rpc still thinks he has to hit the default 42005 port.

configure mitmproxy to run listen on 42005 on this file ~/.mitmproxy/common.conf, then run the proxy as

mitmproxy -R http://0.0.0.0:42004

[nicjo814]

Looking more closely at the issue it requires quite some rewrite, but it is doable. Just can't promise how long it will take...

[subzero79]

I am not a coder, but i am trying to understand a little bit more, I was reading through the function getContainers and I still don't know why do you need to call the images endpoint in the container grid. The name of the parent image for the container already comes out in /containers/json Can you clarify that?

The inspect every container endpoint call I assume is for the copy or modify button right?

[nicjo814]

Not a coder you say, BS I say :-)

You are absolutely correct regarding the api calls to the container endpoints, it's for populating the fields in the "copy container" window. The calls to the image endpoints is to populate the various drop downs with info only available in the images (exposed ports, bind mounts etc.). The way I'm gonna change the implementation is to only do one api call to list all containers (not one for every container) and then call the container/image endpoints when the user clicks the appropriate button. The reason I didn't do it this way from the start is laziness combined with that I didn't really think these api calls would be so degrading on performance.

Send me a PM on the forums when you are ready to do some coding on the project and I'll point you in the right direction (if needed) :-)

[nicjo814]

I just pushed a change to the container listing (the date for the commit is wrong again). It will break copy/modify functionality but could you test it out anyway to verify performance?

[subzero79]

Yes, the grid loaded instantly, before even the image grid. There is an error in the webgui, you already know probably. I'll wait then, thanks for your time

[subzero79]

An i am not a coder BTW :-), i am intermediate BASH and I struggle with PHP. For the plugins I try to adapt existing code from other plugins....of course with a little help from my friends.

[nicjo814]

Copy container should be working again plus some fixes to the container grid.

EDIT: what error were you thinking of in the gui?

[nicjo814]

Ok, so I think all functionality is back in the container grid area. If you give your approval I'll close this issue and start with the other one regarding what happens when you pull a new version of an image.

[nicjo814]

The image grid listing is also updated now to only do one api call.

[subzero79]

EDIT: what error were you thinking of in the gui?

Before i pull this is it

`Error #0: exception 'OMVModuleDockerException' with message 'Error: "" - Code: 0' in /usr/share/omvdocker/Utils.php:63 Stack trace:

0 /usr/share/omvdocker/Image.php(139): OMVModuleDockerUtil::doApiCall('http://localhos...')

1 /usr/share/omvdocker/Utils.php(167): OMVModuleDockerImage->__construct(NULL, Array, '42005')

2 /usr/share/openmediavault/engined/rpc/docker.inc(1349): OMVModuleDockerUtil::getImage('7173d7f50571', '42005')

3 [internal function]: OMVRpcServiceDocker->getDockerRepo(Array, Array)

4 /usr/share/php/openmediavault/rpcservice.inc(125): call_user_func_array(Array, Array)

5 /usr/share/php/openmediavault/rpc.inc(79): OMVRpcServiceAbstract->callMethod('getDockerRepo', Array, Array)

6 /usr/sbin/omv-engined(500): OMVRpc::exec('Docker', 'getDockerRepo', Array, Array, 1)

7 {main}

OK Hide details `

[subzero79]

Updated to latest, is running good so far. We can close this I think. The autocomplete path feature is back also. There is a minor issue with copy/modify we can discuss it in another thread.