Add display of container sizes via system-df

[soyo42]

Container size display proposal.

[TomasTomecek]

Thanks for opening the PR and contributing the code!

So, I am trying it right now, and gotta say, it doesn't work. But it seems like it's fault of dockerd (this is a gentoo box, running docker 17.06 ce). This is what's happening:

Traceback (most recent call last):                                                                                                                                          
  File "/home/tt/dev/sen/sen/cli.py", line 77, in <module>                                 
    sys.exit(main())                                                                                                                                                         
  File "/home/tt/dev/sen/sen/cli.py", line 61, in main                                                                                                                       
    ui.run()                                                                                                                                                                
  File "/home/tt/dev/sen/sen/tui/init.py", line 31, in run                                                                                                                  
    self.ui.run_command(DisplayListingCommand.name, queue=SameThreadPriority())                                                                                             
  File "/home/tt/dev/sen/sen/tui/ui.py", line 295, in run_command                                                                                                           
    command.run()                                                                                                                                                            
  File "/home/tt/dev/sen/sen/tui/commands/display.py", line 67, in run                                                                                                       
    b = MainListBuffer(self.ui, self.docker_backend)                                                                                                                         
  File "/home/tt/dev/sen/sen/tui/buffer.py", line 206, in __init__                                                                                                           
    super().__init__()                                                                           
  File "/home/tt/dev/sen/sen/tui/buffer.py", line 62, in __init__                                       
    self.refresh()                      
  File "/home/tt/dev/sen/sen/tui/buffer.py", line 109, in refresh                                
    refresh_func()
  File "/home/tt/dev/sen/sen/tui/views/main.py", line 54, in refresh
    self.query(query_string=query)
  File "/home/tt/dev/sen/sen/tui/views/main.py", line 155, in query
    query, c_op, i_op = self.d.filter(**backend_query)
  File "/home/tt/dev/sen/sen/docker_backend.py", line 896, in filter
    containers_o = self.get_containers(cached=cached, stopped=stopped)
  File "/home/tt/dev/sen/sen/docker_backend.py", line 122, in wrapper
    response = func(self, *args, **kwargs)
  File "/home/tt/dev/sen/sen/docker_backend.py", line 817, in get_containers
    df = self.client.df()
  File "/usr/lib64/python3.5/site-packages/docker/utils/decorators.py", line 34, in wrapper
    return f(self, *args, **kwargs)
  File "/usr/lib64/python3.5/site-packages/docker/api/daemon.py", line 24, in df
    return self._result(self._get(url), True)
  File "/usr/lib64/python3.5/site-packages/docker/utils/decorators.py", line 46, in inner
    return f(self, *args, **kwargs)
  File "/usr/lib64/python3.5/site-packages/docker/api/client.py", line 189, in _get
    return self.get(url, **self._set_request_timeout(kwargs))
  File "/usr/lib64/python3.5/site-packages/requests/sessions.py", line 488, in get
    return self.request('GET', url, **kwargs)
  File "/usr/lib64/python3.5/site-packages/requests/sessions.py", line 475, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/lib64/python3.5/site-packages/requests/sessions.py", line 596, in send
    r = adapter.send(request, **kwargs)
  File "/usr/lib64/python3.5/site-packages/requests/adapters.py", line 499, in send
    raise ReadTimeout(e, request=request)
requests.exceptions.ReadTimeout: UnixHTTPConnectionPool(host='localhost', port=None): Read timed out. (read timeout=60)

and after that, all I get is:

Traceback (most recent call last):                                                                                                                                          
  File "/home/tt/dev/sen/sen/cli.py", line 77, in <module>                                                                                                                  
    sys.exit(main())                                                                                                                                                         
  File "/home/tt/dev/sen/sen/cli.py", line 61, in main                                                                                                                       
    ui.run()                                                                                                                                                                 
  File "/home/tt/dev/sen/sen/tui/init.py", line 31, in run                                                                                                                   
    self.ui.run_command(DisplayListingCommand.name, queue=SameThreadPriority())                  
  File "/home/tt/dev/sen/sen/tui/ui.py", line 295, in run_command                                       
    command.run()                       
  File "/home/tt/dev/sen/sen/tui/commands/display.py", line 67, in run                           
    b = MainListBuffer(self.ui, self.docker_backend)
  File "/home/tt/dev/sen/sen/tui/buffer.py", line 206, in __init__  
    super().__init__()            
  File "/home/tt/dev/sen/sen/tui/buffer.py", line 62, in __init__  
    self.refresh()                                    
  File "/home/tt/dev/sen/sen/tui/buffer.py", line 109, in refresh   
    refresh_func()                                                    
  File "/home/tt/dev/sen/sen/tui/views/main.py", line 54, in refresh 
    self.query(query_string=query)        
  File "/home/tt/dev/sen/sen/tui/views/main.py", line 155, in query         
    query, c_op, i_op = self.d.filter(**backend_query)
  File "/home/tt/dev/sen/sen/docker_backend.py", line 896, in filter                       
    containers_o = self.get_containers(cached=cached, stopped=stopped)
  File "/home/tt/dev/sen/sen/docker_backend.py", line 122, in wrapper           
    response = func(self, *args, **kwargs)   
  File "/home/tt/dev/sen/sen/docker_backend.py", line 817, in get_containers             
    df = self.client.df()          
  File "/usr/lib64/python3.5/site-packages/docker/utils/decorators.py", line 34, in wrapper
    return f(self, *args, **kwargs)                          
  File "/usr/lib64/python3.5/site-packages/docker/api/daemon.py", line 24, in df  
    return self._result(self._get(url), True)
  File "/usr/lib64/python3.5/site-packages/docker/api/client.py", line 226, in _result
    self._raise_for_status(response)     
  File "/usr/lib64/python3.5/site-packages/docker/api/client.py", line 222, in _raise_for_status
    raise create_api_error_from_http_exception(e)
  File "/usr/lib64/python3.5/site-packages/docker/errors.py", line 31, in create_api_error_from_http_exception
    raise cls(e, response=response, explanation=explanation)
docker.errors.APIError: 500 Server Error: Internal Server Error ("a disk usage operation is already running")   

I will try on my other laptop tomorrow (where I expect it will work just fine and merge it).

[TomasTomecek]

Seems like I am hitting this: https://github.com/moby/moby/pull/31973

How fast is this for you?

[soyo42]

Strange - for me it works without any delay - but I have 4 tiny containers. Anyway - I queried df only once at start time. How long does it take for "time docker system df -v" to finish on your system?

I got:

real    0m0.146s
user    0m0.064s
sys 0m0.020s

Maybe this feature could rather be off by default and activated by env.variable on demand.

[TomasTomecek]

on my laptop with SSD, it's like this:

docker system df -v  0.03s user 0.03s system 0% cpu 6.996 total

On my other (older) laptop with HDD, it was something like one minute. Which is totally WTF.

So, my idea here would be to do the df call in a separate thread and update the interface once it's done. Right now I'm not sure about the precise implementation -- I would probably create a dedicated command and maybe even a dedicated view for df. Is this something you would be interested working on? I might be able to look into it later this week (no promises).

[soyo42]

Well - now it sounds like a bad idea to have this potential resource killer feature implemented. The idea with separate thread partially solves this. Moreover it might be useful to fire the query in ui on demand in order to prevent df from being run by every start of sen.

[TomasTomecek]

I already started working on this: https://github.com/TomasTomecek/sen/tree/add-df-view

[soyo42]

Cool - looks like it is no more a tiny lightweight patch but another serious feature. Just let me know when it will become testable. Thumbs up.

[TomasTomecek]

Closing in favor of #135 (your commit is included).