psutil doesn't know about several connection types used by ZMQ

[GoogleCodeExporter]

What steps will reproduce the problem?

See attached script. The output on my system is:

$ python psutil_fd.py 
startup
0 /dev/pts/1
1 /dev/pts/1
2 /dev/pts/1
3 None
9 /proc/2777/auxv
get_open_files [openfile(path='/proc/2777/auxv', fd=9)]
get_connections []

---

context
0 /dev/pts/1
1 /dev/pts/1
2 /dev/pts/1
3 socket:[24580675]
4 socket:[24580676]
5 socket:[24580677]
6 socket:[24580678]
7 anon_inode:[eventpoll]
8 socket:[24580679]
9 /proc/2777/auxv
10 socket:[24580680]
11 anon_inode:[eventpoll]
12 socket:[24580681]
13 socket:[24580682]
14 None
get_open_files [openfile(path='/proc/2777/auxv', fd=9)]
get_connections []

---

socket
0 /dev/pts/1
1 /dev/pts/1
2 /dev/pts/1
3 socket:[24580675]
4 socket:[24580676]
5 socket:[24580677]
6 socket:[24580678]
7 anon_inode:[eventpoll]
8 socket:[24580679]
9 /proc/2777/auxv
10 socket:[24580680]
11 anon_inode:[eventpoll]
12 socket:[24580681]
13 socket:[24580682]
14 socket:[24580700]
15 socket:[24580701]
16 None
get_open_files [openfile(path='/proc/2777/auxv', fd=9)]
get_connections []

---

What is the expected output?

I expected that psutil has some means to list the connections used by ZeroMQ. 
For example 24601841 is listed in /proc/net/unix as 

Num       RefCount Protocol Flags    Type St Inode Path
0000000000000000: 00000003 00000000 00000000 0001 03 24601841

Original issue reported on code.google.com by tiran79 on 15 Jun 2012 at 11:55

Attachments:

• psutil_fd.py

[GoogleCodeExporter]

Is this report about get_connections() not reporting UNIX-socket connections?
In that case that's already tracked in issue 216.

Original comment by g.rodola on 17 Jun 2012 at 9:42

[GoogleCodeExporter]

I didn't know about #216. That tackles the unix domain socket issue.

It's a report about psutil not showing fds it doesn't understand. For example 
get_open_files() ignores all files that are not regular files: named pipes, 
block devices, char devices. The check os.path.isfile() in 
_pslinux.Process.get_open_files() skips the char device /dev/pts/1. 
os.path.exists() should be sufficient.

I'm not sure how to handle anon_inode:[eventpoll]. The FD is created by the 
epoll() syscall.

Original comment by tiran79 on 27 Jun 2012 at 10:06

[GoogleCodeExporter]

get_open_files() is supposed to return *regular* files only, hence the 
os.path.isfile() check.
/dev/pts/1 is a terminal fd so it's ok to skip it.
If we want to return information about *all* fds that should be done as a 
separate get_open_fds() method mimicking lsof but problem is how to represent 
the data and how to conciliate the differences between UNIX platforms, which, 
IMO, it's likely to be impossible.

Original comment by g.rodola on 28 Jun 2012 at 11:58

[GoogleCodeExporter]

IMHO the limitation is too strict and removes useful information from the 
output. All this files have a fs entry but are not listed

* open directories used by the newly introduced *at() functions like openat()
* char and block devices
* named fifos
* sockets

Deleted are also skipped although they are still opened. Deleted files aren't 
correctly handles by psutil, too. Consider the case fd = open(name), 
unlink(name), touch(name). psutil returns true for fd but the fd refers to a 
different file. You have to compare st_dev and st_ino of fstat(fd) with 
stat(name) to detect the case.

Proposal:
* modify signature of the function to get_open_files(kind='REG', deleted=False)
* modify returned named tuple to contain kind (REG, DIR, SOCK, BLK, CHR, FIFO) 
and deleted (bool) flag

lsof and kinfo_getfile return the file type information.

I can create a patch if you like my idea.

Original comment by tiran79 on 1 Jul 2012 at 3:27

[GoogleCodeExporter]

Point is how to represent the different fd types, which brings to the question: 
what 'fields' the returning namedtuple should provide?
Judging from lsof output we should provide:

FD, TYPE, DEVICE, SIZE/OFF, NODE, NAME

...but if you look at hot lsof output is represented you'll notice how 
depending on the fd type it examines, certain fields are sometimes represented 
with a number, other times with a string.
That alone suggests that this is a feature which is hard to translate in a 
consistent and usable API, and I'm not even mentioning the differences between 
UNIX flavors.

> All this files have a fs entry but are not listed
> [...]
> * sockets

IMO, sockets are fine being treated as a different entity, as we're currently 
doing with get_connections(), mostly because they have a local and remote 
address field, which is not present in other fd types.

Feel free to attempt to write a patch for Linux if you want, but IMO pretty 
soon you'll realize how difficult it is to come up with a consistent API to 
express the data you pull out from the OS.

Original comment by g.rodola on 2 Jul 2012 at 12:54

[GoogleCodeExporter]

Closing this one as won't fix.
Feel free to reopen it if you can come up with a consistent proposal addressing 
the problems outlined above.

Original comment by g.rodola on 13 Dec 2012 at 2:38

• Changed state: WontFix
• Added labels: OpSys-UNIX
• Removed labels: OpSys-Linux