Process.get_connections(): provide an argument to filter for connection types

[GoogleCodeExporter]

As of right now Process.get_connections() returns a list for all TCP and UDP 
connections opened by process, for both IPv4 and IPv6 address families.

One might be interested in filtering those results to, say, collect TCP 
connections only, or TCP + IPv6 only.

There's someone who already solved this problem in a fork of psutil by 
introducing a "kind" parameter accepting a string:
https://github.com/elventear/psutil/commit/f2821283a411e52876384ad4aa73cb4db4973
532

The solution looks pretty nice to me in contrast to introducing something like 
two separate "family" and "type" arguments requiring to use socket.AF_* and 
socket.SOCK_* constants.

The possible solutions for the "kind" parameter would be the following:

        inet            IPv4 and IPv6
        inet4           IPv4
        inet6           IPv6
        tcp             TCP
        tcp4            TCP over IPv4
        tcp6            TCP over IPv6
        udp             UDP
        udp4            UDP over IPv4
        udp6            UDP over IPv6
        all             the sum of all the possible families and protocols

"kind" parameter should default to "inet", so that we remain compatible with 
the current implementation returning TCP + UDP + IPv4 + IPv6.
This would also be compatible when we'll add support for UNIX sockets (issue 
216). 

Example usage:

>>> p = psutil.Process(os.getpid())
>>> p.get_connections(kind="tcp")
[...]
>>> p.get_connections(kind="inet6")
[...]
>>>

Original issue reported on code.google.com by g.rodola on 6 Oct 2011 at 3:39

[GoogleCodeExporter]

Sounds like a good idea, only thing I'd suggest is using a different name for 
the parameter instead of "kind". Some alternative suggestions: 

* filter
* family
* proto (protocol)

Original comment by jlo...@gmail.com on 6 Oct 2011 at 3:45

[GoogleCodeExporter]

We can't use "family" and "proto" because the filter we're going to apply is 
for both of them (e.g. you can use kind="tcp4" to filter for TCP *protocol* AND 
IPv4 *family* in one shot).

-1 about calling this "filter" because we already have a python builtin with 
the same name.

Original comment by g.rodola on 6 Oct 2011 at 3:55

[GoogleCodeExporter]

ok, how about "type" or "conn_type" then :) 

Original comment by jlo...@gmail.com on 6 Oct 2011 at 4:19

[GoogleCodeExporter]

"family", "type" and "proto" are all blacklisted names because they recall the 
arguments passed to the socket constructor: 
http://docs.python.org/library/socket.html#socket.socket
Generally speaking, all 3 of them have a precise meaning in the socket world, 
and the less we conflict with them, the more unambiguous we are.

Is there a particular reason why you don't like "kind"?

Original comment by g.rodola on 6 Oct 2011 at 4:28

[GoogleCodeExporter]

"kind" just isn't very descriptive to me of what it does, I would prefer 
conn_type or something that indicates specifically what the parameter is for. 
Feel free to come up with other ideas, I was just throwing a few things out 
there off the top of my head :)

Original comment by jlo...@gmail.com on 6 Oct 2011 at 4:38

[GoogleCodeExporter]

Isn't "kind" a synonym of "type"?
Again, "conn_type" is ambiguous because it conflicts with "type".
If we exclude "family"/"type" and their variants I personally can't think of 
any other valid substitute (but as you know English is not my native language 
=)).

Original comment by g.rodola on 6 Oct 2011 at 10:21

[GoogleCodeExporter]

The word "kind" has a number of different meanings in English, so it wouldn't 
be my ideal choice for something like a parameter name. For example, it can 
mean "gentle" so it could indicate an option to use a less aggressive 
algorithm. I would prefer something that's as clear as possible and does not 
have multiple meanings. How about conn_filter for example? It indicates that 
the parameter is a filter for the connections which is clear, and does not 
conflict with any built-ins or other meanings within the socket module. 

Original comment by jlo...@gmail.com on 6 Oct 2011 at 11:45

[GoogleCodeExporter]

What we would really need here, if "type" wasn't a blacklisted name because of 
its correlation with sockets, is "type".
I think the most obvious thing do to is look for a synonym of "type" and 
between all the available choices 
(http://dictionary.reverso.net/english-synonyms/type) "kind" looks like the 
better one to me.

It is not perfect, but with the exception of "gentle"/"courteous" both "type" 
and "kind" have almost the exact same meaning, identifying a certain character, 
genre or category.
It appears clear to me what "kind" means in the context of something called 
"get_connections", because connections can't obviously be "gentle".
I mean, this is "get_connections()", not "get_mate()" or "get_husband()", 
right?  =)

Now, I'm not a native English speaker, but here:
http://difference-between.com/english-language/kind-sort-and-type/
..."kind" is described as a "named category, but not necessarily precise", 
which fits well with what we're returning: a certain types/kinds of mixed 
combinations which are not strictly related to each other such as tcp+ipv4, 
udp+ipv6, inet6, unix, etc.

As for "conn_filter":
- it is a repetition of "connection", which is already the name of the function
- it is longer
- it contains a "_", which we've been avoiding so far (see: "percpu", 
"perdisk", "pernic")

Original comment by g.rodola on 7 Oct 2011 at 11:57

[GoogleCodeExporter]

I have a slightly different implementation of get_connections().  Right now, 
get_connections is broken on all but Linux because the OS-specific providers 
weren't updated to have the kind field and then use it.  My refactoring, 
attached, handles the connection kind filtering in 
psutil/__init__.get_connections(kind) much like how we handle totals for 
disk/network I/O counters.  So the provider's get_connections() will return all 
connections and then they are filtered in the get_connections(kind) 
implementation in __init__.py.

Original comment by jcscoob...@gmail.com on 12 Oct 2011 at 11:03

Attachments:

• get_connections_refactor.diff

[GoogleCodeExporter]

Point is getting all connections first (and create a nameduplte for each one of 
them) is less performant. 
This is particularly slow on Linux, where instead of calling a kernel function, 
we read different files in python.
Same for FreeBSD where this is done by parsing lsof output.

I think it's better to apply filter at a lower level (the C extension module in 
case of OSX and Windows).
For OSX I was working on this (uncommitted) patch which passes the desired 
families/types as arguments for the underlying C function. It's not very nice 
though.

Original comment by g.rodola on 13 Oct 2011 at 10:45

Attachments:

• osx.patch

[GoogleCodeExporter]

Well, my purpose was to fix the builds so we can verify other things are 
working properly.  We could just use my approach and make enhancements to make 
it more performant in the meantime instead of having a broken psutil.  As for 
your patch, I'll look at it shortly.

Original comment by jcscoob...@gmail.com on 13 Oct 2011 at 4:14

[GoogleCodeExporter]

Feel free to commit your patch if it's for fixing tests. We can remove it later.

Original comment by g.rodola on 13 Oct 2011 at 4:16

[GoogleCodeExporter]

Per our conversation, I've committed my patch as part of r1147.  We can put a 
plan in place to move the implementation from Python to C where necessary, or 
for all platforms.  At least with this new patch trunk builds no longer fail on 
non-Linux and we can safely add new features and fix bugs due to the test suite 
running properly.

Original comment by jcscoob...@gmail.com on 13 Oct 2011 at 4:23

[GoogleCodeExporter]

I realize now that you wanted to not change _pslinux.py.  I will address this.

Original comment by jcscoob...@gmail.com on 13 Oct 2011 at 4:26

[GoogleCodeExporter]

Original comment by g.rodola on 13 Oct 2011 at 6:52

• Added labels: Progress-3in4

[GoogleCodeExporter]

Original comment by g.rodola on 21 Oct 2011 at 11:44

• Removed labels: Milestone-0.3.1

[GoogleCodeExporter]

Original comment by g.rodola on 21 Oct 2011 at 11:45

• Added labels: Milestone-0.4.0

[GoogleCodeExporter]

Fixed for FreeBSD in r1186.

Original comment by g.rodola on 22 Oct 2011 at 3:51

• Changed state: FixedInSVN
• Added labels: Progress-4in4
• Removed labels: Progress-3in4

[GoogleCodeExporter]

Original comment by g.rodola on 29 Oct 2011 at 3:44

• Changed state: Fixed

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

Updated csets after the SVN -> Mercurial migration:
r1147 == revision 2e5e54fe37bd
r1186 == revision 3b284f40b531

Original comment by g.rodola on 2 Mar 2013 at 12:04