Fix uuid.uuid1() core logic of uuid.getnode() needs refresh

[aixtools]

BPO	28009
Nosy	@ncoghlan, @pitrou, @vstinner, @taleinat, @serhiy-storchaka, @aixtools, @miss-islington
PRs	python/cpython#5183 python/cpython#8672 python/cpython#12777 python/cpython#14115 python/cpython#18369

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields: ```python assignee = None closed_at = created_at = labels = ['tests', 'type-bug', 'library', '3.9'] title = 'Fix uuid.uuid1() core logic of uuid.getnode() needs refresh' updated_at = user = 'https://github.com/aixtools' ``` bugs.python.org fields: ```python activity = actor = 'taleinat' assignee = 'none' closed = True closed_date = closer = 'taleinat' components = ['Library (Lib)', 'Tests'] creation = creator = 'Michael.Felt' dependencies = [] files = [] hgrepos = [] issue_num = 28009 keywords = ['patch'] message_count = 41.0 messages = ['274909', '306562', '306565', '306577', '306580', '306754', '307348', '308893', '309668', '309673', '309679', '309701', '309720', '309933', '309934', '309935', '309952', '318521', '322095', '322342', '328718', '329944', '332667', '332668', '335572', '335587', '345486', '345488', '345692', '345695', '347939', '347968', '353334', '353336', '364436', '364515', '364517', '364518', '364532', '364533', '364568'] nosy_count = 9.0 nosy_names = ['ncoghlan', 'pitrou', 'vstinner', 'taleinat', 'serhiy.storchaka', 'David.Edelsohn', 'italip', 'Michael.Felt', 'miss-islington'] pr_nums = ['5183', '8672', '12777', '14115', '18369'] priority = 'normal' resolution = 'fixed' stage = 'resolved' status = 'closed' superseder = None type = 'behavior' url = 'https://bugs.python.org/issue28009' versions = ['Python 3.9'] ```

[aixtools]

Short version:

the five routines get_node() calls to get the MAC address either fail by definition, or is wrong.

The one routine that works - is wrong because it returns the same value regardless of the system it runs on - wrong character is used to identify the string containing the mac address.

Recommended: correct and call the one routine that can work for AIX.

Details (using Python2.7 as template, applies to all Python3 versions as well)

+511 def getnode(): +512 """Get the hardware address as a 48-bit positive integer. +513 +514 The first time this runs, it may launch a separate program, which could +515 be quite slow. If all attempts to obtain the hardware address fail, we +516 choose a random 48-bit number with its eighth bit set to 1 as recommended +517 in RFC 4122. +518 """ +519 +520 global _node +521 if _node is not None: +522 return _node +523 +524 import sys +525 if sys.platform == 'win32': +526 getters = [_windll_getnode, _netbios_getnode, _ipconfig_getnode] +527 else: +528 getters = [_unixdll_getnode, _ifconfig_getnode, _arp_getnode, +529 _lanscan_getnode, _netstat_getnode] +530 +531 for getter in getters + [_random_getnode]: +532 try: +533 _node = getter() +534 except: +535 continue +536 if _node is not None: +537 return _node +538 +539 _last_timestamp = None

unixdll_getnode depends on finding uuid_generate_time in either libuuid, libc, or None

On a default install of AIX: find_library("uuid") returns None nm -Ae /usr/lib/*.a | grep uuid_generate # does not return a function name, no .a archives are stripped

+339 def _ifconfig_getnode(): +340 """Get the hardware address on Unix by running ifconfig.""" +341 # This works on Linux ('' or '-a'), Tru64 ('-av'), but not all Unixes. +342 for args in ('', '-a', '-av'): +343 mac = _find_mac('ifconfig', args, ['hwaddr', 'ether'], lambda i: i+1) +344 if mac: +345 return mac

Does not work on AIX - why call it?

+347 def _arp_getnode(): +348 """Get the hardware address on Unix by running arp.""" +349 import os, socket +350 try: +351 ip_addr = socket.gethostbyname(socket.gethostname()) +352 except EnvironmentError: +353 return None +354 +355 # Try getting the MAC addr from arp based on our IP address (Solaris). +356 return _find_mac('arp', '-an', [ip_addr], lambda i: -1)

Does not work on one Linux system I tried root@x066:~# arp -an 192.168.129.66 arp: in 2 entries no match found.

on AIX: root@x064:[/data/prj/aixtools/python/python-2.7.12.1]arp -an 192.168.129.64 ? (192.168.129.254) at XX:YY:11:aa:ZZ:ca [ethernet] stored in bucket 27 ... Nothing for it's own IP address again, why call it.

+358 def _lanscan_getnode(): +359 """Get the hardware address on Unix by running lanscan.""" +360 # This might work on HP-UX. +361 return _find_mac('lanscan', '-ai', ['lan0'], lambda i: 0)

Again, from comments - looks like it should work on HP-UX, so why call it on AIX (linux was probably solved via one of the first two, so no impact there)

So, finally, after 4 guaranteed failures the following is called:

+363 def _netstat_getnode(): +364 """Get the hardware address on Unix by running netstat.""" +365 # This might work on AIX, Tru64 UNIX and presumably on IRIX. +366 try: +367 pipe = _popen('netstat', '-ia') +368 if not pipe: +369 return +370 with pipe: +371 words = pipe.readline().rstrip().split() +372 try: +373 i = words.index('Address') +374 except ValueError: +375 return +376 for line in pipe: +377 try: +378 words = line.rstrip().split() +379 word = words[i] +380 if len(word) == 17 and word.count(':') == 5: +381 mac = int(word.replace(':', ''), 16) +382 if mac: +383 return mac +384 except (ValueError, IndexError): +385 pass +386 except OSError: +387 pass

For AIX - lines 380 and 381 do work - except the answer is ALWAYS the same:

See host x071: michael@x071:[/usr/lib]netstat -ia Name Mtu Network Address Ipkts Ierrs Opkts Oerrs Coll en0 1500 link#2 fa.d1.8c.f7.62.4 553220627 0 181051589 0 0 01:00:5e:00:00:01 en0 1500 192.168.129 x071 553220627 0 181051589 0 0 224.0.0.1 en1 65390 link#3 fa.d1.8c.f7.62.5 8004448 0 11655497 0 0 01:00:5e:00:00:01 en1 65390 192.168.2 mail.aixtools.co 8004448 0 11655497 0 0 224.0.0.1 lo0 16896 link#1 197583 0 197583 0 0 lo0 16896 127 loopback 197583 0 197583 0 0 224.0.0.1 lo0 16896 loopback 197583 0 197583 0 0 ff01::1 ff02::1:ff00:1

See host x064: root@x064:[/data/prj/aixtools/python/python-2.7.12.1]netstat -ia Name Mtu Network Address Ipkts Ierrs Opkts Oerrs Coll en0 1500 link#2 0.21.5e.a3.c7.44 192718 0 93218 0 0 01:00:5e:00:00:01 en0 1500 192.168.129 x064 192718 0 93218 0 0 224.0.0.1 lo0 16896 link#1 231 0 240 0 0 lo0 16896 127 loopback 231 0 240 0 0 224.0.0.1 lo0 16896 ::1 231 0 240 0 0 ff01::1 ff02::1:ff00:1 ff02::1

The answer found is always 01:00:5e:00:00:01

Where, for AIX at least, line 380 and line 381 should be looking at the character '.', not ':' which is valid for linux, and perhaps others.

So,

two corrections suggested:

line 380 + 381 (or as appropriate per Python version) - modify:

+378 words = line.rstrip().split() +379 word = words[i] +380 if sys.platform.startswith("aix"): +381 str = "." +382 else: +383 str = ":" +384 if len(word) == 17 and word.count(str) == 5: +385 mac = int(word.replace(str, ''), 16) +386 if mac: +387 return mac

NOTE: if str need to be a chr, then use that instead.

But much earlier in program logic - modify:

+525 if sys.platform == 'win32': +526 getters = [_windll_getnode, _netbios_getnode, _ipconfig_getnode] +527 else: +528 getters = [_unixdll_getnode, _ifconfig_getnode, _arp_getnode, +529 _lanscan_getnode, _netstat_getnode]

to

+529 if sys.platform == 'win32': +530 getters = [_windll_getnode, _netbios_getnode, _ipconfig_getnode] +531 elif sys.platform.startswith("aix"): +532 getters = [_netstat_getnode] +533 else: +534 getters = [_unixdll_getnode, _ifconfig_getnode, _arp_getnode, +535 _lanscan_getnode, _netstat_getnode]

[vstinner]

+531 elif sys.platform.startswith("aix"): +532 getters = [_netstat_getnode]

LGTM. Do you want to write this a pull request?

[serhiy-storchaka]

_unixdll_getnode, _ifconfig_getnode, and _arp_getnode were changed recently. Are they still not working on AIX?

[aixtools]

On 11/20/2017 5:22 PM, Serhiy Storchaka wrote:

Serhiy Storchaka \storchaka+cpython@gmail.com\ added the comment:

_unixdll_getnode, _ifconfig_getnode, and _arp_getnode were changed recently. Are they still not working on AIX? After a clone of master: a) _unixdll_getnode is renamed to _unix_getnode since the function _load_system_functions() still depends on find_library() - that always returns None, still not working. Note also - find_library('uuid') will not work on a standard AIX - as the library libuuid.a does not exist, neither have I ever seen a shared library with the name libuuid.so. FYI: Finally, "nm -Ae /usr/lib/libc.a | grep uuid" returns

root@x071:[/root]nm -Ae /usr/lib/libc.a | grep uuid /usr/lib/libc.a[shr.o]: ._gen_localhost_uuid t     3248320 /usr/lib/libc.a[shr.o]: ._gen_localhost_uuid@AF22_8 t     3256928 /usr/lib/libc.a[shr.o]: .uuidget_os_time   T     3234208 /usr/lib/libc.a[shr.o]: .uuiduemul         T     3236064 /usr/lib/libc.a[shr.o]: .uuid_compare        T     3237088 /usr/lib/libc.a[shr.o]: .uuid_copy           T     3236224 /usr/lib/libc.a[shr.o]: .uuid_create         T     3240480 /usr/lib/libc.a[shr.o]: .uuid_create_nil     T     3240160 /usr/lib/libc.a[shr.o]: .uuid_create_os      T     3256608 /usr/lib/libc.a[shr.o]: .uuid_disk           T     3252448 /usr/lib/libc.a[shr.o]: .uuid_equal          T     3238336 /usr/lib/libc.a[shr.o]: .uuid_equal_nc       T     3248384 /usr/lib/libc.a[shr.o]: .uuid_for_key        T     3249504 /usr/lib/libc.a[shr.o]: .uuid_from_string    T     3238944 /usr/lib/libc.a[shr.o]: .uuid_from_string_nc T     3248960 /usr/lib/libc.a[shr.o]: .uuid_get_address    t     3234720 /usr/lib/libc.a[shr.o]: .uuid_hash           T     3236384 /usr/lib/libc.a[shr.o]: .uuid_is_nil         T     3237984 /usr/lib/libc.a[shr.o]: .uuid_is_nil_nc      T     3249408 /usr/lib/libc.a[shr.o]: .uuid_is_valid_string_nc T     3248512 /usr/lib/libc.a[shr.o]: .uuid_is_valid_string_nc@AF23_10 t 3257184 /usr/lib/libc.a[shr.o]: .uuid_localhost      T     3254208 /usr/lib/libc.a[shr.o]: .uuid_os_md5         t     3245248 /usr/lib/libc.a[shr.o]: .uuid_os_sha1        t     3245376 /usr/lib/libc.a[shr.o]: .uuid_to_string      T     3239648 /usr/lib/libc.a[shr.o]: .uuid_to_string_nc   T     3248608 /usr/lib/libc.a[shr.o]: _uuid_rmutex         B      548664 12 /usr/lib/libc.a[shr.o]: _uuid_rmutex         d      512480 4 /usr/lib/libc.a[shr.o]: getosuuid            U           - /usr/lib/libc.a[shr.o]: setosuuid            U           - /usr/lib/libc.a[shr.o]: uuid_c_version       D      503544 4 /usr/lib/libc.a[shr.o]: uuid_compare         D      498436 12 /usr/lib/libc.a[shr.o]: uuid_copy            D      498412 12 /usr/lib/libc.a[shr.o]: uuid_create          D      498508 12 /usr/lib/libc.a[shr.o]: uuid_create_nil      D      498496 12 /usr/lib/libc.a[shr.o]: uuid_create_os       D      498616 12 /usr/lib/libc.a[shr.o]: uuid_disk            D      498592 12 /usr/lib/libc.a[shr.o]: uuid_equal           D      498460 12 /usr/lib/libc.a[shr.o]: uuid_equal_nc        D      498520 12 /usr/lib/libc.a[shr.o]: uuid_for_key         D      498580 12 /usr/lib/libc.a[shr.o]: uuid_from_string     D      498472 12 /usr/lib/libc.a[shr.o]: uuid_from_string_nc  D      498556 12 /usr/lib/libc.a[shr.o]: uuid_g_nil_uuid      D      432472 16 /usr/lib/libc.a[shr.o]: uuid_g_nil_uuid      d      512476 4 /usr/lib/libc.a[shr.o]: uuid_hash            D      498424 12 /usr/lib/libc.a[shr.o]: uuid_is_nil          D      498448 12 /usr/lib/libc.a[shr.o]: uuid_is_nil_nc       D      498568 12 /usr/lib/libc.a[shr.o]: uuid_is_valid_string_nc D 498532          12 /usr/lib/libc.a[shr.o]: uuid_localhost       D      498604 12 /usr/lib/libc.a[shr.o]: uuid_to_string       D      498484 12 /usr/lib/libc.a[shr.o]: uuid_to_string_nc    D      498544 12

i.e., there is nothing with uuid_generate in it.

_arp_getnode()
AIX does not return any value for itself.
Neither does Centos, (so I expect RHEL will also not), and my old 
debians do not either.

So, after summary - these three functions still do nothing/do not work on AIX 6.1 or AIX 7.1 (cannot test AIX 7.2).

And - as before, netstat -ia - when looking at ':', still returns

01:00:5e:00:00:01

for every interface.

---------- nosy: +serhiy.storchaka versions: +Python 3.7 -Python 3.3, Python 3.4, Python 3.5

---

Python tracker \report@bugs.python.org\ \https://bugs.python.org/issue28009\

---

[serhiy-storchaka]

What return commands ifconfig, ifconfig -a, ifconfig -av, ip link list, arp -an, lanscan -ai (if not error)?

[aixtools]

x064 is AIX 5.3, x071 is AIX 6.1, x072 is AIX 7.1 - in the following output

On 11/20/2017 9:01 PM, Serhiy Storchaka wrote:

Serhiy Storchaka storchaka+cpython@gmail.com added the comment:

What return commands ifconfig, ifconfig -a, ifconfig -av, ip link list, arp -an, lanscan -ai (if not error)? ifconfig:

ichael@x071:[/home/michael]dsh -n x064,x071,x072 ifconfig x071: usage: ifconfig -a [ -d ] [ -u ] [ af ] x071:        ifconfig -l [ -d ] [ -u ] x071:        ifconfig interface x071:         [ af [ address [ dest_addr ] ] [ netmask mask ] [ broadcast addr ] x071:              [ alias ] [ delete ] ] x071:         [ up ] [ down ] [ detach ] x071:         [ af first[alias] address [ ... ] ] x071:         [ site6 site_number ] x071:         [ metric n ] x071:         [ mtu n ] x071:         [ arp | -arp ] x071:         [ link0 | -link0 ] [ link1 | -link1 ] [ link2 | -link2 ] x071:         [ tcp_low_rto n | -tcp_low_rto ] x071:         [ inet6 scope n zone n ] x071:         [[ inet4 | inet6 ][address transfer newinterface]] x071:        ifconfig tap_interface [ create | destroy ] x064: usage: ifconfig -a [ -m ] [ -d ] [ -u ] [ af ] x064:        ifconfig -l [ -d ] [ -u ] x064:        ifconfig [ -m ] interface x064:         [ af [ address [ dest_addr ] ] [ netmask mask ] [ broadcast addr ] x064:              [ alias ] [ delete ] ] x064:         [ up ] [ down ] [ detach ] x064:         [ af first[alias] address [ ... ] ] x064:         [ site6 site_number ] x064:         [ metric n ] x064:         [ mtu n ] x064:         [ arp | -arp ] x064:         [ link0 | -link0 ] [ link1 | -link1 ] [ link2 | -link2 ] x064:         [ tcp_low_rto n | -tcp_low_rto ] x072: usage: ifconfig -a [ -d ] [ -u ] [ af ] x072:        ifconfig -l [ -d ] [ -u ] x072:        ifconfig interface x072:         [ af [ address [ dest_addr ] ] [ netmask mask ] [ broadcast addr ] x072:              [ alias ] [ delete ] ] x072:         [ up ] [ down ] [ detach ] x072:         [ af first[alias] address [ ... ] ] x072:         [ site6 site_number ] x072:         [ metric n ] x072:         [ mtu n ] x072:         [ arp | -arp ] x072:         [ link0 | -link0 ] [ link1 | -link1 ] [ link2 | -link2 ] x072:         [ tcp_low_rto n | -tcp_low_rto ] x072:         [ inet6 scope n zone n ] x072:        ifconfig tap_interface [ create | destroy ]

ifconfig -a:

michael@x071:[/home/michael]dsh -n x064,x071,x072 "ifconfig -a" x071: en0: flags=1e084863,14c0\<UP,BROADCAST,NOTRAILERS,RUNNING,SIMPLEX,MULTICAST,GROUPRT,64BIT,CHECKSUM_OFFLOAD(ACTIVE),LARGESEND,CHAIN> x071:   inet 192.168.129.71 netmask 0xffffff00 broadcast 192.168.129.255 x071:   inet 192.168.90.71 netmask 0xffffff00 broadcast 192.168.90.255 x071:   inet6 fe80::f8d1:8cff:fef7:6204%2/64 x071:    tcp_sendspace 262144 tcp_recvspace 262144 tcp_nodelay 1 rfc1323 1 x071: en1: flags=1e084863,480\<UP,BROADCAST,NOTRAILERS,RUNNING,SIMPLEX,MULTICAST,GROUPRT,64BIT,CHECKSUM_OFFLOAD(ACTIVE),CHAIN> x071:   inet 192.168.2.1 netmask 0xffffff00 broadcast 192.168.2.255 x071:    tcp_sendspace 262144 tcp_recvspace 262144 rfc1323 1 x071: lo0: flags=e08084b,c0\<UP,BROADCAST,LOOPBACK,RUNNING,SIMPLEX,MULTICAST,GROUPRT,64BIT,LARGESEND,CHAIN> x071:   inet 127.0.0.1 netmask 0xff000000 broadcast 127.255.255.255 x071:   inet6 ::1%1/128 x071:    tcp_sendspace 131072 tcp_recvspace 131072 rfc1323 1 x064: en0: flags=1e080863,c0\<UP,BROADCAST,NOTRAILERS,RUNNING,SIMPLEX,MULTICAST,GROUPRT,64BIT,CHECKSUM_OFFLOAD(ACTIVE),LARGESEND,CHAIN> x064:   inet 192.168.129.64 netmask 0xffffff00 broadcast 192.168.129.255 x064:    tcp_sendspace 131072 tcp_recvspace 65536 x064: lo0: flags=e08084b\<UP,BROADCAST,LOOPBACK,RUNNING,SIMPLEX,MULTICAST,GROUPRT,64BIT> x064:   inet 127.0.0.1 netmask 0xff000000 broadcast 127.255.255.255 x064:   inet6 ::1/0 x064:    tcp_sendspace 131072 tcp_recvspace 131072 rfc1323 1 x072: en0: flags=1e084863,480\<UP,BROADCAST,NOTRAILERS,RUNNING,SIMPLEX,MULTICAST,GROUPRT,64BIT,CHECKSUM_OFFLOAD(ACTIVE),CHAIN> x072:   inet 192.168.129.72 netmask 0xffffff00 broadcast 192.168.129.255 x072:    tcp_sendspace 262144 tcp_recvspace 262144 rfc1323 1 x072: lo0: flags=e08084b,c0\<UP,BROADCAST,LOOPBACK,RUNNING,SIMPLEX,MULTICAST,GROUPRT,64BIT,LARGESEND,CHAIN> x072:   inet 127.0.0.1 netmask 0xff000000 broadcast 127.255.255.255 x072:   inet6 ::1%1/64 x072:    tcp_sendspace 131072 tcp_recvspace 131072 rfc1323 1

ifconfig -av:

michael@x071:[/home/michael]dsh -n x064,x071,x072 "ifconfig -av" x071: ifconfig: illegal option -- v x071: usage: ifconfig -a [ -d ] [ -u ] [ af ] x071:        ifconfig -l [ -d ] [ -u ] x071:        ifconfig interface x071:         [ af [ address [ dest_addr ] ] [ netmask mask ] [ broadcast addr ] x071:              [ alias ] [ delete ] ] x071:         [ up ] [ down ] [ detach ] x071:         [ af first[alias] address [ ... ] ] x071:         [ site6 site_number ] x071:         [ metric n ] x071:         [ mtu n ] x071:         [ arp | -arp ] x071:         [ link0 | -link0 ] [ link1 | -link1 ] [ link2 | -link2 ] x071:         [ tcp_low_rto n | -tcp_low_rto ] x071:         [ inet6 scope n zone n ] x071:         [[ inet4 | inet6 ][address transfer newinterface]] x071:        ifconfig tap_interface [ create | destroy ] x064: ifconfig: illegal option -- v x064: usage: ifconfig -a [ -m ] [ -d ] [ -u ] [ af ] x064:        ifconfig -l [ -d ] [ -u ] x064:        ifconfig [ -m ] interface x064:         [ af [ address [ dest_addr ] ] [ netmask mask ] [ broadcast addr ] x064:              [ alias ] [ delete ] ] x064:         [ up ] [ down ] [ detach ] x064:         [ af first[alias] address [ ... ] ] x064:         [ site6 site_number ] x064:         [ metric n ] x064:         [ mtu n ] x064:         [ arp | -arp ] x064:         [ link0 | -link0 ] [ link1 | -link1 ] [ link2 | -link2 ] x064:         [ tcp_low_rto n | -tcp_low_rto ] x072: ifconfig: illegal option -- v x072: usage: ifconfig -a [ -d ] [ -u ] [ af ] x072:        ifconfig -l [ -d ] [ -u ] x072:        ifconfig interface x072:         [ af [ address [ dest_addr ] ] [ netmask mask ] [ broadcast addr ] x072:              [ alias ] [ delete ] ] x072:         [ up ] [ down ] [ detach ] x072:         [ af first[alias] address [ ... ] ] x072:         [ site6 site_number ] x072:         [ metric n ] x072:         [ mtu n ] x072:         [ arp | -arp ] x072:         [ link0 | -link0 ] [ link1 | -link1 ] [ link2 | -link2 ] x072:         [ tcp_low_rto n | -tcp_low_rto ] x072:         [ inet6 scope n zone n ] x072:        ifconfig tap_interface [ create | destroy ]

ip link list: michael@x071:[/home/michael]dsh -n x064,x071,x072 "ip link list" x071: /bin/sh: ip:  not found x064: ksh: ip:  not found x072: ksh: ip:  not found

arp -an:

michael@x071:[/home/michael]dsh -n x064,x071,x072 "arp -an" x071:   ? (192.168.129.254) at e0:28:6d:d6:eb:7b [ethernet] stored in bucket 27 x071: x071:   ? (192.168.129.53) at 0:8:9b:c8:6:e6 [ethernet] stored in bucket 124 x071: x071:   ? (192.168.129.61) at 0:14:5e:c7:f:ca [ethernet] stored in bucket 132 x071: x071:   ? (192.168.129.64) at 0:21:5e:a3:c7:44 [ethernet] stored in bucket 135 x071: x071:   ? (192.168.129.72) at fa:d1:83:60:33:4 [ethernet] stored in bucket 143 x071: x071:   ? (192.168.2.254) at 94:a7:b7:4a:ad:a7 [ethernet] stored in bucket 146 x071: x071: bucket:    0     contains:    0 entries x071: bucket:    1     contains:    0 entries x071: bucket:    2     contains:    0 entries ... x071: bucket:  145     contains:    0 entries x071: bucket:  146     contains:    1 entries x071: bucket:  147     contains:    0 entries x071: bucket:  148     contains:    0 entries x071: x071: There are 6 entries in the arp table.

x064:   ? (192.168.129.254) at e0:28:6d:d6:eb:7b [ethernet] stored in bucket 27 x064: x064:   ? (192.168.129.71) at fa:d1:8c:f7:62:4 [ethernet] stored in bucket 142 x064: x064: bucket:    0     contains:    0 entries x064: bucket:    1     contains:    0 entries x064: bucket:    2     contains:    0 entries x064: bucket:    3     contains:    0 entries x064: bucket:    4     contains:    0 entries x064: bucket:    5     contains:    0 entries ... x064: bucket:  147     contains:    0 entries x064: bucket:  148     contains:    0 entries x064: x064: There are 2 entries in the arp table. x064:

x072:   ? (192.168.129.254) at e0:28:6d:d6:eb:7b [ethernet] stored in bucket 27 x072: x072:   ? (192.168.129.2) at 28:d2:44:1e:ca:c8 [ethernet] stored in bucket 73 x072: x072:   ? (192.168.129.61) at 0:14:5e:c7:f:ca [ethernet] stored in bucket 132 x072: x072:   ? (192.168.129.63) at fa:d1:86:d4:97:4 [ethernet] stored in bucket 134 x072: x072:   ? (192.168.129.71) at fa:d1:8c:f7:62:4 [ethernet] stored in bucket 142 x072: x072: bucket:    0     contains:    0 entries x072: bucket:    1     contains:    0 entries x072: bucket:    2     contains:    0 entries x072: bucket:    3     contains:    0 entries x072: bucket:    4     contains:    0 entries ... x072: bucket:  141     contains:    0 entries x072: bucket:  142     contains:    1 entries x072: bucket:  143     contains:    0 entries x072: bucket:  144     contains:    0 entries x072: bucket:  145     contains:    0 entries x072: bucket:  146     contains:    0 entries x072: bucket:  147     contains:    0 entries x072: bucket:  148     contains:    0 entries x072: x072: There are 5 entries in the arp table. x072:

lanscan -ai: michael@x071:[/home/michael]dsh -n x064,x071,x072 "lanscan -ai" x071: /bin/sh: lanscan:  not found x064: ksh: lanscan:  not found x072: ksh: lanscan:  not found

----------

---

Python tracker \report@bugs.python.org\ \https://bugs.python.org/issue28009\

---

[vstinner]

"LGTM. Do you want to write this a pull request?"

Michael? You didn't answer to my question.

[aixtools]

Sure - I'll work on a PR. This will be the easier one.

Where I am currently 'lost' is to correct _uuidmodule.c - another thing that has always been broken (new issue, about to open).

[aixtools]

Considering that _uuid is now working for AIX bpo-32399 - I need to get some things straight (aka some help please).

Does uuid.py (./Lib/uuid.py) call _uuid.py?

If not, I am curious how _uuid.c is used - because ./Lib/test/test_uuid.py does not seem to be successful in finding it:

/data/prj/python/git/python3-3.7/Lib/test/test_uuid.py(10)\<module>() -> py_uuid = support.import_fresh_module('uuid', blocked=['_uuid']) (Pdb) n /data/prj/python/git/python3-3.7/Lib/test/test_uuid.py(11)\<module>() -> c_uuid = support.import_fresh_module('uuid', fresh=['_uuid']) (Pdb) n /data/prj/python/git/python3-3.7/Lib/test/test_uuid.py(14)\<module>() -> def importable(name): (Pdb) py_uuid \<module 'uuid' from '/data/prj/python/git/python3-3.7/Lib/uuid.py'> (Pdb) c_uuid (Pdb)

So, for now it seems the test is only testing 'uuid.py'. I would like to see the added value of having gotten the _uuid Module to build.

However, if this is 'working as designed' I not worry about it and just go back into the util.py - getnode() etc.

[aixtools]

On 08/01/2018 16:07, Michael Felt wrote:

Michael Felt \michael@felt.demon.nl\ added the comment:

Considering that _uuid is now working for AIX bpo-32399 - I need to get some things straight (aka some help please).

Does uuid.py (./Lib/uuid.py) call _uuid.py?

If not, I am curious how _uuid.c is used - because ./Lib/test/test_uuid.py does not seem to be successful in finding it:

> /data/prj/python/git/python3-3.7/Lib/test/test_uuid.py(10)\<module>() -> py_uuid = support.import_fresh_module('uuid', blocked=['_uuid']) (Pdb) n > /data/prj/python/git/python3-3.7/Lib/test/test_uuid.py(11)\<module>() -> c_uuid = support.import_fresh_module('uuid', fresh=['_uuid']) (Pdb) n > /data/prj/python/git/python3-3.7/Lib/test/test_uuid.py(14)\<module>() -> def importable(name): (Pdb) py_uuid \<module 'uuid' from '/data/prj/python/git/python3-3.7/Lib/uuid.py'> (Pdb) c_uuid (Pdb)

So, for now it seems the test is only testing 'uuid.py'. I would like to see the added value of having gotten the _uuid Module to build.

However, if this is 'working as designed' I not worry about it and just go back into the util.py - getnode() etc.

Got this figured out. tested on too many machines - the last version built was on AIX5 - which does not have uuid() functions in libc.a.

Running on AIX6 - I get:

(Pdb) py_uuid \<module 'uuid' from '/data/prj/python/git/gcc-python3-3.7/Lib/uuid.py'> (Pdb) c_uuid \<module 'uuid' from '/data/prj/python/git/gcc-python3-3.7/Lib/uuid.py'> (Pdb) import _uuid (Pdb) _uuid \<module '_uuid' from '/data/prj/python/git/gcc-python3-3.7/build/lib.aix-6.1-3.7/_uuid.so'> (Pdb)

However, have a new question - why are py_uuid and c_uuid 'equivalent'. (_uuid is shown just to show that it can be imported).

A comment - perhaps a new bpo/issue topic. When trying to run the tests from a freshly built version - it still seems to default to looking at the installed version first. And that can make debugging very very hard (which modules are actually getting loaded?!). In this case, there also seem to be a python2 and python3 incompatibility - notice the python2.7 pdb reference!

michael@x071:[/data/prj/python/git/python3-master]python -m pdb 
./Lib/test/test_uuid.py
Traceback (most recent call last):
   File "/opt/lib/python2.7/pdb.py", line 1314, in main
     pdb._runscript(mainpyfile)
   File "/opt/lib/python2.7/pdb.py", line 1233, in _runscript
     self.run(statement)
   File "/opt/lib/python2.7/bdb.py", line 400, in run
     exec cmd in globals, locals
   File "<string>", line 1, in <module>
   File "./Lib/test/test_uuid.py", line 520
     print(hex, end=' ')

----------

---

Python tracker \report@bugs.python.org\ \https://bugs.python.org/issue28009\

---

[aixtools]

On 08/01/2018 17:22, Michael wrote:

On 08/01/2018 16:07, Michael Felt wrote: > Michael Felt \michael@felt.demon.nl\ added the comment: > > Considering that _uuid is now working for AIX bpo-32399 -

some time differences - this is on POWER6 (10 years old system) - I expect POWER8 and POWER9 to be much quicker in exact times.

michael@x071:[/data/prj/python/git/xlc-python3-3.7]./python Python 3.7.0a3+ (heads/master:9b99747, Jan  8 2018, 15:50:03) [C] on aix6 Type "help", "copyright", "credits" or "license" for more information. >>> michael@x071:[/data/prj/python/git/xlc-python3-3.7]./python -m timeit -n 1000000 -r 5 'import uuid; getnode = uuid.getnode()' 1000000 loops, best of 5: 911 nsec per loop

michael@x071:[/data/prj/python/git/xlc-python3-3.7]python Python 2.7.12 (default, Sep 29 2016, 12:02:17) [C] on aix5 Type "help", "copyright", "credits" or "license" for more information. >>> michael@x071:[/data/prj/python/git/xlc-python3-3.7]python -m timeit -n 1000000 -r 5 'import uuid; getnode = uuid.getnode()' 1000000 loops, best of 5: 2.86 usec per loop michael@x071:[/data/prj/python/git/xlc-python3-3.7]

The second example is a) before the _uuid support was in the code, and b) even if in the code would not have been built on AIX5.

The relative difference is: 911/2860 :: 31.85/100 - or roughly 3x faster with _uuid.

As far as this issue is concerned - with _uuid active the buggy code I mentioned for getnode() is defunct as it does not get called/parsed. However, I shall continue with it.

Question: (must have questions!) - the behavior of getnode() without _uuid is a 'random' value for getnode() with my current setup BUT using the _uuid module it is a 'constant' that is related to (one of) the network interface(s). What is the desired behavior, specifically, of uuid.getnode() - constant, or 'random'?

> > Does uuid.py (./Lib/uuid.py) call _uuid.py?

[pitrou]

What is the desired behavior, specifically, of uuid.getnode() - constant, or 'random'?

I'm also certain getnode() is supposed to return a hardware-determined number (usually the MAC address of an Ethernet interface). A random output wouldn't need so much platform-specific harness.

[aixtools]

On 09/01/2018 16:21, Antoine Pitrou wrote:

Antoine Pitrou \pitrou@free.fr\ added the comment:

> What is the desired behavior, specifically, of uuid.getnode() - constant, or 'random'?

I'm also certain getnode() is supposed to return a hardware-determined number (usually the MAC address of an Ethernet interface). A random output wouldn't need so much platform-specific harness.

FYI

The current implementation (Lib/uuid.py) - python2 as an example with with no _uuid support, compared to the master (as of yesterday) with _uuid support.

Program (x2.py):

michael@x071:[/data/prj/python/git]cat *master/x2.py import uuid import sys print (sys.version) print("getnode = %x\n" % uuid.getnode()) print("getnode = %x\n" % uuid.getnode())

michael@x071:[/data/prj/python/git]python *master/x2.py 2.7.12 (default, Sep 29 2016, 12:02:17) [C] getnode = a92929dc8a78

getnode = a92929dc8a78

michael@x071:[/data/prj/python/git]python *master/x2.py 2.7.12 (default, Sep 29 2016, 12:02:17) [C] getnode = cdff09dd1306

getnode = cdff09dd1306

So it appears, everytime python runs - without _uuid - a new "getnode()" value is established, for as long as the process runs (hoping!)

Below - the value returned is the MAC address of the first ethernet adapter (ent0)

michael@x071:[/data/prj/python/git]gcc/python \master/x2.py 3.7.0a3+ (default, Jan  8 2018, 16:05:08) [GCC 4.7.4] getnode = fad18cf76204

getnode = fad18cf76204

michael@x071:[/data/prj/python/git]xlc/python \master/x2.py 3.7.0a3+ (heads/master:9b99747, Jan  8 2018, 15:50:03) [C] getnode = fad18cf76204

getnode = fad18cf76204

---

Is there any interest for a PR on Python2.7 and/or Python3.5? I guess it could be considered a bug IF the getnode() value is suppossed to be related to the MAC address and/or consistent over sessions.

HTH, Michael

---------- nosy: +pitrou

---

Python tracker \report@bugs.python.org\ \https://bugs.python.org/issue28009\

---

[aixtools]

PR added, please review.

[aixtools]

FYI: - On AIX, no uuid_create support:

./python -m unittest -v test.test_uuid ... Ran 48 tests in 0.261s

OK (skipped=37)

On AIX 6.1 (with ctypes, without _uuid)

./python -m unittest -v test.test_uuid ... Ran 48 tests in 0.274s

OK (skipped=32)

On AIX 6.1 (with ctypes, with _uuid)

./python -m unittest -v test.test_uuid ... Ran 48 tests in 0.379s

OK (skipped=16)

+++

nohup ./python -m unittest -v test.test_uuid | grep skipped test_arp_getnode (test.test_uuid.TestInternalsWithExtModule) ... skipped 'AIX arp does not provide MAC addr' test_ifconfig_getnode (test.test_uuid.TestInternalsWithExtModule) ... skipped 'AIX ifconfig does not provide MAC addr' test_ip_getnode (test.test_uuid.TestInternalsWithExtModule) ... skipped 'requires ip' test_ipconfig_getnode (test.test_uuid.TestInternalsWithExtModule) ... skipped 'requires Windows' test_lanscan_getnode (test.test_uuid.TestInternalsWithExtModule) ... skipped 'requires lanscan' test_netbios_getnode (test.test_uuid.TestInternalsWithExtModule) ... skipped 'requires win32wnet' test_windll_getnode (test.test_uuid.TestInternalsWithExtModule) ... skipped 'requires Windows' test_arp_getnode (test.test_uuid.TestInternalsWithoutExtModule) ... skipped 'AIX arp does not provide MAC addr' test_ifconfig_getnode (test.test_uuid.TestInternalsWithoutExtModule) ... skipped 'AIX ifconfig does not provide MAC addr' test_ip_getnode (test.test_uuid.TestInternalsWithoutExtModule) ... skipped 'requires ip' test_ipconfig_getnode (test.test_uuid.TestInternalsWithoutExtModule) ... skipped 'requires Windows' test_lanscan_getnode (test.test_uuid.TestInternalsWithoutExtModule) ... skipped 'requires lanscan' test_netbios_getnode (test.test_uuid.TestInternalsWithoutExtModule) ... skipped 'requires win32wnet' test_windll_getnode (test.test_uuid.TestInternalsWithoutExtModule) ... skipped 'requires Windows' test_uuid1_safe (test.test_uuid.TestUUIDWithExtModule) ... skipped 'requires uuid_generate_time_safe(3)' test_uuid1_safe (test.test_uuid.TestUUIDWithoutExtModule) ... skipped 'requires uuid_generate_time_safe(3)'

[pitrou]

Is nohup required in the example you just posted or is that a red herring?

[aixtools]

On 14/01/2018 22:01, Antoine Pitrou wrote:

Antoine Pitrou \pitrou@free.fr\ added the comment:

Is nohup required in the example you just posted or is that a red herring?

----------

---

Python tracker \report@bugs.python.org\ \https://bugs.python.org/issue28009\

---

nohup is required (i.e., my lazy way!) to stream both stdout and stderr through the pipe. Perhaps, for this output nohup could have been omitted. But it became a habit while I was testing. Maybe a birthday herring :p

Working on the Windows issue (my mistake). Will be groping in the dark as I have no windows system to test on.

[aixtools]

Please review PR. at least for "master"

[aixtools]

OK. as promised when I closed PR 5183 - a restart.

You may skip the wall that follows - it is just documentation.

The key points:

• AIX ifconfig and arp do not supply maccaddr
• netstat supplies a macaddr, but uses '.' not ':' as a delimiter ** also, AIX netstat value is not always 17 characters (dec 15 is 'f' not '0f')
• since Python3-3.7 there is support for _uuid so test_unix_getnode passes "WithExtModule" but fails "WithoutExtModule"

A minor point: test_find_mac() has always passed on AIX because the test was passed a macaddr using ':' and tested for ':'. The test has been modified to provide a macaddr in "AIX" format. IMHO - the test would be better if the string was not "forced to be 17 characters" (e.g., this is my macaddr as returned by netstat: "0.12.a3.53.c9.46"

My question - getting back to what I had done previously - even though _uuid is working, should the "WithoutExtModule" and support for test_find_mac() be adjusted in uuid.py - OR - are these errors in test just going to be accepted asis?

Details: (skip the wall :p )

So, with only test_uuid.py patched, the verbose returns ('ok' tests not included!):

root@x066:[/data/prj/python/python3-3.8]./python ../git/**3.8/Lib/test/regrtest.py -v test_uuid | egrep -v "ok$" == CPython 3.8.0a0 (default, Jul 20 2018, 00:54:36) [C] == AIX-1-00C291F54C00-powerpc-32bit big-endian == cwd: /data/prj/python/python3-3.8/build/test_python_11468890 == CPU count: 4 == encodings: locale=ISO8859-1, FS=iso8859-1 Run tests sequentially 0:00:00 [1/1] test_uuid test_arp_getnode (test.test_uuid.TestInternalsWithExtModule) ... skipped 'because AIX "arp" does not provide macaddr' test_find_mac (test.test_uuid.TestInternalsWithExtModule) ... FAIL test_ifconfig_getnode (test.test_uuid.TestInternalsWithExtModule) ... skipped 'because AIX "ifconfig" does not provide macaddr' test_ip_getnode (test.test_uuid.TestInternalsWithExtModule) ... skipped 'requires ip' test_ipconfig_getnode (test.test_uuid.TestInternalsWithExtModule) ... skipped 'requires Windows' test_lanscan_getnode (test.test_uuid.TestInternalsWithExtModule) ... skipped 'requires lanscan' test_netbios_getnode (test.test_uuid.TestInternalsWithExtModule) ... skipped 'requires win32wnet' test_netstat_getnode (test.test_uuid.TestInternalsWithExtModule) ... skipped 'requires netstat' test_windll_getnode (test.test_uuid.TestInternalsWithExtModule) ... skipped 'requires Windows' test_arp_getnode (test.test_uuid.TestInternalsWithoutExtModule) ... skipped 'because AIX "arp" does not provide macaddr' test_find_mac (test.test_uuid.TestInternalsWithoutExtModule) ... FAIL test_ifconfig_getnode (test.test_uuid.TestInternalsWithoutExtModule) ... skipped 'because AIX "ifconfig" does not provide macaddr' test_ip_getnode (test.test_uuid.TestInternalsWithoutExtModule) ... skipped 'requires ip' test_ipconfig_getnode (test.test_uuid.TestInternalsWithoutExtModule) ... skipped 'requires Windows' test_lanscan_getnode (test.test_uuid.TestInternalsWithoutExtModule) ... skipped 'requires lanscan' test_netbios_getnode (test.test_uuid.TestInternalsWithoutExtModule) ... skipped 'requires win32wnet' test_netstat_getnode (test.test_uuid.TestInternalsWithoutExtModule) ... skipped 'requires netstat' test_unix_getnode (test.test_uuid.TestInternalsWithoutExtModule) ... skipped 'requires uuid_generate_time' test_windll_getnode (test.test_uuid.TestInternalsWithoutExtModule) ... skipped 'requires Windows' test_uuid1_safe (test.test_uuid.TestUUIDWithExtModule) ... skipped 'requires uuid_generate_time_safe(3)' test_uuid1_bogus_return_value (test.test_uuid.TestUUIDWithoutExtModule) ... skipped 'need uuid._generate_time_safe' test_uuid1_is_safe (test.test_uuid.TestUUIDWithoutExtModule) ... skipped 'need uuid._generate_time_safe' test_uuid1_is_unsafe (test.test_uuid.TestUUIDWithoutExtModule) ... skipped 'need uuid._generate_time_safe' test_uuid1_safe (test.test_uuid.TestUUIDWithoutExtModule) ... skipped 'requires uuid_generate_time_safe(3)' test_uuid1_unknown (test.test_uuid.TestUUIDWithoutExtModule) ... skipped 'need uuid._generate_time_safe' test test_uuid failed

\====================================================================== FAIL: test_find_mac (test.test_uuid.TestInternalsWithExtModule) ----------------------------------------------------------------------

Traceback (most recent call last):
  File "/data/prj/python/git/python3-3.8/Lib/test/test_uuid.py", line 548, in test_find_mac
    self.assertEqual(mac, 0x1234567890ab)
AssertionError: None != 20015998341291

====================================================================== FAIL: test_find_mac (test.test_uuid.TestInternalsWithoutExtModule) ----------------------------------------------------------------------

Traceback (most recent call last):
  File "/data/prj/python/git/python3-3.8/Lib/test/test_uuid.py", line 548, in test_find_mac
    self.assertEqual(mac, 0x1234567890ab)
AssertionError: None != 20015998341291

---

Ran 50 tests in 0.629s

FAILED (failures=2, skipped=23) test_uuid failed

== Tests result: FAILURE ==

1 test failed: test_uuid

Total duration: 797 ms Tests result: FAILURE root@x066:[/data/prj/python/python3-3.8]

[aixtools]

@pitrou: re: Is nohup required in the example you just posted or is that a red herring?

I just use nohup to merge stdout and stderr when grepping for messages - and to, ideally, have the stdout and stderr messages properly synced in the output stream. So, I hope "red herring" :)

My apologies for the belated reply.

[aixtools]

re; the current status of PR8672 - which I shall probably close as it no longer merges.

@taleinat re: the need for lambda

As _find_mac_netstat() is only called once the need for the last two arguments may be unnecessary. My reason for including them was to keep _find_mac_netstat comparable to _find_mac.

1) I am reluctant to make changes to historical code, however, in this case it was needed as _find_mac has, imho, a requirement for input that differs from AIX netstat.

• On linux (debian, centos) I do not find a MAC address for netstat
• On linux, where, e.g., ifconfig does return a value - the lambda i: i±1 is vital as the value follows the keyword on the same line. For AIX lambda I: i is needed because only the first line (header) has keywords, the remaining lines give the values.

Linux: ifconfig (keyword "ether")

root@x074:~# ifconfig eth0: flags=4163\<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 inet 192.168.129.74 netmask 255.255.255.0 broadcast 192.168.129.255 inet6 fe80::f8d1:81ff:fe12:b104 prefixlen 64 scopeid 0x20\<link> inet6 2001:981:56fd:1:f8d1:81ff:fe12:b104 prefixlen 64 scopeid 0x0\<global> ether fa:d1:81:12:b1:04 txqueuelen 1000 (Ethernet) RX packets 1605776 bytes 177990366 (169.7 MiB) RX errors 0 dropped 1027921 overruns 0 frame 0 TX packets 25045 bytes 1567902 (1.4 MiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 device interrupt 20

So, when the keyword is found, as each word on the line is examined "i+1" gives the value

For AIX (in anycase), the keyword is Address - the output is: michael@x071:[/data/prj/python/git/python3-3.8]netstat -ia Name Mtu Network Address Ipkts Ierrs Opkts Oerrs Coll en0 1500 link#2 fa.d1.8c.f7.62.4 2348009992 0 946551098 0 0 01:00:5e:00:00:01 en0 1500 192.168.129 x071 2348009992 0 946551098 0 0 224.0.0.1 en0 1500 192.168.90 x071 2348009992 0 946551098 0 0 224.0.0.1 en1 1500 link#3 fa.d1.8c.f7.62.5 64346336 0 89935059 0 0 01:00:5e:00:00:01 en1 1500 192.168.2 mail.aixtools.xx 64346336 0 89935059 0 0 224.0.0.1

Note - the value with colons - as many (all?) other systems - this seems to be a constant, and a value on a line all by itself - so the old code could never ever find it, even if it could have parsed it. The actual MAC address is on a line with several entries - matching the values given by the "header" line - so lambda i: i is needed to examine the later lines to find a suitably formatted value.

So, should I write _find_mac_netstat for AIX only (and maybe set "skipIf" Linux). There are many assumptions in this code. I do not feel qualified to change things I cannot test - so, as much as possible I follow the tried and true.

I hope this clarifies my intent well enough that you can make a decision.

[aixtools]

Historically, I started this issue because I saw that none of the calls made in uuid.py were working for AIX.

I also assumed that they ALL worked already, at least somewhere.

a) one cause is the difference between AIX and (all) others was the letter chosen to separate the six fields that from the hexadecimal macaddr. (AIX uses '.', others use ':').

b) on AIX only netstat -ia returns a macaddr - so the GETTERS array of routines to call could be shorter

c) as the tests accept (I'll call it) no response aka None this was "passing" for all calls on AIX, including the mock test which had ":" hard coded into the test procedure and the mock data.

d) the current test has to fail (i.e., always return None) when the output is coming from netstat because the output layout is fundamentally different. With netstat the keyword is in the first-line only and it determines the position, or index, of a potential value in the following lines. Other commands, such as ifconfig on linux, have the keyword and value on the same line with the keyword (position:index) proceeding the value (position:index+1)

All this time I thought this was ONLY and AIX issue but in examining other platforms I see that some (e.g., Linux) do not have a macaddr value in netstat output - while others do (e.g., macos).

In short, I would appreciate the current PR being merged so that AIX has at least basic support. However, I am ready to continue, as a new PR, re-working the tests (so that a response of None is at least a warning) when a macaddr is not found for a platform - rather than over customization of test_uuid.py (as I have started to do here for AIX).

For platforms that do return a macaddr using netstat I would like to hear if that platform uses the "inline" method such as the command 'ifconfig' does on Linux, or if it is the "header" method such as on AIX and macos.

Thank you for your consideration.

[aixtools]

As I am not clear on where to have a more general discussion (in a PR conversation) or here - going to start here because I cannot figure out which comment in the PR to reply to.

Generally, before modifying the testuuid.py to based tests on uuid.\_NODE_GETTERS - these need to be defined.

I have my AIX systems, I found a macos I could do some queries on, and downloaded cygwin and came up with this starting point:

_MACOS = sys.platform == 'darwin'
_WIN32 = sys.platform == 'win32'
_CYGWIN= sys.platform == 'cygwin'
_AIX = sys.platform.startswith("aix")

...

if _AIX:
    _NODE_GETTERS = [_unix_getnode, _netstat_getnode]
elif _MACOS:
    _NODE_GETTERS = [_unix_getnode, _ifconfig_getnode, _netstat_getnode]
elif _CYGWIN:
    _NODE_GETTERS = [_ipconfig_getnode]
elif _WIN32:
    _NODE_GETTERS = [_windll_getnode, _ipconfig_getnode, _netbios_getnode]
else:
    _NODE_GETTERS = [_unix_getnode, _ifconfig_getnode, _ip_getnode,
                          _arp_getnode, _lanscan_getnode, _netstat_getnode]

What I am also wondering - is it worthwhile to have a way to only define the getter() routines a platform can actually use? e.g., On AIX I can call uuid._ipconfig_getter(), but get nonsense. Or is it too much effort?

Finally, can someone with access to other platforms where differences may be expected (e.g., Solaris, hpux, or even different flavors of Linux) - to make this _NODE_GETTERS mode complete (specific).

[aixtools]

p.s., removed 2.7 and 3.6 as too old for any interest.

[5f84803f-b5aa-49e0-ba9f-8c85a48183ef]

The current code and proposed changes use 'netstat -ia' to find the node however if netstat needs to perform a reverse DNS query to resolve some interfaces this makes using uuid1 *really* slow especially when reverse DNS queries aren't set up correctly or timeout. mac = _find_mac_netstat('netstat', '-ia', b'Address', lambda i: i)

Would it be possible to change the netstat call to add 'n' to the nestat options, i.e. _find_mac_netstat('netstat', '-ian', ...) to prevent netstat from attempting to resolve addresses?

[aixtools]

On 14/02/2019 23:57, Indra Talip wrote:

Indra Talip \indra.talip@gmail.com\ added the comment:

The current code and proposed changes use 'netstat -ia' to find the node however if netstat needs to perform a reverse DNS query to resolve some interfaces this makes using uuid1 *really* slow especially when reverse DNS queries aren't set up correctly or timeout. mac = _find_mac_netstat('netstat', '-ia', b'Address', lambda i: i)

Would it be possible to change the netstat call to add 'n' to the nestat options, i.e. _find_mac_netstat('netstat', '-ian', ...) to prevent netstat from attempting to resolve addresses? Sounds like an excellent idea. Gives me a reason to get started again. Thx! ---------- nosy: +italip

---

Python tracker \report@bugs.python.org\ \https://bugs.python.org/issue28009\

---

[aixtools]

I have modified -

_NODE_GETTERS_WIN32 = [_windll_getnode, _netbios_getnode, _ipconfig_getnode]

_NODE_GETTERS_UNIX = [_unix_getnode, _ifconfig_getnode, _ip_getnode,
                      _arp_getnode, _lanscan_getnode, _netstat_getnode]

to:

+683 # _OS_GETTERS, when known, are targetted for a specific OS or platform. +684 # The order is by 'common practice' on the specified platform. +685 # Note: 'posix' and 'windows' _OS_GETTERS are prefixed by a dll/dlload() method +686 # which, when successful, means none of these "external" methods are called. +687 # _GETTERS is (also) used by test_uuid.py to SkipUnless(), e.g., +688 # @unittest.skipUnless(_uuid._ifconfig_getnode in _uuid._GETTERS, ...) +689 if _LINUX: +690 _OS_GETTERS = [_ip_getnode, _ifconfig_getnode] +691 elif _DARWIN: +692 _OS_GETTERS = [_ifconfig_getnode, _arp_getnode, _netstat_getnode] +693 elif _WINDOWS: +694 _OS_GETTERS = [_netbios_getnode, _ipconfig_getnode] +695 elif _AIX: +696 _OS_GETTERS = [_netstat_getnode] +697 else: +698 _OS_GETTERS = [_ifconfig_getnode, _ip_getnode, _arp_getnode, +699 _netstat_getnode, _lanscan_getnode] +700 if os.name == 'posix': +701 _GETTERS = [_unix_getnode] + _OS_GETTERS +702 elif os.name == 'nt': +703 _GETTERS = [_windll_getnode] + _OS_GETTERS +704 else: +705 _GETTERS = _OS_GETTERS

The shortened list, and in particular the move of _ip_getnode before _ifconfig_getnode is my experience that the "old" programs such as ifconfig, arp, and netstat are (occasionally) not available - with "ip" being the replacement for all.

Further, re: linux, on the two distros I could test (centos and debian) neither arp nor netstat return a (useable) MACADDR aka "node" value.

Requesting verification from people with other platforms.

Also, would like to know specifics for other platforms (e.g., OpenBSD, HPUX, Solaris).

More generally speaking - if os.name is "posix" or "windows" - this lists are almost irrelevant because the "DLL" _uuid module should provide the needed value.

The "plus" is that on systems that audit such things, there are fewer calls to non-existent programs and/or negative side-effects from calling programs that can/do not provide any useful data.

[aixtools]

p.s. - changed the title: way back when I first started on this I was mainly concerned that the _netstat_getnode() routine was broken for AIX. During the research and follow-up discussions it has become clear that it is more than just an AIX issue. There are multiple aspects that need attention.

Footnote: For most platforms, most of the data accessed via Lib/uuid is actually retrieved via Modules/_uuid. The majority of issues with Lib/uuid occur during testing: ./python -m test test_uuid

At least two PR (8672 - fix bug for AIX), (12777 - make "_getters" lists platform specific when possible). FYI: the first PR (5183) was when I was trying to patch multiple issues.

[ncoghlan]

New changeset 3a1d50e7e573efb577714146bed5c03b9c95f466 by Nick Coghlan (Michael Felt) in branch 'master': bpo-28009: Fix uuid SkipUnless logic to be based on platform programs capable of introspection (GH-12777) https://github.com/python/cpython/commit/3a1d50e7e573efb577714146bed5c03b9c95f466

[miss-islington]

New changeset f0e5c01182daefa20c624383c8a37c25eacfde43 by Miss Islington (bot) in branch '3.8': bpo-28009: Fix uuid SkipUnless logic to be based on platform programs capable of introspection (GH-12777) https://github.com/python/cpython/commit/f0e5c01182daefa20c624383c8a37c25eacfde43

[taleinat]

> The current code and proposed changes use 'netstat -ia' to find the node however if netstat needs to perform a reverse DNS query to resolve some interfaces this makes using uuid1 *really* slow especially when reverse DNS queries aren't set up correctly or timeout. > mac = _find_mac_netstat('netstat', '-ia', b'Address', lambda i: i) > > Would it be possible to change the netstat call to add 'n' to the nestat options, i.e. _find_mac_netstat('netstat', '-ian', ...) to prevent netstat from attempting to resolve addresses?

Sounds like an excellent idea. Gives me a reason to get started again. Thx!

I don't see this in PR python/cpython#72196; should this change be made?

[aixtools]

On 14/07/2019 22:28, Tal Einat wrote:
> Tal Einat <taleinat@gmail.com> added the comment:
>
>>> The current code and proposed changes use 'netstat -ia' to find the node however if netstat needs to perform a reverse DNS query to resolve some interfaces this makes using uuid1 *really* slow especially when reverse DNS queries aren't set up correctly or timeout.
>>>    mac = _find_mac_netstat('netstat', '-ia', b'Address', lambda i: i)
>>>
>>> Would it be possible to change the netstat call to add 'n' to the nestat options, i.e. _find_mac_netstat('netstat', '-ian', ...) to prevent netstat from attempting to resolve addresses?
>> Sounds like an excellent idea. Gives me a reason to get started again. Thx!
> I don't see this in PR python/cpython#72196; should this change be made?
I am testing with "-ian" - on top of the changes you have already made.
I'll push asap, assuming all goes well.
>
> 

---

nosy: +taleinat

---

Python tracker \report@bugs.python.org\ \https://bugs.python.org/issue28009\

---

[taleinat]

New changeset 0bcbfa43d55d9558cdcb256d8998366281322080 by Tal Einat (Michael Felt) in branch 'master': bpo-28009: Fix uuid.uuid1() and uuid.get_node() on AIX (GH-8672) https://github.com/python/cpython/commit/0bcbfa43d55d9558cdcb256d8998366281322080

[taleinat]

Michael, many many thanks for the time and effort that you have poured into getting this fixed, and for your patience and perseverance over the three years that it took to happen. Well done!

[vstinner]

New changeset 0bcbfa43d55d9558cdcb256d8998366281322080 by Tal Einat (Michael Felt) in branch 'master': bpo-28009: Fix uuid.uuid1() and uuid.get_node() on AIX (GH-8672)

This change introduced a regression: bpo-39991.

I pushed the commit eb886db1e99a15f15a2342aa496197a5f88fa9c8 to fix my case, but I'm not sure that the fix covers all cases.

[aixtools]

I'll take a look as well.

On 17/03/2020 16:14, STINNER Victor wrote:

STINNER Victor \vstinner@python.org\ added the comment:

> New changeset 0bcbfa43d55d9558cdcb256d8998366281322080 by Tal Einat (Michael Felt) in branch 'master': > bpo-28009: Fix uuid.uuid1() and uuid.get_node() on AIX (GH-8672) This change introduced a regression: bpo-39991.

I pushed the commit eb886db1e99a15f15a2342aa496197a5f88fa9c8 to fix my case, but I'm not sure that the fix covers all cases.

---------- nosy: +vstinner

---

Python tracker \report@bugs.python.org\ \https://bugs.python.org/issue28009\

---

[aixtools]

I may be mistaken, but I do not think the change introduced a regression.

While it is true that this case would not have appeared if there was still a count of the field-separators an IPv6 address with 5 ':' and 17 characters would have failed as well. The value being examined is based on it's position, and previously the additional assumption is that whatever is in that position is a MACADDR if it uses the system separator ('.' on AIX, ':' elsewhere).

If anyone is interested I will think about what are assumptions are - is this a value we should be considering, or not.

IMHO - while bpo-39991 is resolved - I am not -yet- convinced that the "root cause" has been identified and properly coded

Michael

On 17/03/2020 16:14, STINNER Victor wrote:

STINNER Victor \vstinner@python.org\ added the comment:

> New changeset 0bcbfa43d55d9558cdcb256d8998366281322080 by Tal Einat (Michael Felt) in branch 'master': > bpo-28009: Fix uuid.uuid1() and uuid.get_node() on AIX (GH-8672) This change introduced a regression: bpo-39991.

I pushed the commit eb886db1e99a15f15a2342aa496197a5f88fa9c8 to fix my case, but I'm not sure that the fix covers all cases.

---------- nosy: +vstinner

---

Python tracker \report@bugs.python.org\ \https://bugs.python.org/issue28009\

---

[vstinner]

I may be mistaken, but I do not think the change introduced a regression.

I'm talking about this: https://bugs.python.org/issue39991#msg364435

I don't want to blame anyone. My intent here is to get more eyes on the changes that I merged in bpo-39991 to make sure that I didn't break any existing cases, and that I covered all cases.

While it is true that this case would not have appeared if there was still a count of the field-separators an IPv6 address with 5 ':' and 17 characters would have failed as well.

Right, I pushed a second fix to also handle this case: commit ebf6bb9f5ef032d1646b418ebbb645ea0b217da6.

IMHO - while bpo-39991 is resolved - I am not -yet- convinced that the "root cause" has been identified and properly coded

If you still see cases which are not handled properly with commit ebf6bb9f5ef032d1646b418ebbb645ea0b217da6, feel free to reopen bpo-39991.

[aixtools]

On 18/03/2020 13:55, STINNER Victor wrote:

STINNER Victor \vstinner@python.org\ added the comment:

> I may be mistaken, but I do not think the change introduced a regression.

I meant - I had never considered IPv6 in the Address column, just as I suspect, whoever wrote the original.

Your feedback made me realize that something like "fe80::78:9a:de:f0" would have been mistaken as a valid macaddr.

I'm talking about this: https://bugs.python.org/issue39991#msg364435

I don't want to blame anyone. My intent here is to get more eyes on the changes that I merged in bpo-39991 to make sure that I didn't break any existing cases, and that I covered all cases.

I will look closely at PR19045 - not because I expect to find anything wrong, but because I thought this is what you requested.

Regards, Michael

> While it is true that this case would not have appeared if there was still a count of the field-separators an IPv6 address with 5 ':' and 17 characters would have failed as well.

Right, I pushed a second fix to also handle this case: commit ebf6bb9f5ef032d1646b418ebbb645ea0b217da6.

> IMHO - while bpo-39991 is resolved - I am not -yet- convinced that the "root cause" has been identified and properly coded If you still see cases which are not handled properly with commit ebf6bb9f5ef032d1646b418ebbb645ea0b217da6, feel free to reopen bpo-39991.

----------

---

Python tracker \report@bugs.python.org\ \https://bugs.python.org/issue28009\

---

[vstinner]

I will look closely at PR19045 - not because I expect to find anything wrong, but because I thought this is what you requested.

I cannot run functional tests on AIX. I can only rely on unit tests which contains a dump of AIX commands. That's why a review wouldn't hurt ;-)

[taleinat]

Your fix LGTM, Victor.