failures in test_uuid

[pitrou]

BPO	3581
Nosy	@tim-one, @amauryfa, @pitrou, @devdanzin, @skrah
Files	uuid_mac.patch test_uuid.patch test_uuid2.patch: apply on top of test_uuid.patch getnode.patch test_uuid3.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields: ```python assignee = 'https://github.com/skrah' closed_at = created_at = labels = ['type-bug', 'tests'] title = 'failures in test_uuid' updated_at = user = 'https://github.com/pitrou' ``` bugs.python.org fields: ```python activity = actor = 'devurandom' assignee = 'skrah' closed = True closed_date = closer = 'skrah' components = ['Tests'] creation = creator = 'pitrou' dependencies = [] files = ['16817', '16820', '16842', '16844', '16850'] hgrepos = [] issue_num = 3581 keywords = ['patch'] message_count = 20.0 messages = ['71283', '87937', '87941', '102601', '102608', '102626', '102632', '102712', '102718', '102727', '102763', '102775', '102929', '141143', '141144', '141146', '141147', '141148', '141150', '156668'] nosy_count = 8.0 nosy_names = ['tim.peters', 'nnorwitz', 'amaury.forgeotdarc', 'pitrou', 'ajaksu2', 'kleist', 'devurandom', 'skrah'] pr_nums = [] priority = 'high' resolution = 'accepted' stage = 'resolved' status = 'closed' superseder = None type = 'behavior' url = 'https://bugs.python.org/issue3581' versions = ['Python 2.6', 'Python 3.1', 'Python 2.7', 'Python 3.2'] ```

[pitrou]

I get failures in test_uuid when run inside a qemu virtual machine. It is related to the fake MAC address used by qemu.

\====================================================================== FAIL: test_ipconfig_getnode (main.TestUUID) ----------------------------------------------------------------------

Traceback (most recent call last):
  File "Lib\test\test_uuid.py", line 326, in test_ipconfig_getnode
    self.check_node(node, 'ipconfig')
  File "Lib\test\test_uuid.py", line 288, in check_node
    self.assertEqual(universal_local_bit, 0, message)
AssertionError: 525400123456 doesn't look like a real MAC address

====================================================================== FAIL: test_windll_getnode (main.TestUUID) ----------------------------------------------------------------------

Traceback (most recent call last):
  File "Lib\test\test_uuid.py", line 351, in test_windll_getnode
    self.check_node(uuid._windll_getnode(), 'windll')
  File "Lib\test\test_uuid.py", line 288, in check_node
    self.assertEqual(universal_local_bit, 0, message)
AssertionError: 525400123456 doesn't look like a real MAC address

---

[90baf024-6604-450d-8341-d796fe6858f3]

Hmm, is this a bug in uuid._ifconfig_getnode or just a bad test assumption?

[pitrou]

Apparently, a bad test assumption.

[5531d0d8-2a9c-46ba-8b8b-ef76132a492c]

Confirmed on Windows 7 under qemu with a fresh trunk checkout.

[5531d0d8-2a9c-46ba-8b8b-ef76132a492c]

According to http://standards.ieee.org/regauth/groupmac/tutorial.html , the assertions in check_node are weeding out perfectly valid addresses:

>>> node = 0x525400123456
>>> universal_local_bit = (node >> 40L) & 2
>>> universal_local_bit
2L

This just means that the address is locally administered, but valid. Similarly, shouldn't an individual_group_bit of 1 should just mean multicast mode?

See also: bpo-7650

[5531d0d8-2a9c-46ba-8b8b-ef76132a492c]

The new patch test_uuid.patch fixes bpo-7650, bpo-1481 and this one.

I've applied it to py3k-cdecimal in r79905 and test_uuid is passed on all buildbots (I did not test on ARM and one FreeBSD bot is still hanging in test_subprocess).

Additionally, the patch works on Windows-qemu (this issue) and OpenBSD-qemu.

Changes:

1. check_node accepts addresses with universal_local_bit set to 1. These addresses are valid (like the qemu one).

2. check_node accepts addresses with the multicast bit set to 1. These addresses are used for valid random addresses per the RFC and uuid.getnode() itself actually may return such a random address.

3. On some platforms, _uuid_generate_time can't be set, so _unixdll_getnode fails with a TypeError. This exception is now caught.

4. With these changes, there appear to be no more failures, so all tests are enabled again.

Could this go into trunk, since the tests actually fail needlessly on qemu setups?

[pitrou]

Ok, I trust you on the investigation.

Could this go into trunk, since the tests actually fail needlessly on qemu setups?

Yes, but after the beta: http://mail.python.org/pipermail/python-dev/2010-April/099132.html

[5531d0d8-2a9c-46ba-8b8b-ef76132a492c]

5. uuid.getnode() can fall back to creating a random 48 bit number and so can _windll_getnode() (by using UuidCreateSequential). Therefore, in unlucky cases 0xffffffffffff can be generated and the assert in check_node() will fail.

check_node() is beginning to look quite pointless...

[5531d0d8-2a9c-46ba-8b8b-ef76132a492c]

I do not understand the semantics of uuid.getnode(). Per the docs it's supposed to return a hardware address. This would reasonably make it a UUID version 1.

But then the random fallback should be a 47 bit number as per http://www.ietf.org/rfc/rfc4122.txt section 4.5.

The docs for UuidCreateSequential don't say if the random fallback is a random version 1 UUID.

[5531d0d8-2a9c-46ba-8b8b-ef76132a492c]

I reread http://www.ietf.org/rfc/rfc4122.txt and I'm pretty sure that if getnode() is supposed to return a hardware address, one of the following should be used:

1) If ifconfig etc. returns successfully, we are fine.

2) If uuid_generate_time() or UuidCreateSequential() are used to extract the node ID, we have to make sure that the returned UUID is in fact RFC_4122 and version 1.

3) The fallback node ID should be 47 bits according to section 4.5.

I've got a new patch (getnode.patch) that takes care of these issues.

[5531d0d8-2a9c-46ba-8b8b-ef76132a492c]

Sorry, the random node id generation in uuid.py is correct. The least significant bit of the first octet (which is set to 1) is the first one transmitted on the network, then the "low 47 random bits" follow.

I assumed that the least significant bit of the first octet was part of the low 47 bits.

Patch test_uuid3 addresses points 1-5 and is almost the same as test_uuid.patch, so I reset the resolution to 'accepted'.

As I said, there isn't much left that check_node() can check. One could possibly check that a MAC address retrieved by ifconfig does not have the multicast bit set, but what if a user has set the hardware MAC to a strange value? Correctly retrieving that value can't be regarded as a failure of uuid.py.

Should the patch go into the maintenance branches as well (after the beta is out)?

[pitrou]

Should the patch go into the maintenance branches as well (after the beta is out)?

Yes, I think it would be nice (though not mandatory).

[5531d0d8-2a9c-46ba-8b8b-ef76132a492c]

Applied test_uuid3.patch to trunk (r79954), release26-maint (r79959), py3k (r79960) and release31-maint (r79961).

No buildbot failures so far, so I'm closing this.

[a90cdf70-14a9-41cb-a463-46b235312c0d]

If it could be of interest to anybody:

When running "make test" after building Python 2.7.2, I get the error message "1 test failed: test_uuid".

uname -a Linux h1488277 2.6.18-028stab091.2 #1 SMP Fri Jun 3 00:02:40 MSD 2011 i686 athlon i386 GNU/Linux

This is a "virtual root server" (http://www.strato.de/server/virtual-linux-server/) which I believe is using "Parallels Virtuozzo Containers" (http://www.parallels.com/products/pvcl/).

[amauryfa]

What is the output of this command? ./python -m test.regrtest -v test_uuid

[a90cdf70-14a9-41cb-a463-46b235312c0d]

== CPython 2.7.2 (default, Jul 26 2011, 12:29:47) [GCC 4.2.1 (SUSE Linux)] == Linux-2.6.18-028stab091.2-i686-athlon-with-SuSE-10.3-i586 little-endian == /home/kjk/local/src/Python-2.7.2/build/test_python_18037 Testing with flags: sys.flags(debug=0, py3k_warning=0, division_warning=0, division_new=0, inspect=0, interactive=0, optimize=0, dont_write_bytecode=0, no_user_site=0, no_site=0, ignore_environment=0, tabcheck=0, verbose=0, unicode=0, bytes_warning=0) test_uuid testIssue8621 (test.test_uuid.TestUUID) ... ok test_UUID (test.test_uuid.TestUUID) ... ok test_exceptions (test.test_uuid.TestUUID) ... ok test_getnode (test.test_uuid.TestUUID) ... ok test_ifconfig_getnode (test.test_uuid.TestUUID) ... ERROR test_ipconfig_getnode (test.test_uuid.TestUUID) ... ok test_netbios_getnode (test.test_uuid.TestUUID) ... ok test_random_getnode (test.test_uuid.TestUUID) ... ok test_unixdll_getnode (test.test_uuid.TestUUID) ... ok test_uuid1 (test.test_uuid.TestUUID) ... ok test_uuid3 (test.test_uuid.TestUUID) ... ok test_uuid4 (test.test_uuid.TestUUID) ... ok test_uuid5 (test.test_uuid.TestUUID) ... ok test_windll_getnode (test.test_uuid.TestUUID) ... ok

\====================================================================== ERROR: test_ifconfig_getnode (test.test_uuid.TestUUID) ----------------------------------------------------------------------

Traceback (most recent call last):
  File "/home/kjk/local/src/Python-2.7.2/Lib/test/test_uuid.py", line 306, in test_ifconfig_getnode
    node = uuid._ifconfig_getnode()
  File "/home/kjk/local/src/Python-2.7.2/Lib/uuid.py", line 321, in _ifconfig_getnode
    mac = _find_mac('ifconfig', args, ['hwaddr', 'ether'], lambda i: i+1)
  File "/home/kjk/local/src/Python-2.7.2/Lib/uuid.py", line 311, in _find_mac
    words[get_index(i)].replace(':', ''), 16)
ValueError: invalid literal for int() with base 16: '00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00'

---

Ran 14 tests in 0.315s

FAILED (errors=1)
test test_uuid failed -- Traceback (most recent call last):
  File "/home/kjk/local/src/Python-2.7.2/Lib/test/test_uuid.py", line 306, in test_ifconfig_getnode
    node = uuid._ifconfig_getnode()
  File "/home/kjk/local/src/Python-2.7.2/Lib/uuid.py", line 321, in _ifconfig_getnode
    mac = _find_mac('ifconfig', args, ['hwaddr', 'ether'], lambda i: i+1)
  File "/home/kjk/local/src/Python-2.7.2/Lib/uuid.py", line 311, in _find_mac
    words[get_index(i)].replace(':', ''), 16)
ValueError: invalid literal for int() with base 16: '00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00'

1 test failed: test_uuid

[amauryfa]

hum, maybe an issue with the MAC address of your virtual server? What do you get if you run: ifconfig -a | grep -i -e hwaddr -e ether

[a90cdf70-14a9-41cb-a463-46b235312c0d]

/sbin/ifconfig -a | grep -i -e hwaddr -e ether venet0 Link encap:UNSPEC HWaddr 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00 venet0:0 Link encap:UNSPEC HWaddr 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00

[amauryfa]

Well, without a valid MAC address the function cannot work...

On the other hand, I would not worry too much: uuid._ifconfig_getnode() is an internal function; and since all the other tests pass, uuid.getnode() probably has other ways to get a unique identifier for the machine.

And by the way, in python 3.2, the test function is skipped with this message: """WARNING: uuid._ifconfig_getnode is unreliable on many platforms. It is disabled until the code and/or test can be fixed properly."""

[1562ced0-0aea-4dbd-a287-b8a12bfca88a]

Well, without a valid MAC address the function cannot work... It should not break in such ugly way either, imo.

On the other hand, I would not worry too much: uuid._ifconfig_getnode() is an internal function; and since all the other tests pass, uuid.getnode() probably has other ways to get a unique identifier for the machine. Here test_uuid fails due to this issue. Which in turn prevents my python3 rpm from being build.

And by the way, in python 3.2, the test function is skipped with this message: """WARNING: uuid._ifconfig_getnode is unreliable on many platforms. It is disabled until the code and/or test can be fixed properly.""" That does not seem to be true anymore in Python 3.2.2.

Would be nice if you could detect the all-zero case and just skip the test in that case.