Add tests for the weakref.WeakSet class in Python 3.0

[GoogleCodeExporter]

Python 3.0 has a new 'weakref.WeakSet' class, a set which only weakly
references its members.  There is a unit test for set types, in
'Lib/test/test_set.py', but it can't be applied to WeakSet because
it uses string elements, which in turn can't be weakly referenced.

Factor out new a set test base class that can be used for set, frozenset
and WeakSet, and appropriate subclasses that test the peculiarities of
set, frozenset and WeakSet. The existing classes should give you an idea
of how to generalize them.

Completion:

Attach the results of an 'svn diff' (context diff) against the Python
source tree to this task. Also include text containing the old and new
code coverage numbers and a brief description of the changes you've made.

Relevant wiki pages:

 - HowToTest
 - RunningPythonTests
 - GettingAndCompilingPython

Task duration: please complete this task within 5 days (120 hours) of claiming 
it.

Original issue reported on code.google.com by the.good...@gmail.com on 7 Dec 2007 at 7:23

[GoogleCodeExporter]

I claim this task.

Original comment by m.ottene...@gmail.com on 25 Dec 2007 at 1:38

[GoogleCodeExporter]

This task is Due December 30 at 14:00 UTC

Original comment by istvan.a...@gmail.com on 25 Dec 2007 at 2:18

• Changed state: Claimed
• Added labels: ClaimedBy-m.ottender, Due-20071239.1400

[GoogleCodeExporter]

This task is past due.  Please post a progress report if you intend to continue
working on it.

Original comment by doug.hel...@gmail.com on 4 Jan 2008 at 2:18

• Added labels: Due-20071230.1400, expired
• Removed labels: Due-20071239.1400

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

Hi! Sorry for the delay,holidays always makes me lazy. So I already digged a 
bit into
the code. I've noticed that there is already an extensive weakreft
test(Lib/test/test_weakref.py) I'm not sure if I am able to produce something 
better
than that.

But maybe I'm just misunderstanding this issue.

Original comment by m.ottene...@gmail.com on 5 Jan 2008 at 3:22

[GoogleCodeExporter]

If you look at it you will notice that it doesn't include tests for WeakSet :)

Original comment by georg.br...@gmail.com on 5 Jan 2008 at 7:53

[GoogleCodeExporter]

Of course I looked at it already. But obviously not close enough.
Thank you for the hint!

Original comment by m.ottene...@gmail.com on 5 Jan 2008 at 8:18

[GoogleCodeExporter]

Okay, I'll extend until 01-09.

Original comment by georg.br...@gmail.com on 6 Jan 2008 at 9:55

• Added labels: Due-20080109.1400
• Removed labels: Due-20071230.1400

[GoogleCodeExporter]

Original comment by the.good...@gmail.com on 8 Jan 2008 at 9:15

• Removed labels: expired

[GoogleCodeExporter]

Just too keep you up to date: I'm making good progress and I'll upload atleast 
my
current progress if not the finished code later on today

Original comment by m.ottene...@gmail.com on 9 Jan 2008 at 7:35

[GoogleCodeExporter]

Excellent, thanks!

Original comment by the.good...@gmail.com on 9 Jan 2008 at 7:36

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

But I have a problem understanding what the "ref" function does exactly.  It is 
used
in "weakref.py" the top  of th file says it is imported from _weakref. But I 
could
not find that file. I also tried googling but that did not help. May someone 
explain
it to me?
The function appears for instance here:

    def add(self, item):
        self.data.add(ref(item, self._remove))

I guess it returns weak references...but what for is the second argument then?

Oh and I've got another short question. Can all other objects except strings be
weakly referenced? And why strings not?

Original comment by m.ottene...@gmail.com on 9 Jan 2008 at 8:10

[GoogleCodeExporter]

I don't know the answer to your _weakref question, but here is how you can find 
out :)

>>> import _weakref
>>> _weakref
<module '_weakref' from '/usr/local/lib/python2.5/lib-dynload/_weakref.so'>
>>> _weakref.ref
<type 'weakref'>

Original comment by the.good...@gmail.com on 9 Jan 2008 at 8:11

[GoogleCodeExporter]

Well, I checked out 15minutes ago and I compiled with ./configure && make.(im 
on ox)
and I cant find out myself cause of this: 
tinuviel:py3k mc$ ./python.exe 
Python 3.0a2+ (py3k:59868, Jan  9 2008, 09:24:49) 
[GCC 4.0.1 (Apple Inc. build 5465)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import weakref
[41783 refs]
>>> weakref.ref
[41783 refs]
>>> 

Original comment by m.ottene...@gmail.com on 9 Jan 2008 at 8:29

[GoogleCodeExporter]

I don't get that on Linux; are you sure you haven't made any modifications or 
created
your own file called 'weakref.py'?

Original comment by the.good...@gmail.com on 9 Jan 2008 at 9:38

[GoogleCodeExporter]

Yes I'm sure. But sadly that problem does not seem to bee weakref specific I 
looked
at my logs and found out that make did not finish successfully.
This is what I get(with a  clean  checkout): 

gcc  -u _PyMac_Error -o python.exe \
            Modules/python.o \
            libpython3.0.a -ldl      
make: *** [sharedmods] Error 1

Original comment by m.ottene...@gmail.com on 9 Jan 2008 at 11:00

[GoogleCodeExporter]

re weakref.ref parameters: how about looking in the documentation --
http://docs.python.org/dev/library/weakref ?

Original comment by georg.br...@gmail.com on 9 Jan 2008 at 2:00

[GoogleCodeExporter]

Sorry!  I was looking for 3.0 and were not sure if the 2.6 doc is also suitable.

Original comment by m.ottene...@gmail.com on 9 Jan 2008 at 2:15

[GoogleCodeExporter]

Good point; the 3.0 docs are at 
<http://docs.python.org/dev/3.0/library/weakref>.

Original comment by georg.br...@gmail.com on 9 Jan 2008 at 2:18

[GoogleCodeExporter]

Well,this
http://groups.google.com/group/comp.lang.python/browse_thread/thread/3be2247add2
06736 
is delaying the completetion..sorry

Original comment by m.ottene...@gmail.com on 12 Jan 2008 at 3:25

[GoogleCodeExporter]

Well due my Python compiling problems I cant finish this Task at the moment. So 
I
drop it. 
Sorry!

Original comment by m.ottene...@gmail.com on 12 Jan 2008 at 7:06

[GoogleCodeExporter]

Sorry about that Michael!

Original comment by doug.hel...@gmail.com on 14 Jan 2008 at 1:44

• Changed state: Open
• Removed labels: ClaimedBy-m.ottender, Due-20080109.1400

[GoogleCodeExporter]

Can I stop claiming issue 280 and claim this instead? I thought this was taken, 
but
it evidently freed up! (Just in case: I claim this Task.)

Original comment by novan...@gmail.com on 17 Jan 2008 at 2:16

[GoogleCodeExporter]

As long as you don't make a habit of switching tasks, that's ok.

This task is due January 22, 2008 14:20:00 UTC.

Original comment by doug.hel...@gmail.com on 17 Jan 2008 at 2:55

• Changed state: Claimed
• Added labels: Due-20080122.1420, ClaimedBy-novanasa

[GoogleCodeExporter]

Well, there isn't much time left to make a habit of anything :-)

Thanks!

Original comment by novan...@gmail.com on 17 Jan 2008 at 2:59

[GoogleCodeExporter]

Can I just have a base class that is used for testing all three, then have the
current tests be called from that (since they are, in a sense, pecular aspects 
shared
by set and frozenset but not weakref.WeakSet), then just add weakref.WeakSet 
tests?

Original comment by novan...@gmail.com on 17 Jan 2008 at 10:10

[GoogleCodeExporter]

Oh, I should state that tests which make sense on WeakSet will be moved to the 
base
class as well.

Original comment by novan...@gmail.com on 17 Jan 2008 at 10:44

[GoogleCodeExporter]

I am having some problems with my WeakString implementation for migrating the 
tests
to WeakSet. As described in the weakref docs:
    Several builtin types such as list and dict do not directly support weak
references   
    but can add support through subclassing:

    class Dict(dict):
        pass

However, when I attempt this with String, strange stuff happens:

>>> from weakref import WeakSet as WS
>>> from weakref import ref
>>> class WeakString(str):                                      
...     pass
... 
>>> x = WS(WeakString('abcba')).union(WS(WeakString('cdc')))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.0/weakref.py", line 351, in __init__
    self.update(data)
  File "/usr/local/lib/python3.0/weakref.py", line 393, in update
    self.add(element)
  File "/usr/local/lib/python3.0/weakref.py", line 367, in add
    self.data.add(ref(item, self._remove))
TypeError: cannot create weak reference to 'str' object
>>> from weakref import ref
>>> ref(WS)
<weakref at 0x574210; to 'type' at 0x631ac0 (WeakSet)>
>>> ws1 = WeakString('hi')
>>> ws2 = WeakString('bye')
>>> ws1
'hi'
>>> ws2
'bye'
>>> r1 = ref(ws1)
>>> r2 = ref(ws2)
>>> r1
<weakref at 0x5b4f60; to 'WeakString' at 0x5a48b8>
>>> r2
<weakref at 0x5b4fc0; to 'WeakString' at 0x5a4998>
>>> ws1.__class__
<class '__main__.WeakString'>
>>> ws2.__class__
<class '__main__.WeakString'>
>>> x = WS(ws1).union(WS(ws2))                              
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.0/weakref.py", line 351, in __init__
    self.update(data)
  File "/usr/local/lib/python3.0/weakref.py", line 393, in update
    self.add(element)
  File "/usr/local/lib/python3.0/weakref.py", line 367, in add
    self.data.add(ref(item, self._remove))
TypeError: cannot create weak reference to 'str' object
>>> S = WS(ws1)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.0/weakref.py", line 351, in __init__
    self.update(data)
  File "/usr/local/lib/python3.0/weakref.py", line 393, in update
    self.add(element)
  File "/usr/local/lib/python3.0/weakref.py", line 367, in add
    self.data.add(ref(item, self._remove))
TypeError: cannot create weak reference to 'str' object

And yet I do the same thing with Dict and it works fine:

>>> class Dict(dict):
...     pass
... 
>>> obj = Dict(red=1, green=2, blue=3)
>>> ref(obj)
<weakref at 0x5bf060; to 'Dict' at 0x52d160>

Any ideas?

Original comment by novan...@gmail.com on 18 Jan 2008 at 2:28

[GoogleCodeExporter]

Ok, so I wanted to use figleaf to report test coverage, as required by the 
task, and
it requires setuptools, which doesn't have a version for Python 3.0. Should I 
use to
SVN versions or is there a better choice?

Original comment by novan...@gmail.com on 18 Jan 2008 at 5:29

[GoogleCodeExporter]

You *should* be able to run figleaf out of the distribution directory, e.g. get 
the
.tar.gz, unpack it, and run directly from bin/.

Original comment by the.good...@gmail.com on 18 Jan 2008 at 6:31

[GoogleCodeExporter]

I think I am going use coverage.py instead, simply because there are less py3k 
syntax
errors to change (every print statement, etc). Or just count it myself -- there 
are
more pressing issues to deal with.

Original comment by novan...@gmail.com on 18 Jan 2008 at 10:37

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

More interesting fun. This time, I just tried to emulate exactly what WeakSet 
does
when adding an item, and it worked:

>>> from weakref import WeakSet
>>> from weakref import ref
>>> class Str(str):
...     pass
... 
>>> ref(Str())
<weakref at 0x5cf600; dead>
>>> s = Str("hi")
>>> s
'hi'
>>> s.__class__
<class '__main__.Str'>
>>> ref(s)
<weakref at 0x5cf630; to 'Str' at 0x5cb420>
>>> set = WeakSet(s)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "~/py3k/Lib/weakref.py", line 351, in __init__
    self.update(data)
  File "~/py3k/Lib/weakref.py", line 393, in update
    self.add(element)
  File "~/py3k/Lib/weakref.py", line 367, in add
    self.data.add(ref(item, self._remove))
TypeError: cannot create weak reference to 'str' object
>>> ref(s,WeakSet()._remove)
<weakref at 0x5cf6f0; to 'Str' at 0x5cb420>
>>> st = set()
>>> st.add(ref(s,WeakSet()._remove))
>>> st
{<weakref at 0x5cf630; to 'Str' at 0x5cb420>}
>>> set = WeakSet(Str())
>>>

Is there something I am missing as to why Str can't be put in a WeakSet?

Original comment by novan...@gmail.com on 18 Jan 2008 at 11:18

[GoogleCodeExporter]

WeakSet(s) creates a set from the *items* of s; that is, for Str("hi"), "h" and 
"i".
Try WeakSet([s]).

Original comment by georg.br...@gmail.com on 19 Jan 2008 at 9:58

[GoogleCodeExporter]

Yeah, I want the items of s to be in the the WeakSet. I found a solution -- 
redefine
the __setitem__ and __getitem__ functions to return Str's instead of str's.

Original comment by novan...@gmail.com on 19 Jan 2008 at 2:46

[GoogleCodeExporter]

This version basically wraps all tests in a WeakString, which makes a string 
into a
list of ints. I am going to implement some class other than Int to actually 
work,
since it does not.

Original comment by novan...@gmail.com on 20 Jan 2008 at 1:35

Attachments:

• test_set.diff

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

Here is a new version, which fails just as badly. Am I allowed to add methods to
WeakSet? It seems quite difficult to test WeakSet instances without either 
getting a
reasonable repr or having an eq method. The only other thing I can think of is
running a symmetric_difference and seeing if it is equal to self.thetype().

Original comment by novan...@gmail.com on 20 Jan 2008 at 2:11

Attachments:

• test_set2.diff

[GoogleCodeExporter]

Ok, because of the way that WeakSet is implemented, it isn't possible to 
succeed with
the current difference/or/sub/etc tests -- it always accepts general iterators:

>>> from weakref import WeakSet
>>> from UserString import UserString
>>> s = list(map(lambda x: UserString(x), 'simsalabim'))
>>> s
['s', 'i', 'm', 's', 'a', 'l', 'a', 'b', 'i', 'm']
>>> set = WeakSet(s)
>>> set
<weakref.WeakSet object at 0x5d8610>
>>> set.data
{<weakref at 0x5cff60; to 'UserString' at 0x5d8090>, <weakref at 0x5cffc0; to
'UserString' at 0x5d84b0>, <weakref at 0x5cff00; to 'UserString' at 0x5d8170>,
<weakref at 0x5cff30; to 'UserString' at 0x5d85b0>, <weakref at 0x5cff90; to
'UserString' at 0x5d8450>, <weakref at 0x5cfea0; to 'UserString' at 0x5d8110>}
>>> ow = list(map(lambda x: UserString(x), 'madagascar'))
>>> i = set.difference(ow)
>>> i
<weakref.WeakSet object at 0x5d83b0>
>>> i.data
{<weakref at 0x5cff00; to 'UserString' at 0x5d8170>, <weakref at 0x5cffc0; to
'UserString' at 0x5d84b0>, <weakref at 0x5cff90; to 'UserString' at 0x5d8450>}
>>> (set-ow).data
{<weakref at 0x5cff00; to 'UserString' at 0x5d8170>, <weakref at 0x5cffc0; to
'UserString' at 0x5d84b0>, <weakref at 0x5cff90; to 'UserString' at 0x5d8450>}

The easiest (and only, for that matter, that I know of) way to fix this is to 
just
call TestAllOps.test_* from TestJointOps.test_* and then add in the relevant 
code to
TestJointOps. This could get nasty...

Original comment by novan...@gmail.com on 20 Jan 2008 at 4:31

Attachments:

• test_set3.diff

[GoogleCodeExporter]

Ok, WeakSet has a bug. union should be called with _apply, not _apply_mutate. It
doesn't change the set object, so for all practical purposes it is ignored. 
Should I
provide a patch for that as well? Pretty much other than that (and writing up 
code
coverage, which is pretty simple given the number of lines that WeakSet is), I 
am done.

Original comment by novan...@gmail.com on 20 Jan 2008 at 4:57

[GoogleCodeExporter]

Whoops, the required diff and proof:
>>> from UserString import UserString
>>> s = list(map(lambda x: UserString(x), 'simsalabim'))
>>> o = list(map(lambda x: UserString(x), 'madagascar'))
>>> from weakref import WeakSet
>>> set = WeakSet(s)
>>> set
<weakref.WeakSet object at 0x5d84b0>
>>> set.data
{<weakref at 0x5cff90; to 'UserString' at 0x5d8030>, <weakref at 0x70e030; to
'UserString' at 0x5d85b0>, <weakref at 0x5cff30; to 'UserString' at 0x5d84d0>,
<weakref at 0x5cff60; to 'UserString' at 0x5d8510>, <weakref at 0x5cffc0; to
'UserString' at 0x5d8070>, <weakref at 0x5cfbd0; to 'UserString' at 0x5d8470>}
>>> u = set.union(o)
>>> u
>>> set.data
{<weakref at 0x5cff90; to 'UserString' at 0x5d8030>, <weakref at 0x70e030; to
'UserString' at 0x5d85b0>, <weakref at 0x5cff30; to 'UserString' at 0x5d84d0>,
<weakref at 0x5cff60; to 'UserString' at 0x5d8510>, <weakref at 0x5cffc0; to
'UserString' at 0x5d8070>, <weakref at 0x5cfbd0; to 'UserString' at 0x5d8470>}

The set wasn't changed and didn't return a new value. Hmm...

The rest is ready.

Original comment by novan...@gmail.com on 20 Jan 2008 at 5:04

Attachments:

• test_set4.diff

[GoogleCodeExporter]

Ok, I set everything up as patches to SVN tree and wrote the testing document. I
could not get the update functions to work on WeakSets at all, so I couldn't 
test
them. I also have the coverage document.

Original comment by novan...@gmail.com on 21 Jan 2008 at 4:51

Attachments:

• weakrefpatch.diff
• coverage.txt
• full-diff.diff

[GoogleCodeExporter]

I've tested your patch, and it works! Setting the task to completed, I'll 
commit it
as soon as we get a contributor agreement from you. (If you don't know about 
the CA,
please mail the list about it.)

Original comment by georg.br...@gmail.com on 21 Jan 2008 at 4:41

• Changed state: Completed
• Added labels: NeedsCommit
• Removed labels: Due-20080122.1420

[GoogleCodeExporter]

I sent one out a few weeks ago, so it should be arriving pretty soon.

Original comment by novan...@gmail.com on 21 Jan 2008 at 5:59

[GoogleCodeExporter]

Original comment by georg.br...@gmail.com on 18 May 2008 at 4:22

• Removed labels: NeedsCommit