metaclasses, __getattr__, and special methods

[9565fb43-c98c-43f8-b90c-6b193e4e3149]

BPO	729913
Nosy	@mwhudson, @arigo, @birkenfeld

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields: ```python assignee = None closed_at = created_at = labels = ['interpreter-core', 'invalid'] title = 'metaclasses, __getattr__, and special methods' updated_at = user = 'https://bugs.python.org/bpettersen' ``` bugs.python.org fields: ```python activity = actor = 'arigo' assignee = 'none' closed = True closed_date = None closer = None components = ['Interpreter Core'] creation = creator = 'bpettersen' dependencies = [] files = [] hgrepos = [] issue_num = 729913 keywords = [] message_count = 4.0 messages = ['15766', '15767', '15768', '15769'] nosy_count = 4.0 nosy_names = ['mwh', 'arigo', 'georg.brandl', 'bpettersen'] pr_nums = [] priority = 'normal' resolution = 'not a bug' stage = None status = 'closed' superseder = None type = None url = 'https://bugs.python.org/issue729913' versions = [] ```

[9565fb43-c98c-43f8-b90c-6b193e4e3149]

__getattr__ on metaclasses aren't called when it would seem "logical" \<wink> for it to be. E.g.:

>>> class meta(type):
...   def __getattr__(cls, name):
...     if name == '__len__':
...        print "meta.__getattr__('__len__')"
...        return lambda: 42
...     else:
...        print 'meta.__getattr__', name
...        return name
...
>>> class S(object):
...   __metaclass__ = meta
...
>>> S.__len__()
meta.__getattr__('__len__')
42
>>> len(S)
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
TypeError: len() of unsized object
>>>

I was told that special method "foo(x, arg)" was implemented as "type(x).__foo__(x, arg)", which doesn't seem to be the case always... Compare:

>>> class meta(type):
...   def __len__(cls):
...     return 42
...
>>> class S(object):
...   __metaclass__ = meta
...
>>> S.__len__()
42
>>> len(S)
42
>>>

So, it looks like it's looking up __len in the metaclass, but not falling back on __getattr when it isn't there? I've looked at the C code and it seems like special methods each have their own way of finding the function they're needing.

From Alex Martelli: Ah yes, I see now! Yes, functions such as len() rely on slots in the type object, e.g. as you've noticed:

finding the function they're needing, e.g. for len, it looks like it uses:

m = o->ob_type->tp_as_sequence; if (m && m->sq_length) return m->sq_length(o);

return PyMapping_Size(o);

and the "incredibly complex thinking" (quoting from typeobject.c) in update_one_slot doesn't seem to work except for operations the which "the class overrides in its dict" (again from a comment in typeobject.c, this one for fixup_slot_dispatchers).

The issue may be with _PyType_Lookup (again in the same ,c file), which just gives up if it can't find a name somewhere along the mro (it doesn't "look upwards" to the metaclass) while type_getattro DOES work upwards on the metaclass too. Hmmmm.
I'm not sure I really understand all that's going on here - it IS a rather hairy zone of the code. Maybe you can post this as a bug in 2.3 beta 1 on sourgeforge (ideally showing where in the docs it defines the semantics that it then doesn't respect) so we can get this looked at by the few people who really DO grasp these parts...;- ). There is probably some sound implementation reason for the current behavior, but if so it should be better documented, I think.

Back to me: The point being that I haven't found any place in the documentation that defines what the attribute lookup is on new-style classes (and the C code is too hairy for me to understand :-)

As a special case of this problem, super() can't create an object which intercepts the special methods like it does for other methods, e.g.:

super(MyCls, self).__getitem__(5)

works, but not

super(MyCls, self)[5]

I don't know if that is intended or not, but it's not documented, though neither is exactly _what_ super is? (i.e. it looks like it's an object, that when you call a method, 'm', on it, uses the superclass method 'm', but the subclass versions of all other methods, although as above, not in all contexts, and I'm not sure whether you're supposed to be able to treat it as a first class object [pass as arg, return, etc])....

-- bjorn

[mwhudson]

Logged In: YES user_id=6656

You could try

http://starship.python.net/crew/mwh/hacks/oop-after-python22.txt

(or attach pdf to the end instead...)

You say:

The point being that I haven't found any place in the documentation that defines what the attribute lookup is on new-style classes

That's not the problem -- attribute lookup is fairly easy. What you're missing is that attribute lookup != special method lookup.

This probably should be in the core documentation, yes.

[birkenfeld]

Logged In: YES user_id=1188172

I closed bpo-789262 as a duplicate of this one. More info may be there.

[7aa6e20b-8983-474f-b2ae-de7eff1caa04]

Logged In: YES user_id=4771

This is a known documentation bug: all this is expected, but under-documented. Indeed, len(x) calls the special method __len of 'x', but what is not specified is the real definition of "calling a special method" on an object 'x': it is to look up the name "__len" in the dict of type(x), then in the dict of the parent types in MRO order. It's really not the same thing as an attribute lookup.