Find a way to clean up API

[ronaldoussoren]

It would be nice to optionally use a more swift-like naming convention with PyObjC, but doing that while maintaining backward compatibility and performance will be hard.

1. Expose @property properties as properties instead of requiring to use the getter/setter methods. Doing this is backward compatibility break because unlike ObjC Python doesn't have a separate namespace for properties (note: see the "_" special attribute on instances, but not yet on classes)

2. Swift exposes shorter names for enums in the enum namespace, that is ObjC (and Python): NSColorTypeComponentBased, Swift: NSColorType.ComponentBased

   Not sure yet if this will be useful for Python. At the very least this will help with introspection (and possibly tab completion, although the modern names already support this by way of having a common prefix)

   Extendable enums are problematic here. Performance is also an issue.

3. Python 3.6 has OrderedDictionaries by default (more or less, and definitely not ABI), this should make it easier to following the Swift naming convention at least for calling methods (especially when coupled with first class @property support).

   Not sure if this can be done performantly and if this can also be done for declaring methods.

   A challenge will be effectively overloaded versions of methods.

[ronaldoussoren]

2 should already work with the current long names (NSColorType should show a list of completions). The major win w.r.t. how the names are exposed in Swift is that it cleans up the global namespace (which PyObjC won't be able to do initially due to backward compatibility concerns), and better documentation. In Swift you can als leave out the type name in a lot of cases (e.g. use .ComponentBased because the compiler can deduce the type used based on other information), but that's not possible in Python.

I'm slightly more interested in 1 and 3 because that will allow for cleaner code (by doing away with the idiotically long names we have to use now for 3).

Anyways.... it might be a while before I actually find time to work on this and backward compatibility may well be a deal breaker here.

[schriftgestalt]

Would it be possible to enable those properties with a switch in from __future__ import?

It would save me a lot boilerplate code in my wrapper. I have a lot stuff like this:

GSDocument.font = property(lambda self: self.pyobjc_instanceMethods.font(),
                            lambda self, value: self.setFont_(value))

[ronaldoussoren]

A __future__ import would not work here because there has to different behaviour for classes and future statements affect a module (even ignoring that I can't add a new future import in PyObjC).

In theory it could be possible to hack something together using sys._getframe() and introspection of the code object attached to the frame of the caller, but that's a crude hack and likely very slow.

The best option I've found so far is a project-wide switch, something like:

import objc
objc.enable_properties = True

Implementing that is doable, but likely requires changes to the framework bindings as well because not all ObjC properties are marked as such in the ObjC headers and runtime.

[schriftgestalt]

That option looks very good.

[ronaldoussoren]

Maybe, but it is also problematic in that it can fragment an already small community with code that works either with this setting on or with the setting off, but not both. Supporting both options in a library (such as the utility libraries in PyObjC itself) is more work and hard to get right unless setting the option is done before any other code using PyObjC is imported.

The main problem here is that Python has too few namespaces ;-). In ObjC all of these refer to different things:

[object name];    // -[SomeClass name] (instance method)
object.name;      // \@property
object->name;   // instance variable
[[object class] name]   // +[SomeClass name] (class method)

In Python these are all mapped on the same construct: object.name. This is already visible in plain PyObjC by way of the description method:

anObject = NSObject.new()
anObject.description()
NSObject.description()

Both calls work, anObject.description is a bound instance method, and NSObject.description is a bound class method (and not an unbound instance method as it would be in a plain Python class).

I have been thinking about exposing ObjC properties as Python properties, but haven't found a way yet that won't break a lot of existing code. There are basically two options:

1. Live with the status quo. Not nice, but avoid code churn for users of PyObjC
2. Bite the bullet and start a migration, that is introduce the option mentioned earlier, deprecate setting it to False in a future version, remove the option again in yet another version.

I might pick option 2 because it leads to nicer code, but probably only after asking the community about this and some other changes I'm thinking about (but haven't been able to fully flesh out).

[schriftgestalt]

I see all the problems with backwards compatibility. And don’t have a good idea about it.

But what would help me a lot is a more streamlined way to have a nicer, more pythonic API for my own classes.

[schriftgestalt]

Just to illustrate what I’m speaking about: https://github.com/schriftgestalt/GlyphsSDK/blob/Glyphs3/ObjectWrapper/GlyphsApp/__init__.py

[ronaldoussoren]

Thanks for the reference.

One of the things I really want to look into is a way of introducing more pythonic aliases for ObjC methods, to have an option for replacing those long and ugly selector names with something cleaner. With some luck I can come up with a mostly generic algorithm for generating those and use that in the framework wrappers. Likewise for generating __init__ and __new__ methods. I really want to avoid having to write all of those by hand, that's undoable for me.

That doesn't solve the issue with properties, but could help sell a change in code using PyObjC.

But the hard part of this is actually doing the work. I theoretically have more time to work on PyObjC since the start of the pandemic, but in practice that didn't materialise because the whole pandemic situation is mentally hard and other stuff eats up time (for example, I no longer have a commute, but do spent at least as long on bike riding to stay fit).

[schriftgestalt]

if you have some stuff to delegate, I could help (or get help).

[ronaldoussoren]

I'm planning to spent time on this during the year, likely going for the option I described earlier: A runtime switch that exposed ObjC properties as Python properties, defaulting to off for now (but with the intention to changing that later).

Current plan is to start with some design work for the "Improved user friendliness" milestone I just created with which I can ask the community for feedback before committing to these changes.

[ronaldoussoren]

It might be possible to avoid a flag when turning Objective-C properties in Python properties. The script below contain a fairly crude implementation of a property that can be used both as a property and as a getter method. This requires a lot more work, and testing, to turn something that can be used in production. But I'm fairly optimistic that it is possible to use something like this for real.

The longer term plan would then be:

• Add a new warning class, something like class DeprecatedGetter(DeprecationWarning): pass
• Add the production-ready variant of the code below in PyObjC 9 and use for all properties
• Make the new warning silent by default
• Add documentation on how to convert code and how to turn on printing a message for the new warning
• In some future version PyObjC 11, maybe even PyObjC 10, remove the new mechanism and only allow using properties as. properties (no more getters)

Actions:

• [ ] Create production ready version of callable_proxy, with tests including tests touching the C API

  This might be a stumbling block, the "result" of a property that returns (for example) a string should be usable with CPython APIs that deal with strings, in particular with PyArg_Parse* and whatever is used by argument clinic.

• [ ] Update metadata tooling to find properties and generate updated framework bindings
• [ ] Add tests for this to the framework bindings
• [ ] Update documentation (as described earlier)
• [ ] Create followup issue for removing the backward compatibility code

class callable_proxy:
    __slots__ = ('_value')

    def __init__(self, value):
        self._value = value

    def __call__(self):
        return self._value

    def __getattribute__(self, nm):
        if nm == '_value':
            return object.__getattribute__(self, nm)

        try:
            return self._value.__getattribute__(nm)

        except AttributeError:
            return object.__getattribute__(self, nm)

    def __setattr__(self, nm, value):
        if nm == '_value':
            return object.__setattr__(self, nm, value)
        return self._value.__setattr__(nm, value)

class maybe_property:
    def __init__(self, getter, setter=None):
        self._getter = getter
        self._setter = setter

    def __get__(self, instance, owner):
        result = self._getter(instance)
        return callable_proxy(result)

    def __set__(self, instance, value):
        if self._setter is None:
            raise TypeError("readonly property")

        return self._setter(instance, value)

    def setter(self):
        def update_setter(func):
            self._setter = func
        return update_setter

class Object:
    @maybe_property
    def description(self):
        return f"<Object at 0x{hex(id(self))}"

o = Object()
print(o.description.upper())
print(o.description().upper())
print(dir(o.description))