Pass shape and strides at construct-time via introspection

[arteymix]

This is currently not possible because we use size_t* and ssize_t* types.

[rainwoodman]

What would be a possible type to pass in?

[arteymix]

The idea I had would be to edit the introspection file to teach PyGObject how to obtain the array size from dimension. I'm pretty sure it's possible, but there's no thorough documentation out there on the format.

[rainwoodman]

I am still not quite sure where the problem is.

Is it about passing the _new functions arrays [1], or about passing arrays to g_object_newv?

If it is _new functions, could you try add a _new_arrays where you do not annotate [CCode has_length=false] to the arguments?

[1] https://wiki.gnome.org/Projects/PyGObject/IntrospectionPorting#Passing_arrays

[rainwoodman]

If it is the second case, maybe emailing Simon to see if he has any advices? Are there any GTK widgets that takes an array as a input?

[rainwoodman]

Another possibility, is to make use of the construct block:

• define shape and strides in as pointer properties, construct only, saving to _in_shape and _in_strides

• getter will return _shape and _strides which are arrays; before construct block is called the value in these arrays are undefined.

• in construct block, (where we know dimension), copy dimension entries from _in_shape and _in_strides to _shape and _strides.

I was doing that at some point and it worked at vala level. I am not sure if this plays nice with GI + Python, but from what I read it seems to be quite promising -- GI will turn Python List to a pointer, keeping the reference till the constructor returns, and the construct block makes a copy into _shape and _strides.

[arteymix]

In PyGObject, there's no _new function being used. It's pure g_object_newv calls. The problem is that the property is not defined as an array, so there is no introspection generated for that from the Vala compiler.

We want to keep the pointers actually because for C and Vala, it's basically the same and it's very convenient.

Maybe we could mail Simon to see what kind of introspection is necessary in that case?

[rainwoodman]

I just did this

public class Vast.TestPointer : Object
{
    size_t * _in_shape;
    size_t _shape[32];

    public size_t * shape {
        get {return _shape;}
        construct { _in_shape = value;}
    }
    public size_t dimension {get; construct;}
    construct {
        for(var i = 0; i < dimension; i ++) {
            _shape[i] = _in_shape[i];
        }
    }

    public TestPointer(int dimension, size_t [] shape)
    requires(dimension == shape.length)
    {
        base(dimension: dimension, shape : shape);
    }
}

and

s = Vast.TestPointer.new(dimension=3, shape=[1, 2, 3])

works.

So in principle we can override Vast.TestPointer to use Vast.TestPointer.new instead -- by passing g_object_newv from Python.

But there is a bug with PyGObject that you can't pass python type into a function as a GType (but you can pass it to a constructor) -- I was to submit it but bugzilla.gnome gives me a 500 error.

On the other hand, I don't think you can easily teach GObject about a property is an array. The infrastructure is just not there: I searched GTK gir file and could only find use cases for string arrays, which I believe has always been specially cased. The syntax for string arrays are really simple: """ <property ....>

"""

But I don't think it is up to us to fix this in GIR..

[rainwoodman]

I'll copy my unsumitted bug report here for reference and so I don't lose it when I close the tab.

---

When a property is declared as GObject.Type, there is auto conversion from Python type to GType in the GObject constructor function.

When a parameter to any other function is declared as a GObject.Type, calling the function raises an error TypeError:

Traceback (most recent call last): File "/home/yfeng1/source/vast/tests/gi-test.py", line 5, in Vast.print_type(int) TypeError: Must be number, not type

The example function in vala is

namespace Vast
{
    public void print_type(Type type)
    {
        message("%p\n", (void*)type);
    }
}

and python code calling this is

from gi.repository import Vast

Vast.print_type(int)

The relevant gir file segment is

   <function name="print_type" c:identifier="vast_print_type">
      <return-value transfer-ownership="full">
         <type name="none"/>
      </return-value>
      <parameters>
         <parameter name="type" transfer-ownership="none">
            <type name="GObject.Type" c:type="GType"/>
         </parameter>
      </parameters>
   </function>

This does smell like a bug in PyGObject, but I cannot rule out it could be somewhere in GIR. Any hints?

[rainwoodman]

No it is actually a bug in vala -- the gir file shall contain GType not GObject.Type.

Is there a way to patch gir file with meson?

[arteymix]

Sure, we just need to to a XSLT transformation on the XML file via a custom_target. That's what I planned to do to get some decent introspection.

[rainwoodman]

I submitted a patch to vala.

https://bugzilla.gnome.org/show_bug.cgi?id=775591

Is this the proper way of doing things or is there a github project that we can file PR?

[rainwoodman]

So if this is fixed, a workaround is then replace new with a pygobject override that calls .new_full()?

[arteymix]

The sanest fix would be to expose shape and strides as proper array in the introspection. If we cannot do that, we'll have to use dummy intermediary constructor.

[arteymix]

Maybe we could have explicit getter and setter and set the size? However, this won't result in a GObject property.

We could write that piece in C if that matters. I don't know if Vala knows how to deal with extern properties.

[astavale]

I don't really understand what you are trying to achieve, but from my superficial reading of the Vala bug report and the comments above I think this document may be helpful: https://wiki.gnome.org/Projects/PyGObject/IntrospectionPorting The most relevant parts are likely to be the sub-sections on 'Constructors' and 'Passing arrays'. This document is written by probably the same Simon you are thinking of emailing, so you may already have seen it.

[rainwoodman]

@astavale we are passing arrays of ssize_t via Object.new from GI. The problem is two fold:

• GObject doesn't support array property
• VALA doesn't expose the underlying ssize_t via GI to the property (it is declared as gpointer), thus PyGObject cannot cast python sequence-like into the proper type

The functional interface (_new helpers) works fine, though there is a bug in Vala GIR writer that stops another parameter (GType) due to gnome bz775591 I previously submitted.

[rainwoodman]

@arteymix

I thought we have been trying really hard to make sure Array is GObject compatible.

If we don't have to go through that, we can probably just give up on GObject and use a vala class instead? We'll lose the construct block though (if I remember correctly).

It appears to me to use GObject.new in Python, we just need a patch to GObject and GIR and VALA and PYGObject to add proper support of element type of an array/pointer property, which could be relatively simple if we know where to hack.

I am not sure how 'open' the devs are about these patches though.

[rainwoodman]

But for now let's just try the pygo override trick first.