Support for image loading from Python data with QQuickImageProvider

[M4rtinK]

First I would like to say I really like pyotherside and I'm looking forward to using it for a Qt5 GUI for both modRana and Mieru (for Sailfish and beyond :) ).

I've looked through the code and so far there seems to be only one important thing missing for my use-cases and that is being able to load images to QML from Python data objects. The use-cases look like this:

Tile loading in modRana

• modRana downloads tiles from the internet and stores them in a sqlite database using Python
• the QML map widget sends tile loading requests (using the Image element source property for the tiles) and the Python code fetches the tiles from the database (or from the Internet) in the form of Python data object, which is then converted to QImage and sent to the QML context
• the current Qt4 GUI is using the QDeclarativeImageProvider through PySide to do this

Page loading in Mieru

• the Python code uses zipfile/tarfile/rarfile to fetch single images from comic book/manga archives as needed
• when the user turns a page, a new source property for the Image comprising the page is set, Python gets a request, fetches the image to Python data object (string of file like object, not sure at the moment), converts it to QImage and sends it to the QML context
• again, the current Qt4 GUI uses the QDeclarativeImageProvider to accomplish this

Currently, these two (and other similar) use-cases doesn't seem to be possible with pyotherside (if I disregard such horrible hacks as creating a temporary file for each image and using file path loading or running a local webserver to abuse image loading over network).

This is how I think the current situation could be improved:

• the C++ code registers a single QQuickImageProvider for some keyword (lets say "python")
• the Python code would get the id & requested size and will return a string or a filelike object
• once an Image has its source property set to correspond to to the "python" image provider:
  • the provider calls the Python-side callback and passes it the id and requested size
  • the Python code does what needs to be done and returns the string/file-like object
  • the provider converts the string/file object to QImage and returns it to the QML context I hope I didn't forget any obvious "gotchas" that would prevent this from working. :)

How could it look like ?

Simple example - Python:

import pyotherside

def get_image_data(image_id, width, height):
    # this is just a simplified example how you can get
    # some image data in Python, but in a real
    # application the data might come from a blob
    # in a sqlite database, from a zipfile, 
    # a rendering library, etc.
    f = open("images/%s.png" % image_id , "rb")
    data = f.read()
    f.close()
    return data

pyotherside.register_image_data_callback(get_image_data)

Simple example - QML side:

import QtQuick 2.0
import io.thp.pyotherside 1.0

Image {
    id : imageFromPython
    width: 200
    height: 200
    source : ""

    Python {
        // make sure the Python code
       // gets a chance to register the callback
        id: python
        Component.onCompleted: {
            addImportPath('examples/atexit');
            importModule_sync('main');
            // callback should be now registered
            // and source can be set
            imageFromPython.source = "image://python/foo"
        }

        onError: console.log('error in python: ' + traceback);
    }
}

Sure, advanced stuff like registering multiple providers with custom keywords could be also supported, but one hardcoded provider & one callback is IMHO all that would be needed for now.

So, what do you think think about this enhancement proposal ? :)

[thp]

I like it very much, well thought through. If you structure your code well enough, it might even be possible to still declaratively set the image source without having to procedurally set it afterwards (like in your example). This could be accomplished by instantiating the Python object very early (and loading the module) and then e.g. only loading images from sub-pages that get loaded as dynamic QML object after the Python loading is done. But that can be mostly solved in user code.

The API design looks fine. I assume the string from Python would then be passed to a QImageReader from which the image is obtained? You probably have to make Image loading on the QML side asynchronous to get a responsive UI (image loading requests could only be synchronous I think). Would that work for you?

[M4rtinK]

I like it very much, well thought through.

Thanks ! :)

If you structure your code well enough, it might even be possible to still declaratively set the image source without having to procedurally set it afterwards (like in your example). This could be accomplished by instantiating the Python object very early (and loading the module)

Good point! It makes sense - as there is of course only one Python context & one instance of the QML side plugin, you just have to "configure" the Python callback just once at startup and it should "just work" for all Image elements using it afterwards.

and then e.g. only loading images from sub-pages that get loaded as dynamic QML object after the Python loading is done. But that can be mostly solved in user code.

Is there something preventing the already instantiated Image elements from seeing the provider or the Python object ? Of course you shouldn't set their source property to the URI for the provider until everything is properly setup.

Still, this is probably moot, as one should load as much as possible dynamically anyway, to make the application startup as fast as possible.

The API design looks fine. I assume the string from Python would then be passed to a QImageReader from which the image is obtained?

Yes, exactly. For example, this is how it is currently done in Mieru with PySide:

imageFileObject = page.popImage()
img = QImage()
img.loadFromData(imageFileObject.read())

imageFileObject object is the file-like object returned by Zipfile/Rarfile for the given page image from the current archive img is the resulting QImage that is returned by the ImageProvider subclass

It's basically the same for modRana tile loading, with the only difference being that it is working directly with Python strings. The strings might come from a sqlite tile database, from tiles stored as files or from direct tile downloads.

You probably have to make Image loading on the QML side asynchronous to get a responsive UI

Yep, I'm looking forward to using asynchronous calls where possible anyway, so asynchronous image loading should be fine. :)

(image loading requests could only be synchronous I think).

How do you mean that ? If I understand it correctly, this depends on the state of the asynchronous property of the given Image element:

• it it is set to false (default), the GUI thread will call the image provider directly and wait for the image to be returned
• if it is set to true true, the image provider is called by a separate thread (or maybe even by multiple threads as the QQuickImageProvider documentation mentions that the requestImage method needs to be reentrant).

But I guess that due to GIL only one image loading Python callback will be running at the same time anyway? Well, as long it's not blocking the GUI thread, this should not be that much of an issue.

Or is there more to it ?