p5py using PyQt5 Backend

[arihantparsoya]

Since few days I have been working on a different implementation for p5py. There are a few major issues with the existing p5py library:

• Performance (#110). There are two possible reasons for poor performance. First, the tessellation is done p5py slows down the rendering pipeline. Processing Java and p5.py are built on top of technologies which provide high level abstractions for path rendering (p5.js uses CanvasRenderingContext2D for example). This allows those libraries to avoid tessellation of shapes altogether. Second, the vispy library is not optimised for handling multiple calls to OpenGL at high frame rate.
• High resolution support/ Antialiasing (#19 , #13)

To address these issues, I explored the use high performance rendering engine (possible written in C++). I found that Qt was a good fit for the implementation of p5py library because it is optimised in C++ and it has existing python wrappers wrappers (PyQt5 and PySide2). Qt also has QPainter module which has path rendering APIs suitable for the rendering needed for p5py (similar libraries such as Kivy, Tkinter and WxPython does not support these APIs). However, the drawback of using PyQt5 is that it requires the installation of Qt software separately which is quite large(~1.5 GB).

I have created a prototype library using PyQt with support for 2D primitives: https://github.com/parsoyaarihant/ProcessingQt . Right now I am calling it ProcessingQt but the name can be changed.

It utilises the existing anti-aliasing functionality from Qt and has better performance than p5py. I ran the following sketch in p5py, ProcessingQt and p5.js:

import random 

def setup():
    size(500, 500)

def draw():
    background(255, 255, 255) # only (r, g, b) API is supported for now

    s = 50
    strokeWeight(5)

    for x in range(s):
        for y in range(s):
            fill(255, 0, 0)
            point(random.randint(0, width), random.randint(0, height))

    print(frameRate)

run()

The frameRate for the above sketch were:

• p5py: ~2fps
• ProcessingQt: ~10fps
• P5.js: ~37fps

Right now it only supports 2D primitives (https://github.com/parsoyaarihant/ProcessingQt/blob/master/example1.py). Support for other functionalities like fonts, images can also be added.

@abhikpal @jeremydouglass @marcrleonard

[jeremydouglass]

Exciting!

Processing evolved to use different rendering engines over time, which are passed to sketch size() or to creatGraphics() as arguments -- JAVA2D is the default, with P2D and P3D modes that each use OpenGL, then FX2D, PDF etc.

For p5.js this is WEBGL or P2D as renderer options to createCanvas().

The nice thing about an opt-in experimental render mode is that it can be bundled while it is incomplete--as QT or PYQT, vs the default VISPY. The actual QT dependency would be up to the user if they want to use it.

[vvzen]

Hi! interesting approach. I have a little bit of experience with PySide2, and I don't think it's really suited for high performance stuff. It's heavily aimed at building UIs, responding to UI events, etc.. and doesn't have a classic draw loop approach, which is very important for games/realtime apps. I've tried QPainter in the past and I've never found it to be very suited for this kind of Processing/Openframeworks stuff, simply because by design it was trying to address a different need, so in the end it never performed well enough (unless you actually go there and rewrite most of the stuff). I wonder though, if it would be possible to somehow use Qt Quick and QtQML.

QML uses a scene graph which greatly improves the performances by minimizing draw calls, plus it runs on a different thread. I've played around with the python backend a little bit, but I'm not an expert. QML is a really good approach for separating the backend logic from the frontend interaction/graphics layer, but once again, as with all the Qt family, its main aim is building UIs. So I don't know how much sense & how easy it would be to wrap a Processing-like API for python that then renders to a QML canvas.

I personally think that vispy is still a better option, I've seen incredibly complex visualizations being done in that library.. maybe p5py is lacking some optimisations?

My 2 cents! Happy coding everyone :)

[jeremydouglass]

One other thought about models / precedents -- Processing also supports static renderers, such as PDF. So high performance animation isn't the only use case for a renderer. However, if the whole reason you are trying to use a new renderer is performance then that would be good to assess / benchmark early....

[arihantparsoya]

I wonder though, if it would be possible to somehow use Qt Quick and QtQML.

@vvzen , I have read little bit about Qt Quick. I will explore how that can be used as renderer.

[arihantparsoya]

However, if the whole reason you are trying to use a new renderer is performance then that would be good to assess / benchmark early....

@jeremydouglass , For the sketch I have mentioned in the first comment, the fps using Qt is 5x times faster than p5py. I am not sure if there is an established way of comparing performance for processing because of the different APIs which Processing supports. At best, we can have a sketch containing all the different graphic APIs of processing and use that as a benchmark.

[abhikpal]

I don't have much experience with PyQT, but this looks very interesting!

The nice thing about an opt-in experimental render mode is that it can be bundled while it is incomplete--as QT or PYQT, vs the default VISPY. The actual QT dependency would be up to the user if they want to use it.

This sounds like a very good approach to me. Vispy already had support for working with different backends (via vispy.use()). Perhaps with some extra work we can expose this interface to the user. The only technical hurdle I can anticipate is to somehow find a way to call setup() before vispy is initialised.

For the sketch I have mentioned in the first comment, the fps using Qt is 5x times faster than p5py. I am not sure if there is an established way of comparing performance for processing because of the different APIs which Processing supports. At best, we can have a sketch containing all the different graphic APIs of processing and use that as a benchmark.

Seeing that p5py is really slow, comparing fps with Processing (JAVA) should be a reasonable benchmark for now. I tried writing the flocking example once and it was very unbearably sluggish.

[vvzen]

It’s also definitely worth having a read at this: https://www.qt.io/blog/qt-offering-changes-2020 just so that you’re aware of the future implications in terms of adopting Qt.

[marcrleonard]

Has anyone seen these two projects?

https://github.com/moderngl/moderngl https://github.com/moderngl/moderngl-window

The most intriguing part is the fact that it can take numpy arrays to fill the buffer (though, I think it's more in the context of a image/pixel array ... not vertices). Regardless, it seems like they have done a lot of plumbing to make the gl api more accessible, and they support many different windowing options.

Anyways, just thought. I thought it would be worth dropping that here 😄

[ReneTC]

I'm very curious about what the status of these issues is. Is it something that will be worked on in the google summer code 2020 as well?

[ziyaointl]

@ReneTC I think this issue in particular is not in the GSOC 2020 plan but if there's time left in the end (and others have not beaten me to it) I'm more than happy to work on it.

Cheers, Mark

[arihantparsoya]

@ReneTC , I have resumed my work on ProcessingQt. I will be able to implement most of the APIs in 2-3 days. However, I dont plan to implement Font and Images. If that is something you require, then let me know.

[ReneTC]

It's an amazing package @parsoyaarihant. I'm a huge fan. But I would like to add to the discussion: sadly not only does Qt take 1,5 gb but also requires an time consuming registration, with email confirmation and everything

[arihantparsoya]

@ReneTC , yes. This was the drawback I mentioned in my first comment. Unfortunately there aren't any alternatives to Qt which support the path APIs as well as Qt does. Making our own path rendering engine is quite tricky and it needs to be optimised in C++ (or any other efficient language) to get good performance.

[ziyaointl]

Another alternative is to use Skia - the vector drawing library used by Chrome and Firefox. https://skia.org/ I am not aware of a stable python binding though.

[jeremydouglass]

Interesting. It seems like the two most prominent python skia resources under active development are:

https://github.com/kyamagu/skia-python https://github.com/fonttools/skia-pathops

Skia itself is also under ongoing active development.

At a glance, I think that most of the p5py API looks like they could be covered using a SKIA backend renderer using skia-python.

https://kyamagu.github.io/skia-python/tutorial/overview.html#apis-at-a-glance

[arihantparsoya]

I have setup a basic skia-python sketch:

import skia
import glfw
from OpenGL import GL
import time
import random 

width, height = 200, 200

def init_surface(width, height):
    context = skia.GrContext.MakeGL()
    backend_render_target = skia.GrBackendRenderTarget(
        width,
        height,
        0,  # sample count
        0,  # stencil bits
        skia.GrGLFramebufferInfo(0, GL.GL_RGBA8))
    surface = skia.Surface.MakeFromBackendRenderTarget(
        context, backend_render_target, skia.kBottomLeft_GrSurfaceOrigin,
        skia.kRGBA_8888_ColorType, skia.ColorSpace.MakeSRGB())
    assert surface, 'Failed to create a surface'
    return surface

if not glfw.init():
    raise RuntimeError('glfw.init() failed')

glfw.window_hint(glfw.STENCIL_BITS, 0)
glfw.window_hint(glfw.DEPTH_BITS, 0)

window = glfw.create_window(640, 480, 'Demo', None, None)
glfw.make_context_current(window)
glfw.swap_interval(1)

surface = init_surface(width, height)
canvas = surface.getCanvas()

frameRate = 60
elapsedTime = time.perf_counter()

# Loop until the user closes the window
while not glfw.window_should_close(window):
    #glfw.wait_events()

    # calculate frame rate
    frameRate = round(1/(time.perf_counter() - elapsedTime), 2)
    elapsedTime = time.perf_counter()
    print(frameRate)

    # Render here
    canvas.clear(skia.ColorGREEN)

    paint = skia.Paint()
    paint.setAntiAlias(True)
    paint.setStyle(skia.Paint.kStroke_Style)
    paint.setStrokeWidth(8)
    paint.setStrokeCap(skia.Paint.kRound_Cap)

    s = 50
    for x in range(s):
        for y in range(s):
            px, py = (random.randint(0, width), random.randint(0, height))
            path = skia.Path()
            path.moveTo(px, py)
            path.lineTo(px, py)
            canvas.drawPath(path, paint)

    surface.flushAndSubmit()

    # Swap front and back buffers
    glfw.swap_buffers(window)

    # Poll for and process events
    glfw.poll_events()

glfw.terminate()

Speed Comparison for rendering 50 points on the canvas:

• skia: ~25fps
• p5py: ~2fps
• ProcessingQt: ~10fps
• P5.js: ~37fps

[postvak]

I came across this post while searching for processing-ish 2D drawing libraries for python and started to do some speed tests. I got somewhat different results than @parsoyaarihant. The following Procesing code:

void setup() {
  size(500, 500);
  strokeWeight(5);
  fill(255, 0, 0);
}

void draw() {
  background(255, 255, 255);
  for (int n=0; n<2500; n++) {
    ellipse(random(width),random(height),20,20);
  }
  println(frameRate);
}

gets me framerate of ~22 fps in Processing and a framerate of ~14 fps in P5.js (using the P5.js web editor in Firefox)

Doing the same drawing in PyQt5 using QPainter I get a framerate of ~26 fps. Quite unexpectedly (for me) is that I got the fastest results with pygame. The code below gets me a frame rate of ~82 fps. Unfortunately, stroke drawing in pygame is quite ugly.

import pygame
from random import random
import time

# SETUP
pygame.init()
w = 500
h = 500
screen = pygame.display.set_mode([w,h])
strokeweight = 5
framenr = 0
tic = time.perf_counter()

# DRAW
while True:   
    for event in pygame.event.get():
        if event.type == pygame.QUIT:
            pygame.quit()

    screen.fill((255, 255, 255))
    for id in range(2500):
        x = random()*w
        y = random()*h
        pygame.draw.ellipse(screen, (255,0,0), (x,y,20,20))
        pygame.draw.ellipse(screen, (0,0,0), (x,y,20,20), strokeweight)

    pygame.display.flip()

    framenr += 1        
    toc = time.perf_counter()
    print(framenr / (toc-tic))

[arihantparsoya]

Thank you for sharing this @postvak . Anti-aliasing is used to make the strokes look smooth. I suspect, the presence of anti-aliasing is making processing slow.

I am surprised that PyQt5 is better than Processing. There are no standardised benchmarks to measure performance. So, we are currently relying on a diverse set of sketches to do speed comparisons: https://github.com/p5py/p5/tree/master/profiling

[swirly]

I came across the post from another about slow drawing. Reading the proposals, I can't understand why p5.js does a better job than any other implementation proposed. It's javascript in the browser ! a native app should do a better job, no ? Wouldn't it be possible to directly adress an openGl API ? Like pyOpenGl for example ? Without a intermediate library, speed should be increased. Am I wrong ?