Add pixel buffer object support

[cbaggers]

https://www.khronos.org/opengl/wiki/Pixel_Buffer_Object

[malcolmstill]

Quite interested in learning more about these. I added screenshotting to https://github.com/malcolmstill/ulubis the other day, using gl:read-pixels. It seems very slow, but I believe this is probably due to gl:read-pixels allocating a Common Lisp byte array, taking multiple seconds. I think if I allocate a buffer with CFFI and call glReadPixels it'll be much faster (I read somewhere about 16ms).

What I'm more interested in is doing video recording from OpenGL. I obviously can't use screen recording stuff for X on my KMS/DRM backend. I believe the speed up from PBOs would allow me to stream from KMS/DRM (though I'll have to somehow pass that to ffmpeg or I'll be dumping out raw frames)

[cbaggers]

Very cool. I'll start reading into this part of GL then, feature wont land soon but knowing someone is interested certainly makes it a much higher priority than it was :)

[cbaggers]

Also @jobez seeing your thumbs up across the repos has been very motivating, I hope you're doing well.

[cbaggers]

I went down the coffee shop and had a read. Here's the notes so far:

---

PBOs are gpu-buffers PBOs are about pixel transfers to/from the user from/to images in OpenGL

When data is in a texture it is stored however the gpu likes, we get an image-fromat to tell use about the components but the gpu is free to lay things out pretty much however it likes (tiling for mobile etc)

Data in buffers or c-memory however is different, we have concrete layouts that must be adhered to.

GLs terms are then best understood from the point of the view of the gpu. It has data the way it wants and when it needs to write to a buffer c-memory it has to pack it in a specific fashion. This is why the 'download from gpu' operations are called pack operations and the 'upload to the gpu' operations are called unpack operations.

A pixel transfer has to specify the both ends of this exchange and GL specifies conversiosn that will happen in the process of achieving it

The syncronisation is managed by GL, by both the source and target being in gpu memory it has freedom to do things async, until you try do something with the buffer again, then it has to stall. (use sync objects to avoid this)

CEPL TODOs:

• There isnt much new behind the scenes code to add. We can already make gpu-buffers and bind them to targets. We have transfer operations set up already, we just need to allow the user to make buffer backed gpu-arrays as the initial contents for uploads.

• We do need to add more down 'download' (pack) operations.

• We need to add the pull-g-into family of functions, these will handle all manner of pack operations

• We could also add more explicit push-g-to.. hmm terminology is a bit interesting when it's gpu to gpu transfer.

[cbaggers]

haha well this feature uncovered some bugs :p https://github.com/cbaggers/cepl/issues/278

Code is progressing though, I needed to make all CEPL arrays aware of row-alignment so we could pack/unpack data whilst respecting the various alignments. This is working nicely and you get nice, transparent, uploads and downloads.

CEPL indexes data in a coordinate style (e.g. x y z) where:

• x is position in 1st dimension (row)
• y is postion in 2nd dimension (plain)
• x is position in 3rd dimension (cube)

however in lisp arrays its the reverse. You make a array with dimensions (10 20) which means 10 rows of 20 columns, and index into is like (aref foo 4 5) means 5th element of 4th row, wheras in CEPL the x,y,z style of indexing means 4th element of 5th row.

This mismatch lead to some nasty data fuckups with uploading multidimensional lisp arrays to the gpu (however all 1d stuff, sqaure arrays and textures loaded from c-libs were fine, so nobody really noticed)

I looked into matching the lisp style of indexing but it makes things more confusing as a texture should definitely be defined as x,y,z, but what about a texture backed gpu-array? if it has lisp array indexing then a 20x10 texture would have a 10x20 gpu-array at (texref some-tex 0) which is just nasty. To avoid the inconsistency I've gone for x,y,z everywhere in CEPL so we only have to explain the switch when uploading from multi-dimensional lisp arrays to CEPL's array types.

That's fixed on a branch but not merged yet, it needs tests added first.

I've also begun adding the copy-g function which can be used for moving data between various places in CEPL. I have got implementations for all the data[0] transfer stuff that [pull/push]-g can currently do, next I'm going to add the PBO specific stuff, which should mainly amount to binding a gpu-array to GL_UNPACK_BUFFER. I'd then like to make make-texture able to take buffer-backed gpu-arrays.

Then naturally we need the reverse and then texture to texture copying (at which point make-texture can start taking texture-backed gpu-arrays too)

Lots to come it seems :)

[0] pull-g also works on gpu-functions and pipelines which I'm not going to add to copy-g

[cbaggers]

data layout, row alignment & mutable textures fixes all merged to master. I can start thinking about pbos now :p

[cbaggers]

In the following I'm going to use type names a bunch gpu-array-t is texture-backed gpu-array, gpu-array-bb is buffer-backed gpu-array.

The feature/pbo branch can how handle copying data:

• gpu-array-bb -> gpu-array-t
• gpu-array-t -> existing gpu-array-bb
• gpu-array-t -> new gpu-array-bb
• gpu-array-t -> existing c-array

along with all the existing transfer kinds.

I've also exported the typed transfer functions for when you know the types, but copy-g works in all directions so can be used in place of any of these: https://github.com/cbaggers/cepl/commit/0fdf3637f458a2ea9e3a613077efa67c8b2941ae

I still need to look into loosening up some type restrictions CEPL currently imposes. I only let you transfer to/from arrays with compatible element-types, however I'm pretty sure you can convert on the fly. Which would allow :rgba8 -> :vec2 for example.

[cbaggers]

merged to master. There is still more that can be done, but this is a nice start

[cbaggers]

Worth noting that I havent added read-pixels yet, that reads from bound framebuffers. I'll have a think about whether there is some more CEPL'ish way to present the functionality.