FrameBuffer, GLFramebuffer, scattered functionality... OpenGL package?

[louisponet]

So I'm trying to implement the Weight blended OIT, as was mentioned by Simon in isosurface/transparency issue.

I'm really finding it hard to figure out all the components in the OpenGL related julia stack. There seem to be 3-5 packages which are so intertwined with each other (ModernGL.jl, GLFW.jl, GLWindow.jl, GLAbstraction.jl ), with similar stuff being defined throughout the three last ones, such as render(...) etc.

The most obvious place is the FrameBuffer type defined in GLAbstraction.jl and the GLFramebuffer in GLWindow.jl. It seems that the first one is not complete and is nowhere used.

This makes me wonder if it wouldn't be an option to group all the OpenGL packages together, in doing so maybe figure out what's useful/used, and what the best structure of the package might be. After all, this is supposed to be an API, i.e. understandable even without everything being documented minutely.

This would then mean that all the non-core stuff (with 'core' I don't mean boilerplate, I mean functionality around types like GLVertexArray, GLFramebuffer, etc) can be sifted out to kind of a middleware package which implements some often used, not directly 'API', functionality.

I wonder if this is something you agree on, and if you think this should be addressed.

[SimonDanisch]

Oh yeah, totally... the whole OpenGL stack is ripe for a rewrite :(

It seems that the first one is not complete and is nowhere used.

That's true!

Yeah, I want to trim down GLAbstraction, to really just be a pure abstraction over ModernGL, without anything like the Camera etc. Also, I want to integrate it with Transpiler.jl to make it straight forward to compile Julia functions to OpenGL shaders.

It would also be great to extend GLAbstraction to include uniform buffers etc and further integrate it with GPUArrays, to actually have the whole range of array operation (running on the GPU) available for GLBuffers/Textures.

I started prototyping a better vertex array type and uniform buffers in Visualize.jl.

Also it would be great, if GLWindow would get renamed to GLFWindow, and really just holds the GLFW specific functionality.

I did that in Visualize.jl: https://github.com/SimonDanisch/Visualize.jl/blob/master/src/backends/opengl/glfw.jl, although the renderloop shouldn't be there either.

The renderloop should really be part of a higher level package, allowing for different ways of rendering GLAbstraction renderobjects, with different passes etc. Then GLVisualize could just use that to implement it's API.

[louisponet]

Yes exactly! Ok great to hear, yesterday I played around with how I would do it, I took ModernGL, GLFW, GLWindow, GLAbstraction, and merged / splitted some functionality.

The idea is to split off all the higher level stuff into an "extended package", this holds extensions to the base package (signalled by basefilenameEXT.jl) which at this point are mainly all the Reactive.Signal related functionality, then there are also some new functionalities which are just new filenames.

I arrived at a structure with 5 submodules: CoreGL (ideally I would like to call it Core or Base, but yea Julia.Core Julia.Base, doesn't mingle it seems), GLFW, Maths, Abstractions, Render.

First I took some parts of GLExtendedFunctions.jl and put then inside and extra file in the CoreGL package, since they are just nice functionalities, nothing high level.

Secondly I merged all the Window, GLContext and MonitorProperties into GLFW, for example create_glcontext() is now just a constructor for Window, and added some not too high level functionality there too, mostly related to callback construction, standards etc. This is partly in a new file utils.jl and added to the end of api_functions.jl. The Screen type I have renamed to GLScreen in light of consistency with GLContext (should Window be renamed to GLWindow?), but moved it to the higher level package as an extension.

Lastly, the hardest part was sifting through 'GLAbstraction'. I went with seperate filenames for each type, rather than grouping types and scattering functionality to other files. So for example there is 'gpu_arrays.jl' (which i believe should be redone so as to use the GPUArrays.jl package?), 'buffer.jl', 'texture.jl', 'texturebuffer.jl', 'framebuffer.jl', etc. There is also some stuff I didn't quite know what to do with, so I put them into 'constants.jl' and 'utils.jl'. I'm wondering if it would be useful to make a subfolder purely with types/ type-related functionality, and one with constants and utils, but Idk up for discussion. Most of the signal related functionality is 1-1 put into filenameEXT in the higher level package, together with RenderObject.

Base render functionality was thrown all together into the Render submodule, with a lot of extended functionality into the higher level package. This is one of the big points I'm not sure about: Should we group all the render functionality, having a render submodule, with filename per object/thing that is being rendered in various ways? Should we have a render subfolder in the Abstractions? Not sure.

Finally I put all the math related things into Maths submodule, straightforward.

I didn't touch the camera functionality, because I'm not sure where it belongs. I think either a folder in abstractions ( if we go the folder route ), or inside a screen/camera folder in the higher level package.

This is a very brief summary of my adventure yesterday, everything in the core package precompiles and the GLFW, ModernGL tests work, so at least theres that.

As said, I did this in one day, so a lot of things probably aren't exactly where they belong and the structure might be improved. This should all be discussed. However as a small starting point I'll attach the package as is, so you can have a look at it. I annotated the files so that #Came from ... signals where blocks of code originated from (especially related to GLWindow, GLAbstraction, not so much the other two because they were straight copies). #change: signals small things I've changed trying to get things a bit more clear/ logical/ consistent. I also maybe have put some functions etc in a different order that I found more logical, but again this is completely up for discussion.

OpenGL.zip

I do not want to force anything on anyone, especially because this is all code that I have done pretty much 0 work on ( I did copy+ rewrite small functions so they work also without signals). This makes me think there should be a discussion between all the developers of the 4 packages under consideration. What is the best way to go about this? Slack? Gitter? Discourse?

I will have a look at the things you link, come up with a strategy to put them all together.

Sidenote: I realized there was never an OpenGL.jl package before probably because of corporate overlord related topics, name needs to change (to what?).

EDIT: there's also an annotation #question: where I put some comments/observations/questions. EDIT2: This is the first time I work with such a big package, so there might be some obvious mistakes in the design, so please everything is up for discussion/change/objections/anything :) EDIT3: If you want I can also attach the extended package, but I didnt put work in getting it to work, neither in the structure too much.

[SimonDanisch]

Okay that's a big rewrite :D

: I realized there was never an OpenGL.jl package before

Oh, there was one, but I guess it got deleted

This makes me think there should be a discussion between all the developers of the 4 packages under consideration.

Well, that's basically mostly me :P You can decide where you want to discuss things :) I dislike gitter, but am on discourse and slack. My favorite chat client is telegram though (SimonDC), if you have that.

I will look into your refactor. As a first note, this shouldn't be a single package. First of all, it's quite important that the most basic/core package doesn't at all rely on GLFW - I know this isn't the case right now, but that should be one crucial goal for any restructuring.

More comments to follow. Feel free to create forks and PRs even if they are WIP. I think besides the rename of GLWindow -> GLFWWindow, the package structure should basically stay the same.

[louisponet]

Okay that's a big rewrite :D

Yesterday was adventurous indeed. It's mostly reshuffling though. The rewrites i believe are mainly:

• Changed GLContext = Symbol to GLContextID in GLAbstraction, and the functionality about the globals mimics this naming scheme
• Changed Context <: Composable to Composition <: Composable to not confuse mysefl with GLContexts and the like

Aside from that I think most naming etc remained fairly constant.

I agree on that the basic functionality and the Abstractions shouldn't depend on GLFW, I think I managed to achieve that, the abstractions and core package don't use anything in GLFW. I put GLFW in there because to my knowledge, it seems this is the only way of creating OpenGL contexts that work on each os ( see the Gtk issue with opengl). The ideas behind the submodules is that they are providing basic functionality, each with it's seperate task. Want to use pure OpenGL and thats it import OpenGL.CoreGL Want to use the abstractions, add import OpenGL.Abstractions Want to create a basic window, add import OpenGL.GLFW etc

Want to use higher functionality import OpenGLEXT or maybe GLReactive or whatever.

I understand the idea behind splitting of the GLFW => you can use any context providing package you'd want. But this is still the case by only importing the OpenGL.Core, and the GLFW part doesn't add any noticeable overhead anyway. I think it's nice to provide users with all the basic functionality they need to get going with OpenGL, all in one package, with submodules.

From my experience coming to julia fairly recently (last april), the amount of small packages with scattered but related functionality, is pretty annoying/hard to know what you need what you don't. I think especially if we aim to provide an API around basically CoreGL, why not just include the bare minimum of context creating functionality too.

Theres also the option to keep ModernGL, rename it, and then put the windowing, abstractions, whatever in a mid-level package, and the extended functionality in an extended package.

I'll check out telegram, I like the idea of having a slack to figure all of this out (is there a JuliaGL one?). Gitter => same feeling.

[SimonDanisch]

There is the Julia slack: https://slackinvite.malmaud.com/

[SimonDanisch]

About the modules, we don't need to decide immediately as long as you restructure the code in a way, that it would be easy to split out the functionality into multiple packages.

[louisponet]

There is the Julia slack: https://slackinvite.malmaud.com/

Thanks I'll put myself in there!

About the modules, we don't need to decide immediately as long as you restructure the code in a way, that it would be easy to split out the functionality into multiple packages.

You're right, since I'm trying to keep the modules self-contained, splitting things up should remain as easy as a copy-pase, which I'm getting very fluent at. I'm writing up something to start the discussion on discourse as well.

[SimonDanisch]

Ok, two module where I'm pretty convinced that they should stay independent are ModernGL + GLFW! They are perfectly fine, self sufficient packages, and there is no reason why creating a simple opengl app should force you to be able to build GLFW... The other way around, there is no reason why you should be forced to use a package with all those abstractions, when you just want to use ModernGL.jl.

A big problem is maintainability. The larger the package, the higher the probability that one line of code disables the whole package in some environment / for a new Julia version. I had to learn this for GeometryTypes, which was originally my package and a lot of people helped me grow it, but in the end I was the only one maintaining it - so I suddenly was forced to maintain a lot of functionality which I didn't need, just to draw a cube with GLVisualize.

If all you're concerned about is visibility, we should rather do something about that ;)

[louisponet]

Discourse discussion link:

They are perfectly fine, self sufficient packages, and there is no reason why creating a simple opengl app should force you to be able to build GLFW... The other way around, there is no reason why you should be forced to use a package with all those abstractions, when you just want to use ModernGL.jl.

This makes perfect sense. However, if there is no valid alternative to use except for GLFW.jl, and even the tests of both packages depend on eachother, does it really make sense to seperate them?

A big problem is maintainability.

Makes a lot of sense too, I'm very experienced dealing with this. I don't directly see why, on the api-level, it wouldn't be possible to implement new abstractions/functionality while maintaining outward compatibility, but again I'm not terribly experienced.

one line of code disables the whole package in some environment / for a new Julia version

This is true, but obfuscation of things like this, by separating updates etc scattered through different packages, rather than being clear on what breaks all in one go makes it hardly an improvement. Also, we don't have to precompile the entire shebang, wouldnt't it be possible to precompile each seperate module, disabling it when it doesn't work (i'm more inclined to what i mention above)? On the other hand, i think it shouldn't be too hard to maintain GLFW and Core to be working always.

Having one coherent bundle, functioning bundle of core api tools that provide a minimal starting point sounds very nice to me.

[SimonDanisch]

However, if there is no valid alternative to use except for GLFW.jl, and even the tests of both packages depend on eachother, does it really make sense to seperate them?

There are STL.jl, GTK.jl, QML.jl, and more ;)

[louisponet]

Welllllll, GTK = linux, STL.. google can't find it?, QML.jl ... yes but CxxWrap.jl has never worked even on my mac which I presume is the easiest OS to reliably get stuff working on. Part of the reason why GLFW.jl is so self-contained is because well it is, and it seems it 'just works'™. Also, the overhead is so minimal anyways. But okay I get your point.

[SimonDanisch]

Yeah I get that. But we can make peoples life that want to use those much easier, without any effort. The GTK issue you mention is actually the reason why I want to become stricter with this, instead of integrating everything into one package. GTK seems to segfault, as soon as you load GLFW anywhere in the Julia process. Probably fixable, but why bring a whole project to halt, just because you want to signal, which packages belong together :P We can still have a meta package, that does "using GLFW" somewhere.

GTK is btw cross platform and works relatively well on windows ;)

[louisponet]

I see, I didn't know about this. Maybe the best Idea is then indeed isolate ModernGL and GLFW, and put an api-layer around them.

How about the following: Instead of explicitely depending on a windowing package, there is a submodule implementing some Abstract window/context type. Then in GLFW there is GLFWWindow <: AbstractWindow etc. Would that be a more useful way of handling this? Or would you just not put any window related stuff in the main api package?

A different way that makes sense to me would be some structure like:

CoreGL => GLAbstraction 
                             => GLToolkit
GLFW                                 
GTK     => GLWindow     
QML    
STL

However, I would already put a lot of stuff inside the GLFW package, like I did in the refactor, not sure where GLWindow comes into play, as I would postpone adding signal relating stuff to GLToolkit in such a structure.

EDIT: This is starting to make more and more sense to me. GLWindow would manage the backends related to the different possibilities, maybe defaulting to GLFW.jl similar to how Plots.jl defaults to GR. This way we have two stacks taking care of their end of the bargain, all culminating and a wonderful stacked GLToolkit package, bringing together the functionality. What could then happen is either we put the Reactive.jl stuff in there too, or there is a GLReactive.jl package built on top of he stack. I like this a lot, not in the least because the packages/ supposed structure would remain relatively the same.