erinnovations commented 12 years ago

As mentioned here: http://forum.openframeworks.cc/index.php?topic=9107.0 OpenGLES2 not working. It is always falls back to ES1. I tested it only on iPad2 with iOS 5.0.1.

ofTheo commented 12 years ago

@julapy could you coordinate with @andreasmuller to get this into the develop branch? maybe @arturoc @elliotwoods @damiannz @memo could keep an eye out also on the opengl stuff

https://github.com/andreasmuller/openFrameworks/tree/develop-opengles2

julapy commented 12 years ago

yeah no worries, i'll make this a priority.

arturoc commented 12 years ago

Hey

the main problem with this is that it modifies lots of parts of the core, almost everything that uses openGL, and makes it rely on a modified version of gles2-bc:

http://code.google.com/p/gles2-bc

which is an unmantained library, last commit is from 2009:

http://code.google.com/p/gles2-bc/source/list

that changes lots of direct calls to opengl inside classes like ofTexture (which is already super messy) to calls like _ofXXXX making the code way harder to mantain.

ie: https://github.com/andreasmuller/openFrameworks/blob/develop-opengles2/libs/openFrameworks/gl/ofTexture.cpp#L372

glBindTexture(...) -> _ofBindTexture(...)

which in turn is some ifdef'd function in ofGraphics

I think at some point we should think on how to make OF be more "non-fixed pipeline" but this solution seems super invasive just to have shaders in iphone and android.

don't know if something like this would work, but what about having an ofOpenGLES2.h file that is included from ofConstants.h for openGL ES 2 only and does things like:

define glBindTexture(target,id) gl->glBindTexture(id,target)

an ofGLES2Renderer

that way we won't need to modify other parts of the core like ofTexture, ofVbo... and in the future we can look into moving to something that is more compatible with non-fixed pipeline and doesn't rely on external libraries.

a

El 12/06/12 21:19, Theodore Watson escribió:

@julapy could you coordinate with @andreasmuller to get this into the develop branch? maybe @arturoc @elliotwoods @damiannz @memo could keep an eye out also on the opengl stuff

https://github.com/andreasmuller/openFrameworks/tree/develop-opengles2

Reply to this email directly or view it on GitHub: https://github.com/openframeworks/openFrameworks/issues/1178#issuecomment-6281511

julapy commented 12 years ago

hmm didn't realise its going to be so tricky... to keep things consistent i would naturally try to modify ofTexture, ofFbo etc, but i do see what you mean @arturoc, that this could be a gateway to #ifdef hell.

think my approach will be to first get ES2Renderer working on iOS. it will be a good start and should run ok without using the other OF gl classes. if the user then decides to use their own OpenGL ES2 code, then at least they can work with that for now.

think i'll then move onto ofTexture, see what's required to make it work with ES2. having those two things working, we'd be off to a good start...

L.

andreasmuller commented 12 years ago

Hi All,

Sorry am out and about in Ecuador on holiday, so am a little bit analogue at the moment.

I think we definitely need to come up with a workable solution for re-directing graphics calls to a custom renderer, but the idea of ifdef'ing does scare me as well.

I think the _ofXX solution I had is ugly, but can be made neater by just formalising that we want to have one level of indirection for every graphics call, based on the OpenGL model and then leave it up to the custom renderers to implement/interpret what to do based on this. ( ofAdvBindTexture, ofAdvEnable for instance )

This may seem like a lot of work to just get shaders working on iOS and Android, but I think it's also a really good way to future-proof the OF basecode.

The work that went into custom renderers in the OF core made it a breeze to implement the GL ES2 renderer, take this one step along and we'll be well prepared for deprecation of GL1 or implementing OF on (really) exotic platforms (XNA perhaps?) and would make it really easy to do for raytracer scene files, etc what Arturo (I think) did for the Cairo exporter!

Any questions about the code as it stands just shoot them my way, I'll be a little bit slow on the reply for another week or so.

/A

On 13 June 2012 13:31, lukasz karluk < reply@reply.github.com

wrote:

hmm didn't realise its going to be so tricky... to keep things consistent i would naturally try to modify ofTexture, ofFbo etc, but i do see what you mean @arturoc, that this could be a gateway to

ifdef hell.

think my approach will be to first get ES2Renderer working on iOS. it will be a good start and should run ok without using the other OF gl classes. if the user then decides to use their own OpenGL ES2 code, then at least they can work with that for now.

think i'll then move onto ofTexture, see what's required to make it work with ES2. having those two things working, we'd be off to a good start...

L.

Reply to this email directly or view it on GitHub:

https://github.com/openframeworks/openFrameworks/issues/1178#issuecomment-6299578

ø¤°°¤ø,¸¸¸,ø¤°°¤ø,¸¸»«¸,ø¤°`°¤ø,¸

Nanika

p: +44.(0)20.7729.3090 f: +44.(0)20.7729.3396

t: @nanikawa

http://www.nanikawa.com

ø¤°°¤ø,¸¸¸,ø¤°°¤ø,¸¸»«¸,ø¤°`°¤ø,¸

ofTheo commented 12 years ago

ahh okay - thats a good explanation, I'm def hesitant to add more funky code to ofTexture / ofFbo :) I like the idea of trying to make OF more non-fixed pipeline, I mean the whole idea of having a swappable render was that in theory the whole rendering engine could be swapped.

Would also allow for DirectX :-P

arturoc commented 12 years ago

yes, theoretically the swappable renderer should do this but the change in paradigm from fixed to non-fixed pipeline is so big that right now it doesn't cover all the cases. i haven't thought about this that much but i imagine in the future the swappable renderer would provide something similar to what gles2-bc does, so for example to bind a texture the renderer would have a specific method and ofTexture could do:

ofGetRenderer()->bind(*this)

that would allow to have for example directx textures.

another posibility is that things like ofTexture or ofFbo are platform specific (ie: gl, directx...) and we could have ofGLTexture ofDXTexture... where ofGLTexture could support non fixed pipeline and use a pattern similar to what we use for the videoplayer/grabber, sound...

my main concern with this solution is having to rely on gles2-bc that's why i was suggesting the ifdefs which is definetely ugly but not so invasive as the _ofXXX methods and doesn't polute ofGraphics either

all of this solutions are really long term and it's probably useful to find an intermediate solution to have support for shaders in openGL ES but i'm a little bit concerned about the reliance on gles2-bc in most of the gl code.

probably we could begin by looking for functions that are needed in gl es 2 and could be useful to have in OF as it is right now, things like ofMult/LoadMatrix(ofMatrix4x4), removing completely glu and substituting it with ofMatrix4x4, having our own stack of matrices... then see what's missing for openGL ES 2 compatibility. For example a class like ofVbo shouldn't require any change, since vbo's are supposed to be supported in GL ES2 but the way we have them implemented right now is not compatible so we could take a look at what needs to be changed there so the same version is compatible with fixed and not-fixed pipeline if that's possible at all. After that if something else is needed we can find some solution by using gles2-bc or implementing our own functions.

apart from that, this code is a really good start since it shows what we need to change for having compatibility with GL ES2 and openGL >=3

andreasmuller commented 12 years ago

Hi!

Back in London, some thoughts on this.

So what has been holding a GLES2 (or other) renderer back from release is basically:

Consensus on how to handle these changes to the core.
Of course testing testing testing.

So these are the functions I had to add to GLRenderer from OF 0.07, don't get put off by the ugly _names, that is just cosmetic:

    // "Private" GL hooks needed to start the work on abstracting the rendering from OpenGL 1.1 
    void _enable( unsigned int capability );
    void _disable( unsigned int capability );

    void _alphaFunc( unsigned int func, float ref );
    void _blendFunc( unsigned int sfactor, unsigned int dfactor);

    void _matrixMode( unsigned int mode );  

    void _lightModelfv(unsigned int pname, const float *params );
    void _lightf(unsigned int light, unsigned int pname, float param);
    void _lightfv (unsigned int light, unsigned int pname, const float *params);

    void _getMaterialfv(unsigned int face, unsigned int pname, float *params);
    void _materialfv(unsigned int face, unsigned int pname, const float *params);
    void _materialf(unsigned int face, unsigned int pname, float param);

    void _activeTexture( unsigned int texture );
    void _bindTexture(unsigned int target, unsigned int texture);
    void _texImage2D (unsigned int target, int level, int internalformat, int width, int height, int border, unsigned int format, unsigned int type, const void *pixels );

    void _texEnvf(unsigned int target, unsigned int pname, unsigned int param);

Some matrix functions as well, but these are a no-brainer.

    void loadIdentityMatrix (void);
    void loadMatrix (const ofMatrix4x4 *m);
    void loadMatrix (const float *m);
    void multMatrix (const ofMatrix4x4 *m);
    void multMatrix (const float *m);

This is what I found I needed as I was porting an old app of mine that needed shaders, there might be a few more. These of course have global functions that in turn call the renderers' functions.

Now to keep things neater we can of course add things like:

ofBaseRenderer::bind( ofTexture* _tex );
ofBaseRenderer::bind( ofVbo* _vbo );
ofBaseRenderer::bind( ofMaterial* _mat ); 
ofBaseRenderer::bind( ofFbo* _fbo ); // we already have ofBaseRenderer::setCurrentFBO(ofFbo * fbo);

Personally I favour internally following the OpenGL model above (plus renderer->bind(_tex) ) for communicating with all renderers and putting the work onto the custom renderer implementers to interpret what the OpenGL commands would mean in their renderers, at least as an interim solution.

The reasons for this being that:

We are going to be OpenGL focused in the near to medium term (right?) The alternative is to come up with a generic graphics API, which could be pretty tricky.
That it would make it easy for people to follow tutorials they find on the Internet even if they are using a custom renderer like the GLES2 one. Even if they have to do: ofGetCurrentRenderer()->enable( GL_TEXTURE_2D ); Same for advanced users, just get a pointer to the renderer and feed it the code you found.

That's my gut feeling, do you guys/girls think that last argument is valid?

In terms of relying on gles2-bc, I do agree it's not ideal that it seems abandoned at the moment, but the functionality it fills is emulating the fixed function pipeline, something that most projects that need shaders might do less of.

The extreme example of this NOT happening would be an app that rendered everything just like the fixed function pipeline would, but has a fullscreen blur/bloom shader applied to an FBO, but one would imagine most apps would have some shader based objects and render GUIs and little bits and bobs with the fixed function emulation, meaning they won't touch on the advanced stuff where gles2-bc might need some bugs fixed.

And something I was thinking about, I don't think people are asking for a shader capable renderer on iOS or Android for OF, because they largely assume that it works like desktop GL where you can mix them, an easy mistake to make unless you dig into it a little bit, I know I was bitten by this a few years ago.

But yes, consensus on how we approach the changes to the core is step 1.

/A

bakercp commented 11 years ago

Any movement on this? Apologies for the multiple thread pings, but this discussion is particularly relevant as Raspberry Pi dev continues to ramp up.

bilderbuchi commented 11 years ago

Don't feel bad about thread pings, on the contrary! They are needed at times to prod stalled issues along. :-)

julapy commented 11 years ago

im working on this at the moment... need shaders for an iOS project so perfect time to do it.

i don't think ES2 will come in one big PR so im taking a few steps to manually enable/disable it for the people working on it. currently ofxiPhone tries to load ES2Renderer, fails and loads ES1Renderer instead. i think a good approach is to have ES1 load by default and if enabled, ES2 can load...

it can be done like this in main, ofAppiPhoneWindow * window; window = new ofAppiPhoneWindow(); window->enableES2Renderer();

and these are the methods that will be added to ofAppiPhoneWindow

void ofAppiPhoneWindow::enableES1Renderer() { esRendererVersion = ESRendererVersion_11; }

void ofAppiPhoneWindow::enableES2Renderer() { esRendererVersion = ESRendererVersion_20; }

bool ofAppiPhoneWindow::isES1RendererEnabled() { return esRendererVersion == ESRendererVersion_11; }

bool ofAppiPhoneWindow::isES2RendererEnabled() { return esRendererVersion == ESRendererVersion_20; }

int ofAppiPhoneWindow::getRendererVersion() { return esRendererVersion; }

this way ES2 can be enabled when its in development and won't get in the way of other users who are happy using ES1.

how does everyone feel about this?

andreasmuller commented 11 years ago

One of the OF heads can expand on this, but as far as I understand the 2 main issues are:

To get emulation of ES1 functionality (i.e we want OF drawing functions to continue working without writing shaders for everything) we, in my implementation, use http://code.google.com/p/gles2-bc which hasn't been maintained for a while. I personally don't see this as a major issue as it seems to work pretty well for the basic stuff and if you need shaders on iOS, chances are you plan on mostly using your own shaders anyways, but you just don't want to have to write some and bind them just to draw UIs and debug text, etc.
The bigger issue is the big changes to the core, to pass along more information to the renderer. This is something that longterm would be great as it opens up the possibility for non-GL renderers (realtime raytracers!) and advanced scene exporters, but obviously needs to be done with some care.

There is one project that looks promising as a solution, it wouldn't require us to change very much in the core as it binds GL commands to it's own library and then does it's magic to emulate it behind the scenes (if you call an ES1 function in ES2 you crash).

I haven't had a chance to dig deep into it yet I'm afraid: https://github.com/p3/regal

ofTheo commented 11 years ago

@julapy what you described looks good to me!

@andreasmuller is right though, to get OF drawing the way we're used to in ES 2.0 - seems like it requires a fair amount of changes.

Would it make sense to start with just enabling ES2 and maybe an example that includes some shaders for setup and drawing of things on screen?

Then work towards a simple way to get the OF drawing commands -> ES 2.0 ?

Both those libraries look like their worth checking out. Though http://code.google.com/p/gles2-bc says it doesn't support glColor4f :/

andreasmuller commented 11 years ago

@ofTheo It definitely supports glColor4f, I think what he means is that it doesn't support glColor4i, glColor4b, glColor4d, etc etc.

My iOS version of Hana ( http://itunes.apple.com/us/app/hana/id556557031?ls=1&mt=8 ) uses the ES 2.0 branch of OF.

One thing to be careful with is that if you call any unsupported glX commands in ES 2.0 it will simply crash, so this isn't very user friendly. In theory it should be possible to have ES2 libraries linked without any ES1 stuff there so the compiler complains from the start, but I wasn't focusing on that at that point.

I think going the way you describe with enabling ES2 needs to be very clearly documented as not supporting ANY openFrameworks drawing functions.

I think it will be pretty disappointing to most people if we seem to support ES2 and for them then to realise that doing anything with it is going to be a fair but more work than they are used to. Most people assume that it works like desktop GL where you can have shaders but still keep your nice and simple fixed function drawing commands.

Definitely a step in the right direction though, for the advanced users it would certainly help. We need to double check if ofVbo and ofShader will "just work" though.

arturoc commented 11 years ago

regal seems like a really good solution, something that is totally transparent might be better than how gles2-bc works. but haven't really looked at it.

i've just sent a PR that adds to the PR i merged yesterday from andreas for loading and multiplying matrices, with that we have independence from openGL in ofCamera, we could start a new renderer like ofGLES2 or ofModernGL or something similar that uses ofMatrix + a custom matrix stack the way it's done in the cairo renderer and begin to identify what else we need to implement in a different way to have a system that works with modern openGL without need for external libraries even if at first we rely on some library so the change is easier. Most of those functions are already in andreas branch but relying on gles2-bc

andreasmuller commented 11 years ago

Oh and if Regal indeed does what we need, it is definitely the better option I think, actively maintained and has a NaCL path as well, nothing anyone is actively pursuing for OF (I think), but who knows.

arturoc commented 11 years ago

yeah, just saw NaCL in there, that could be really great! i've been looking into it for a while but just lately seen that they've added support for openGL

danzeeeman commented 11 years ago

Correct me if I am wrong, but we would only be using OpenGL ES 2.0 on the devices that support it? Those devices are iOS Devices, Android Device, and other ARM-Linux devices (such as the Rpi), correct? There are Android devices that only support ES 1 but they are becoming hard to find.

arturoc commented 11 years ago

yes, perhaps also NaCL and it can also be the base to move OF desktop to openGL 3/4

julapy commented 11 years ago

@arturoc, your suggestion of having a ofGLES2Renderer makes sense to me. atm im trying to get the iOS ES2Renderer working with the ofxiOSEAGLView and the one thing that seems to be getting in the way is ofSetupScreen() which is calling ofGLRenderer::setupScreenPerspective and (i think) because those calls are ES1 specific, its messing things up.

by swapping out the renderer for ofGLES2Renderer and making ofGLES2Renderer::setupScreenPerspective ES2 specific should solve this.

the rest of ofGLES2Renderer can be empty function but at least ES2 on ios will be working and it will be a start for us to drop in other ES2 code... what do you think?

arturoc commented 11 years ago

as far as i remember almost all other gl related classes had problems too, like ofTexture, ofVbo, ofFbo... you could probably compile OF without them but doesn't seem really useful without those

julapy commented 11 years ago

completely agree @arturoc, the move to a programmable pipeline is huge! and may require a lot of changes. for now, i think the way forward is baby steps, slowly enabling ES2 in the develop branch and slowly building up on it across iOS, android and all other ES2 platforms.

looking at @andreasmuller's branch and from whats been discussed above, it seems everyone agrees that a ofGLES2Renderer is the way to go, right?

ive got a simple iOS ES2 example running now that im working on from devApps. it enabled ES2 and uses a really bare bones version of ofGLES2Renderer (without gles2-bc), drawing some shapes using ofRect, ofCircle etc... at this junction we have a lot of choices. we can use gles2-bc, or strip out its code and implement our own version, or possibly use regal... either way it would be good just to have ofGLES2Renderer in the core so we can build on it and enable all the other ES2 platforms to test as its being put together.

i can get a PR together a bit later today if everyone is happy with this first baby step?

julapy commented 11 years ago

ive noticed in @andreasmuller's branch that ES2 is enabled by defining OPENGLES_VERSION_2

on iOS, if the user enables ES2 then the device tries to create the ES2Renderer but if it fails it falls back on the ES1Renderer, so we need to be able to switch between ES1 and ES2 code at runtime. unfortunately OPENGLES_VERSION_2 defines won't work at runtime...

just trying to figure out an elegant way of switching between running ES1 and ES2 code in the core. maybe something like this is the way to go,

if(ofGetCurrentRenderer()->getType() == "GLES2") { // run ES2 code. } else { // run ES1 code. }

or maybe a OF global function, something like ofGetRendererType() and then we can also create a bunch of renderer types, OF_RENDERER_GL OF_RENDERER_GLES2 OF_RENDERER_CAIRO etc.

julapy commented 11 years ago

hi, ive put through a PR with a ofGLES2Renderer that works with iOS and think it should work with android. hopefully its a start... @arturoc @andreasmuller @ofTheo - what are your thoughts?

julapy commented 11 years ago

regarding getting ofShader to work with GL ES, would this be in the form of #ifdef statements inside ofShader? or some other approach?

@andreasmuller - i saw that you had a edited ofShader version in your branch to work with GL ES, was that complete?

andreasmuller commented 11 years ago

Hi, yeah it worked with my gles2-bc branch just fine, I have to say I didn't test it extensively but I made a blur shader and a few little bits and bobs as I was dev'ing.

Give it a shot in your branch, or perhaps have a look at it for reference as my ofShader version was from 007. There is a useful #ifdef or two there by @arturoc for Android.

And of course anywhere you see:

ofGetGLES2Context()->glUseProgram

Just swap it for

glUseProgram

abergmeier commented 10 years ago

Is there some additional documentation for this? I could use this for porting to WebGL, which is ~OpenGL ES 2.0 + 3.0.

arturoc commented 10 years ago

there's support for GLES2 (and should be easy to provide support for GLES3) in the core, you just need to setup the right renderer before calling ofSetupOpenGL in main by calling;

ofSetCurrentRenderer(ofGLProgrammableRenderer::TYPE);

abergmeier commented 10 years ago

I am writing a cmake compilation, but this thing fails all over the place. Mostly with errors for not present fixed pipeline calls. E.g. in ofGLProgrammableRenderer.cpp there is no GL_MULTISAMPLE when using OGLES2.0.

arturoc commented 10 years ago

we added that specific flag almost at the end of the release cycle so it's possible that we didn't tried it with GLES2 or even GL3 on desktop.

ofGLProgrammabelRenderer should compile without problem under GLES2 so if that flag doesn't exist it's probably a bug and we should find an alternative

For other classes we haven't used ifdef's but instead we are doing the detection on runtime which and always including both versions of headers and libraries which will not work for webgl or anything that only supports gles2. So for the classes in the gl folder we might need to do some ifdef's

abergmeier commented 10 years ago

It is present in desktop OGL and OGLES1.x, but not in OGLESx >= 2.0 (see here). There is AFAIK no alternative to using GL_MULTISAMPLE. So it makes sense to have ofGLProgrammableRenderer and ofGLDesktopProgrammableRenderer and ofGLESProgrammableRenderer, with the former having the antialias switch.

arturoc commented 10 years ago

i think GL_SAMPLE_COVERAGE, should be able to do something similar. in any case we shouldn't add a new class for one line of code, we have some other ifdefs for functionality that differs from desktop to GLES.

also i think there's already an issue for webgl, can you post there? since this issue is already closed

abergmeier commented 10 years ago

Since Desktop GL is far bigger than Embedded, I would expect this new class to fill up over time. When you do not have this divide at class level, you will probably gradually converge to ifdef hell.

A few questions: Why do you not use namespaces? Why is there no build system? Should all architecture specific code go into exports? Are the modules documented in any way? What are the GL defines you use to differentiate? I noticed TARGET_OPENGL not containing the GL version.

arturoc commented 10 years ago

The idea for the renderer and in general all of our GL code is that for the most part we try to be compatible with GLES which is usually compatible with desktop GL so the ifdefs should be minimal since we don't aim at wrapping all of openGL

also we've discussed most of those points in the past, there's a roadmap towards a 1.0 version here:

https://docs.google.com/document/d/1GjgjQZCHTcOD3cT9OFSl0AZ1nhfYwwvxTl5L4OJE6qA/edit#heading=h.d22z38m4hhzj

where some of them are discussed, if you are planning to do changes like this it might be useful to go over it so you can check some of the decisions that have already been taken

About not having a build system, there's makefiles now that work for osx android and linux in libs/openFrameworksCompiled/projects but not yet for windows, contributions to that are welcome

openframeworks / openFrameworks

OpenGLES2 not working #1178

define glBindTexture(target,id) gl->glBindTexture(id,target)

ifdef hell.