About OpenGL ES maximum texture size

[doctb]

Hi again!

I am currently implementing a photo roll image picker binding and thereby I met an issue that seems to be pretty common: the whateverImage larger than MAX_TEXTURE_SIZE followed by image.onload called but with empty data.

About the limitation, the maximum texture size is different between the devices (pre-3gs was 1024, everything until today 2048, and the iPad3 4096) but seems to be retrievable with glGetIntegerv and GL_MAX_TEXTURE_SIZE. So I was thinking that it would be a nice thing to provide an option allowing EJTexture's loadPixelsFromUIImage to automatically downsize images bigger than the MAX_TEXTURE_SIZE.

Of course in my case I can implement this directly in the image picker, but I think it may not be the only situation where the programmer wants to load an image with an undetermined size until loaded (like an image from internet, a Data URI retrieved from a save that may have been created on another device, etc.).

What do you think about it? I am not asking to you to implement it but just if you are cool with this idea so I would give it a shot.

If you are, should it be an opt-in option (the picture quality is going to be lowered so it should be an explicit choice of the developer) or not (assuming most developers want this behavior)? And how to expose this option? On ejecta's object like ejecta.GL_MAX_TEXTURE_AUTO_DOWNSIZE or on the ctx object, or in a configuration's plist?

And about what that seems to be a bug (the onload fired) I will try to have a look.

Cheers!

[phoboslab]

whateverImage larger than MAX_TEXTURE_SIZE followed by image.onload called but with empty data.

Well, the fact that it calls onload instead of onerror is a bug. Shouldn't be difficult to fix I think. We'd just have to abort texture creation when the warning is logged. I'll look into it.

(pre-3gs was 1024, everything until today 2048, and the iPad3 4096)

Ejecta only runs on ARMv7 (3GS and newer), so I guess we can safely assume a minimum texture size of 2048px.

So, to answer your initial question: I don't think automatically downsizing images (at least those loaded from the file system) to fit into this limit is a good idea. Resampling images is a costly operation, especially at larger sizes. I believe the max texture size is an issue the developer should consider.

If we were to fix this issue, I would rather subdivide large images into smaller textures and transparently handle those as one big image. I believe this is what browsers are currently doing. Of course this is way more complicated than just downsampling an image, but it provides the same quality on all devices and has no "bad surprises".

So, right now, I would propose that the image you pull from the camera is automatically downsized to fit into this texture size limit. Or even let the us specify the desired image size in JS.

There's currently no browser API to access the camera roll, right? So your API design is not based on anything existing?

[doctb]

I see, you're completely right.

So I am going to add this to the picker, and no, I don't design this on anything existing.

HTML5 provides <input type="file"> tag to let the user select a media. It's implemented in mobile Safari since iOS6, and let the code access a picture selected by the user in his photoroll. But I think it's meaningless to try to mimic something really DOM related, where the developer would have to fake click events to open the window, etc.

There is also getUserMedia() but this is for accessing camera and mic only, not the photo library.

But yes... maybe instead I should try to mimic how cordova did it: http://docs.phonegap.com/en/edge/cordova_camera_camera.md.html#Camera. There is already all the options I implemented so far and also things I was thinking to do. And it's providing also exactly what you talked about :

targetWidth: Width in pixels to scale image. Must be used with targetHeight. Aspect ratio remains constant. (Number)

targetHeight: Height in pixels to scale image. Must be used with targetWidth. Aspect ratio remains constant. (Number)

[phoboslab]

Yeah, providing <input type="file"> may be quite counter intuitive and doesn't let us specify any options.

However, I'm not sure about the Cordova API either. Specifically, I think it's weird that you get a Base64 encoded image data string instead of the image itself.

Correct me if I'm wrong - I think the most common use case when getting an Image from camera roll is to actually draw it on a Canvas or use it as a texture for WebGL. So I think it's quite backwards to first encode a JPEG, than encode the JPEG data with Base64 only to decode the Base64 string again, decode the JPEG to get the pixel data and finally upload it to the GPU as a texture.

Also, with the way Ejecta currently handles Data URIs for images, the original Data URI string is always kept in memory for as long as the image exists. Providing an actual image should not only be faster but also more memory efficient.

So I'd propose something like this:

var cam = new Ejecta.Camera();
cam.getPicture(function(error, image) {
    if( !error ) {
        ctx.drawImage(image, 0, 0);
    }
}, options);

Where options would be an object with .width, .height etc. Note that this API only uses one callback, with an error parameter that is either an error string or null, similar to how In App Purchases and GameCenter works in Ejecta.

Getting this all right is a bit fiddly, I believe: Creating the Image object in Obj-C, making sure it's properly loaded before calling the getPicture callback, making sure it can be handed over to WebGL (may need to call EJTexture's .pixels method) etc. But I'm happy to help with anything; filling in the details if you get this working in principle.

[doctb]

That makes sense. My goal with the Cordova API was to try to simplify the life of developers (like me) who are trying to get their apps on multiple platforms by using different wrappers.

But right, it's better to fit Ejecta's style, and in fact I prefer the Node.js single callback syntax more than the Cordova's double one.

What I don't know yet is how to return an image object from an UIImage. But by having a look here and there in the code I should be able to figure out and if not I will ask you.

Thank you a lot for the help!