google / gxui

An experimental Go cross platform UI library.
BSD 3-Clause "New" or "Revised" License
4.44k stars 298 forks source link

Support wider set of platforms. Mobile? Web? #49

Closed dmitshur closed 9 years ago

dmitshur commented 9 years ago

First off, thank you for creating this incredible Go library! I've recently discovered it and I'm a huge fan! I really like the approach of using OpenGL for platform-independent, high performance UI rendering, since it allows one to stay general and not to be constrained by each platform's UI toolkits. I've experimented with OpenGL UIs here.

Introduction

Now, the description of this project says:

GXUI - A Go cross platform UI library.

Currently, it is cross platform in that it will run and look/behave the same on OS X, Linux and Windows. However, with relatively little work, it seems it could be expanded to support more platforms: mobile and perhaps even web.

First, let me be a little imprecise and describe OpenGL APIs in this way:

OpenGL 3.2 Core API is currently used in gxui [1]. But with a few changes (outlined in considerations section), it's possible to reduce what it used so that it can mapped to WebGL API.

I wanted to see what it would take to get gxui to run in browsers (and see how it'd perform), so I made a quick little prototype tonight in just a few hours. Some screenshots:

1

2

3

Demo

Here's a demo you can try in your browser (no gzip compression on served assets, sorry):

http://dmitri.shuralyov.com/projects/gxui-tree/index.html

Approaches

There are two different approaches to supporting additional platforms that I can see.

First approach is to create additional drivers, similar to the existing gl one. This seems to be suggested in related issue #45.

Another approach, which is the one I've used so far, is a little different.

Since the three OpenGL APIs are similar enough, it's possible to create a Go package that has a single OpenGL-like API, but using build tags, it can have three different backend implementations depending on what is available on the given platform:

The github.com/shurcooL/gogl package used in my quick prototype is an example of that. It only has OpenGL 2.1 and WebGL backend implementations (incomplete), but adding OpenGL ES one would be trivial since it's a superset of WebGL. It's a WIP package with unpolished API, but I think the general approach is solid and can be turned into a production ready package.

Considerations

Here are some implementation details of the changes I needed to do to make gxui samples run in the browser.

  1. glPolygonMode is not available in WebGL (since its implementation is very inefficient; a shame because it's useful for debugging). I commented it out since it was just a debugging feature.
  2. Index buffers with uint32 types were used. It seems my WebGL implementation (latest stable Chrome on OS X) did not support that type, so I dropped it down to uint16 which worked well.
  3. My WebGL implementation seemed not to support gl.UniformMatrix3fv calls with transpose equal to true, so I had to transpose the matrix before calling the gl func. I'm guessing the WebGL implementation opts to support fewer choices for performance reasons. It'd be better to avoid needless conversions between one format to the other when rendering.
  4. WebGL fragment shaders require precision to be set, so I added this to all fragment shaders to allow them to compile under all OpenGL/WebGL versions:

    #ifdef GL_ES
       precision lowp float;
    #endif
  5. Previous commits read the font directly from disk via ioutil.ReadFile. This can't work on all other platforms, so it's better to read from a virtual filesystem and each platform can provide that. It's no longer a problem since latest version embeds the font in code.
  6. My existing gogl package happens to use a higher level-style API (similar to x/mobile/gl) with types like *Texture instead of uint32, and it favors using int over int32 or uint32 the way the low-level C-style github.com/go-gl/gl/... packages do. This is a design decision that is orthogonal to all other API changes, but it required me to make some changes like:

    -framebuffer uint32
    -texture     uint32
    +framebuffer *gogl.Framebuffer
    +texture     *gogl.Texture
    -gl.DeleteFramebuffers(1, &f.framebuffer)
    -gl.DeleteTextures(1, &f.texture)
    -f.framebuffer = 0
    -f.texture = 0
    +gl.DeleteFramebuffer(f.framebuffer)
    +gl.DeleteTexture(f.texture)
    +f.framebuffer = nil
    +f.texture = nil
    -gl.Viewport(0, 0, int32(fw), int32(fh))
    -gl.Scissor(0, 0, int32(fw), int32(fh))
    +gl.Viewport(0, 0, fw, fh)
    +gl.Scissor(0, 0, fw, fh)
  7. Consts like gl.TRIANGLE_STRIP are not const on all platforms. For example, to get that value in a browser, you either need to create a WebGLContext first (which can't be done at const-time), or just hardcode the const value from spec. I went with the latter for now.

You can see the entire code change I made at https://github.com/google/gxui/commit/469940010c8be6abd4feb535ba14a8b4e707720d. Keep in mind it's a quick prototype to test what it would take, and not a fully finished and polished version.

The performance of the web version (with Go compiled to JavaScript via GopherJS) is not as good as the desktop version with native Go. I am seeing around 20 fps. This is the result I got after doing the minimum work required to get samples to run without any profiling or optimizations. I am sure the performance can be improved if time is spent on it.

Conclusion

With just a few changes (do not use transpose true, use uint16 type for index buffers, etc.) it is possible to avoid using OpenGL-only features that are not available in OpenGL ES and WebGL. Doing that seems like a good idea as it enables gxui to be more cross platform.

This issue is to demonstrate what I've created. I'd love to hear feedback and thoughts. If you are interested, I'd love to help out with adding support for mobile and/or web and taking this prototype further.


[1] The imported package is v3.2-core/gl, so that's the API used, but since context version glfw hints are not set, it creates an OpenGL 2.1 context. Add this snippet after gl.Init() call to confirm:

{
    var samples int32
    gl.GetIntegerv(gl.SAMPLES, &samples)
    fmt.Printf("OpenGL %s %s %s; %s; %v samples.\n", gl.GoStr(gl.GetString(gl.VENDOR)), gl.GoStr(gl.GetString(gl.RENDERER)), gl.GoStr(gl.GetString(gl.VERSION)),
        gl.GoStr(gl.GetString(gl.SHADING_LANGUAGE_VERSION)), samples)
}

On my Mac, I got OpenGL ATI Technologies Inc. AMD Radeon HD 6770M OpenGL Engine 2.1 ATI-1.30.5; 1.20; 4 samples.. OpenGL 2.1.

ben-clayton commented 9 years ago

@shurcooL - wow. That's a might impressive demo you have there.

OpenGL 3.2 Core API is currently used in gxui [1]. But with a few changes (outlined in considerations section), it's possible to reduce what it used so that it can mapped to WebGL API.

This was a consideration I always had - I intentionally tried to keep the GL API to something that was relatively close to GLES 2 (and therefore, WebGL). Its great to see that you've proved the migration wasn't too hard.

Here's a demo you can try in your browser (no gzip compression on served assets, sorry): http://dmitri.shuralyov.com/projects/gxui-tree/index.html

This is so figgin cool, I woke up my wife this morning and showed her. She had no idea what she was looking at, but I know she recognized it as figgin cool as I'd normally get punched for waking her that early on a Saturday.

Since the three OpenGL APIs are similar enough, it's possible to create a Go package that has a single OpenGL-like API, but using build tags, it can have three different backend implementations depending on what is available on the given platform

I like this idea - it doesn't stop us building new drivers for non-gl target either.

  1. glPolygonMode is not available in WebGL (since its implementation is very inefficient; a shame because it's useful for debugging). I commented it out since it was just a debugging feature.

No worries on this - it's put behind a false conditional anyway. If we wanted to keep it, we could put a function behind a +build rule, and stub the unsupported platforms.

  1. Index buffers with uint32 types were used. It seems my WebGL implementation (latest stable Chrome on OS X) did not support that type, so I dropped it down to uint16 which worked well.

I think needing uint32 for indices is going to be pretty rare. I have no objections to making it uint16 by default.

My WebGL implementation seemed not to support gl.UniformMatrix3fv calls with transpose equal to true, so I had to transpose the matrix before calling the gl func.

This surprised me, and I was even more surprised to find that actually this is completely correct behaviour. The GLES 2 spec (on which WebGL is based) states: The matrix is specified in column-major order. transpose must be FALSE.

We can just make the transpose flag false and then flip the order of multiplies in the the shader to be VEC * MAT.

WebGL fragment shaders require precision to be set, so I added this to all fragment shaders to allow them to compile under all OpenGL/WebGL versions:

low_p is probably too low for high resolution mobile displays. I'd also probably inject this based on a +build rule again - GLSL shader compilers are notoriously good at throwing wobblys over anything mildly foreign, even if it is perfectly legal. I'm actually amazed that I've not seen any shader compilation failure reports yet.

  1. Previous commits read the font directly from disk via ioutil.ReadFile. This can't work on all other platforms, so it's better to read from a virtual filesystem and each platform can provide that. It's no longer a problem since latest version embeds the font in code.

Acknowledged. @crawshaw's solution is a great one, leaving this complication to the application developer. I don't believe complexities of virtual filesystems belong in a UI package.

  1. My existing gogl package happens to use a higher level-style API (similar to x/mobile/gl) with types like *Texture instead of uint32, and it favors using int over int32 or uint32 the way the low-level C-style github.com/go-gl/gl/... packages do.

Go way, way, back to Wednesday and we were using go-gl-legacy which has a similar set of APIs. I actually really liked this interface, but in order to drop the GLEW dependency that was giving so many people grief we switched to go-gl/v3.2-core.

We could switch again, but I'm guessing @crawshaw might want to get involved in this discussion first.

  1. Consts like gl.TRIANGLE_STRIP are not const on all platforms.

Really? Do you have any references to back this? I assume that's why you're copying all the GL constants into struct fields?

This issue is to demonstrate what I've created. I'd love to hear feedback and thoughts. If you are interested, I'd love to help out with adding support for mobile and/or web and taking this prototype further.

My thoughts are this is awesome and we should definitely discuss ways of making the gl driver more flexible to make this a thing. I'd like to wait for @crawshaw to share his thoughts, as he's been looking into what's involved in getting something running on mobile.

The imported package is v3.2-core/gl, so that's the API used, but since context version glfw hints are not set, it creates an OpenGL 2.1 context. Add this snippet after gl.Init() call to confirm:

Yeah, I kind of suspected this - I'll hold off making any changes until we figure out if we want to change GL libraries again.

Thank you Dmitri.

crawshaw commented 9 years ago

Nice.

You can tell I'm a supporter of a single GL ES 2 package that switches itself based on build tags. If you take a look at the internals of golang.org/x/mobile/gl, you'll see it is exactly that.

    https://github.com/golang/mobile/tree/master/gl

Right now it can be compiled (with no dependencies other than cgo+having GL installed on your system on) android, linux, iOS, OS X. It also deliberately hides all cgo types from the exported API, as I have long been wondering about a WebGL backend.

If you wanted to use that package I'd be willing to add conditional build targets for web. The other major piece of work is windows, which is https://github.com/golang/go/issues/9306.

The more interesting question is a unified input event library. I've been effectively writing my own glfw (which would be impressive if I had done windows, but I haven't yet). You can compile and run golang.org/x/mobile/app on linux+os x desktops, and input events are faked as touches.

In retrospect, it was probably a mistake to do this for desktop. What the mobile/app package should do is, for android/iOS what it does now, and on desktop, it should depend on a standard package that uses glfw.

That same standard package could be used by a gxui driver. It could use glfw on linux/mac/windows, and custom event code on gopherjs. Weirdly, on android/iOS it could use the app package. (That leads to a really weird state where it inverts its dependencies: on android gxwt depends on mobile/app, on desktop app depends on gxwt. But it will work, because the cycle never exists in a single build.)

Proposal

Thoughts?

ben-clayton commented 9 years ago

What the mobile/app package should do is, for android/iOS what it does now, and on desktop, it should depend on a standard package that uses glfw.

This sounds ideal, but are you happy for golang.org/x/mobile to depend on libraries (directly or indirectly) outside of golang.org?

I modify x/mobile/app to use gxwt in its desktop configuration.

Similar question, where will gxwt be hosted?

crawshaw commented 9 years ago

I would be happy to have github.com/google/gxui/gxwt. The mobile/app package would depend on it when compiling for desktop, but would not use it for its primary purpose, building apps. That seems fine to me.

(Ideally there would be a golang.org/x/ui, which would host mobile/gl and gxwt. But we are a very long way from knowing what to put in there.)

ben-clayton commented 9 years ago

Even if it's not used for its primary purpose, you'd still end up with go get golang.org/x/mobile pulling projects from github though, right? Are there any other external dependencies with golang.org projects?

gttp commented 9 years ago

|| Since the three OpenGL APIs are similar enough, it's possible to create a Go package that has a single || OpenGL-like API, but using build tags, it can have three different backend implementations depending || on what is available on the given platform

Is it good to have a gl.Api interface ? so that we could implement it for GL, GLES, WebGL....

crawshaw commented 9 years ago

It's an acceptable dependency for the desktop development case. It does not introduce any notices or auditing burdens on apps, as that code will not be compiled (or even downloaded) when building for android/iOS. So gxui/gxwt is OK.

dmitshur commented 9 years ago

Thank you for the feedback. I'm really glad you liked this! :D

This is so figgin cool, I woke up my wife this morning and showed her. She had no idea what she was looking at, but I know she recognized it as figgin cool as I'd normally get punched for waking her that early on a Saturday.

Haha, thanks!

I think needing uint32 for indices is going to be pretty rare. I have no objections to making it uint16 by default.

Sounds good.

This surprised me, and I was even more surprised to find that actually this is completely correct behaviour. The GLES 2 spec (on which WebGL is based) states: The matrix is specified in column-major order. transpose must be FALSE.

We can just make the transpose flag false and then flip the order of multiplies in the the shader to be VEC * MAT.

It's great that this is specified in the spec. However, I cannot find that statement in the OpenGL ES 2.0 spec, not WebGL 1.0 or 2.0 specs. Your link points to WebGL 2.0 spec (not yet implemented by most browsers), and I don't see it there either.

In any case, this is no longer an issue since it's been changed to use false in google/gxui@597c9ea32bd859b64c65ed3cc6572e92101f8135.

low_p is probably too low for high resolution mobile displays.

I'm okay with that, we can use mediump or highp. I've used the lowest setting (highest performance) as an initial starting point, and I'm ready to increase it as soon as I notice precision issues. It just needs to be set to some value for the shader to compile in WebGL.

Go way, way, back to Wednesday and we were using go-gl-legacy which has a similar set of APIs. I actually really liked this interface, but in order to drop the GLEW dependency that was giving so many people grief we switched to go-gl/v3.2-core.

We could switch again, but I'm guessing @crawshaw might want to get involved in this discussion first.

I've seen that commit, and I think it's great you moved off of go-gl-legacy/gl, thus making this package go-gettable without manual GLEW installation being required.

The main change/advantage of go-gl/gl/... over go-gl-legacy/gl is that the former is go-gettable. I am fairly certain the type change was not a deliberate change. It's just that go-gl/gl/... packages are automatically generated via glow from XML specs for OpenGL, and so they follow the C spec quite closely. But at the time, we likely did not know which API style is best in the long run, so I think the existing go-gl/gl/... API is a starting point that can evolve if needed.

(Please also note that packages in go-gl organization have been around since many years ago, and the goal has always been to make it easier for Go developers to use OpenGL and related tech. This was way before more official efforts like x/mobile/gl came around. As such, the packages in go-gl are always looking for a better home as more official Go packages and can be modified as needed. Don't think of them as something that cannot be changed and needs to be reinvented.)

Consts like gl.TRIANGLE_STRIP are not const on all platforms.

Really? Do you have any references to back this? I assume that's why you're copying all the GL constants into struct fields?

Sorry, let me rephrase that in a less confusing way.

Due to implementation details of the WebGL backend, constant values such as TRIANGLE_STRIP and others are not available as Go constants. This is because they are fetched from the existing WebGLContext which is created later on. The desktop OpenGL packages tend to hardcode the constants from the OpenGL spec rather than fetching them dynamically from the OpenGL context. They are still constant values (as defined by the spec).

Because gxui tries to define them as Go consts, it couldn't work.

My suggestion here is the following. Either use gl.TRIANGLE_STRIP directly if its value is needed. Otherwise, just use own constants like const (TRIANGLE_STRIP = iota; ...). I don't see the advantage of doing const TRIANGLE_STRIP = gl.TRIANGLE_STRIP.

But I think the WebGL backend should be changed to define them as consts, and this problem goes away.

My thoughts are this is awesome and we should definitely discuss ways of making the gl driver more flexible to make this a thing. I'd like to wait for @crawshaw to share his thoughts, as he's been looking into what's involved in getting something running on mobile.

I am really glad to hear this! This is exciting and I look forward to seeing this package become simpler and more cross platform.


You can tell I'm a supporter of a single GL ES 2 package that switches itself based on build tags.

Right now it can be compiled (with no dependencies other than cgo+having GL installed on your system on) android, linux, iOS, OS X. It also deliberately hides all cgo types from the exported API, as I have long been wondering about a WebGL backend.

If you wanted to use that package I'd be willing to add conditional build targets for web.

I think I'd be happy to use that package and add a web backend to it. That way it will support OS X, Linux, Windows (after golang/go#9306), iOS, Android and web.

My only reservation about it initially is that I didn't think it was go-gettable, unlike go-gl/gl/.... To me, having a Go package go-gettable (meaning there are no manual steps aside from go get -u; all Cgo dependencies are embedded) is extremely valuable, and once some functionality becomes go-gettable via some package, I refuse to "go back" to any other package that doesn't have this feature.

After looking at it more closely now, it appears that it doesn't require manual dependencies to be installed (at least on OS X), any more so than go-gl/gl/.... If so, that is excellent!

The more interesting question is a unified input event library. I've been effectively writing my own glfw (which would be impressive if I had done windows, but I haven't yet). You can compile and run golang.org/x/mobile/app on linux+os x desktops, and input events are faked as touches.

The key realization about GLFW is that its value is not in that it can create a window with OpenGL contexts and get input on OS X, Linux and Windows. That is the "easy" part - relatively speaking. The hard part is the maintenance and support of all idiosyncrasies of the platforms and versions it supports.

The goal of GLFW is to abstract out those platform differences, bugs and problems, and create as consistent an environment as possible for your application.

The Windows, Cocoa, X11 and other OS APIs are extremely unpleasant to work with and have tons of issues, undocumented features, bugs, etc. The development history of GLFW spans more than a decade. Just version 3.0+ of GLFW is already at 2423 commits, with 285 issues closed. Most of that work is about resolving those platform-specific issues. Supporting those platforms, and especially multiple versions, is a very large amount of work, and thus it's best to centralize it in once place. When there's a new issue that requires a workaround, it's best to have to fix it just in GLFW, rather than multiple libraries that are all equally affected. GLFW is free and open source, so I really don't see a good technical reason to avoid using it (aside from NIH, but that's not a good reason).

I like this blog post from Azul3D where it made the decision to drop a similar custom solution in favor of using GLFW.

In retrospect, it was probably a mistake to do this for desktop. What the mobile/app package should do is, for android/iOS what it does now, and on desktop, it should depend on a standard package that uses glfw.

The above is why I am very glad to hear this, and I fully support using GLFW (in the backend) for desktop rather than trying to replicate it! :)

It sounds very reasonable to use GLFW only for desktop (since that's what it supports), but the existing code for iOS and Android.

One additional (experimental) package that I used in making the above demo possible, that I haven't yet discussed, is:

github.com/shurcooL/goglfw

Package goglfw experimentally provides a glfw-like API with desktop (via glfw) and browser (via canvas) backends.

It is used for creating a GL context and receiving events.

Unstable, incomplete API: goglfw is currently in development. The API is incomplete and may change.

This package uses an approach similar to gogl. It effectively has two backends, one for desktop (OS X, Linux, Windows), and it's simply a copy of glfw. The other backend uses HTML5 APIs to implement the same glfw interface.

For instance, it uses things like document.AddEventListener("keydown", ...) to implement the SetKeyCallback, "mousedown" and "mouseup" event listeners to implement SetMouseButtonCallback, "wheel" event listener to implement SetScrollCallback. It uses requestAnimationFrame to implement SwapBuffers. It's not completely finished, but it does work and can be developed further to fill in the missing pieces.

My goal with it was to build some prototypes and if it made sense, its web backend implementation could be moved into the Go glfw package. I am still unsure if it belongs there or not.

But it can be used to fill in the web backend for the suggested gxwt package.

Proposal

  • We expand golang.org/x/mobile/gl to support WebGL and windows, and use it in the GXUI driver.

My only ask is for golang.org/x/mobile/gl to satisfy the "it should be go-gettable" requirement (to the same degree that github.com/go-gl/gl/... is go-gettable). If that's the case, I am completely in favor. I support the decision to add the WebGL backend to it and can help out.

My github.com/shurcooL/gogl package was just an experimental package and I'd love for its functionality to be made available in a more official Go package.

  • We introduce a gxwt package. It provides a pure-Go view of windowing events with no dependencies in its exported interface. It uses build tags to switch its implementations: linux/windows/OS X: glfw android/iOS: x/mobile/app web: gopherjs?

Absolutely in favor of using GLFW for the desktop backend.

Sounds good to use x/mobile/app for iOS and Android backends.

Can use the approach demonstrated in github.com/shurcooL/goglfw for the web backend. Happy to help out here.


Sorry for the long winded comment, I wanted to reply to all points.

Once again, I appreciate the feedback and the I am really liking the suggested thoughts so far. :+1: Making gxui more general and capable of running on mobile and web is a huge win (and really cool if you ask me)! The best part: it's already so close!

Thank you!

dmitshur commented 9 years ago

I've ported GLFW's event linter program to Go so I can better test the remaining functionality of the web implementation of github.com/shurcooL/goglfw.

I've put it up so you can try out the current version (open the console and watch for events; note that using keyboard shortcuts won't work since they get intercepted):

http://dmitri.shuralyov.com/projects/glfw-events/index.html

Many of events/keys are missing and there's some work to go. Up until now, I've only implemented things that I needed - so that I could test that it worked.

Also, there may be a better, concurrency-friendly interface for processing events in Go (for example, by sending them over channels), so glfw's callbacks mechanism may not be the most ideal. But it's a solid start.

crawshaw commented 9 years ago

Hey @shurcooL your goglfw looks a lot like what I was thinking of for gxwt. One extra ideal I'd try to reach for: no external references to any gl packages. That way it could be used by other drivers (for example, html5 canvas, or one of the many GL replacement languages that are appearing).

As for golang.org/x/mobile/gl, it is generally go gettable today. You need to install an opengl header on a linux machine, but that is commonly done on any system with the compilers for cgo already. The headers come with OS X's compilers. Windows is the odd system out, but it is simply not done yet. If you ever find yourself unable to go get it, please file a bug.

dmitshur commented 9 years ago

Hey @shurcooL your goglfw looks a lot like what I was thinking of for gxwt.

Sounds great!

I am thinking, to start with, gxwt should go for the best public interface, and it can use github.com/go-gl/glfw/v3.1/glfw for its desktop backend, x/mobile/app for mobile backend, and also glfw interface for web backend (implemented via goglfw initially). But later on, the web backend can be rewritten to avoid going through the glfw interface and be implemented more efficiently/directly.

Until then, using goglfw should be fine and it will allow me to test/develop goglfw and its implementation further before gxwt interface is ready.

One extra ideal I'd try to reach for: no external references to any gl packages. That way it could be used by other drivers (for example, html5 canvas, or one of the many GL replacement languages that are appearing).

I agree. This is not currently the case, but I will start thinking about how to make that possible in the future. One non-ideal approach I can think of would be something that returns an interface{} and you can use type assertions to work with it (ala FileInfo.Sys()). Perhaps there are better ways.

But the current reason why goglfw imports and uses gogl is because goglfw.CreateWindow actually creates a canvas with WebGLContext and that WebGLContext is needed for gogl to work.

Anyway, I agree and will work towards that goal.

As for golang.org/x/mobile/gl, it is generally go gettable today. You need to install an opengl header on a linux machine, but that is commonly done on any system with the compilers for cgo already. The headers come with OS X's compilers. Windows is the odd system out, but it is simply not done yet. If you ever find yourself unable to go get it, please file a bug.

Sounds good.

In that case, my plan will be the following. I'll work with x/mobile/gl on a branch (perhaps a fork on my github) and start adding a web backend to it, and see what issues I run into. Once everything works and I know what changes (if any) are required, I can start making a CL for x/mobile repo.

crawshaw commented 9 years ago

Very cool. I'm excited to see how your goglfw turns out!

dmitshur commented 9 years ago

Proposal

  • We expand golang.org/x/mobile/gl to support WebGL ...

I've finally resolved everything needed to make this possible, and created a CL:

https://go-review.googlesource.com/8793 - gl: Add WebGL backend.

/cc @crawshaw It needs review. There is a single known TODO at this time, to finish implementing all remaining (but not used in any of my code or or gxui code) methods in the WebGL backend.

The corresponding change to goglfw to use ContextSwitcher is at https://github.com/shurcooL/goglfw/pull/1.

dmitshur commented 9 years ago

I've posted an update regarding the status in the CL.

Looking at the current code, if I'm not mistaken, the only gl package test I see is in x/mobile/gl/glutil/glimage_test.go file, which renders to offscreen buffer texture and compares to expected image.

But it only seems to do it for linux and darwin, not android:

// +build darwin linux,!android

// TODO(crawshaw): Run tests on other OSs when more contexts are supported.

If so, can WebGL also be a "TODO" item?

If not, I am thinking about how it can be done now.

Perhaps something like https://github.com/stackgl/headless-gl can be used. It's a gutted version of node-webgl with no external dependencies, DOM emulation or window handling. It can be used to create an offscreen framebuffer/WebGL contexts for server-side rendering in node.

It would require node as a dependency, which I'm unsure how to feel about (but I don't know of any other way). It would also require the GopherJS compiler, which is currently not versioned. So a failed test may mean either the code is failing, or node, or GopherJS compiler.

I can also volunteer to maintain an out of tree fork of x/mobile/gl under my github or go-gl org (if others are okay) and try to keep it API compatible with the upstream x/mobile/gl. But I would want to do that with the hope that once required issues are resolved, it could be merged upstream.

I have not seen any comments on the actual code changes, so I'm not sure if it's been reviewed aside from not having tests. Suggestions on how to proceed are highly welcome.

ben-clayton commented 9 years ago

Is it okay to close this now @shurcooL ?

dmitshur commented 9 years ago

I think it's too early to close this. PR #86 was recently merged which adds a WebGL backend, but that is only a prerequisite for browser support. It still uses go-gl/glfw/v3.1/glfw which does not support browsers, so gxui cannot run in browsers as is.

I'd like to wait a few days to make sure the new goxjs/gl backend works okay and there are no new issues reported. Then I'll open a second PR to switch to goxjs/glfw. Once that PR is merged, this issue will be resolved. Does that sound okay?

cryptix commented 9 years ago

Horray!

dmitshur commented 9 years ago

It's been a great journey from initial idea and prototype to final version and being merged into gxui. Thanks to everyone for helping make this happen. :)

After 1.5 comes out, it's likely I will be playing with iOS support, and will see what it'll take to get gxui to run there also. The gl backend should support iOS as is, but there's no support on glfw side yet. There may be code organization changes that come up by then as well. But it's much closer now than before I opened this issue!

awoland commented 9 years ago

gxui looks fine ...

ben-clayton commented 9 years ago

@BigBoneDaddy - It rather depends on the hardware. Some GPUs (mostly mobile) have limited floating-point precision, and so on some high-DPI display devices rendering may not be 'pixel-perfect'. However, for those devices, the error would probably manifest themselves as tiny texture bi-linear sampling errors and you'd probably not notice them. Most desktop GPUs I'm aware of would probably be pixel-perfect, ignoring bad GL implementations.