Work: Unicode and callback tedium (two discussions)

[m-7761]

I would rather discuss this privately, but this is the de facto discussion zone :)

I want to do ambitious work on GLUI. I have 3 points:

1) Can I help close some of these feature requests? Many seem like they are implemented. Can I implement them? Does Github have a way to let me act as a moderator of some form?

2) Just a minor reputation building exercise; I think the callback registration workflow is tedious: when a control needs to do a callback, they can in most cases get by with using the parent's callback. In this case assigning callbacks to individual controls, while an important option, is a lot of code to type and if you are not assigning the same callback its spaghetti code. To load dialogs (in Win32 speak) from a file (a resource) callbacks are usually assigned after the container is loaded. In this way of thinking individual callbacks are normally reserved for controls that the container doesn't know about (workarounds) and spot fixes.

3) While I want to vastly improve GLUI; I choose GLUI because I want to minimize problems. I think it can even be improved while simplifying at the same time. I'm embarrassed by Unicode's absence in this space. I read there is actual interest in not using GLUT with GLUI. GLFW is one suggestion, but it is not interested in Unicode either. Still it might be an improvement (or it might not.)

As a more ambitious, ongoing project, I wonder if it is not more practical to render text labels to OpenGL textures and display the textures to refresh. Real-time text entry might be better done with strategically positioned native input boxes. I think freeglut's menus are a second OpenGL context. That might make them overlays; or it might be a state management strategy, but exploring overlays can be good to make a drop down menu work. GLUI needs its own menu system. If leaving GLUT behind is desired, then having text and menus covered might leave little reason to hang on (except back compatibility.)

This link explains glXCreateGLXPixmap and its Windows counterparts.

https://msdn.microsoft.com/en-us/library/windows/desktop/dd368997%28v=vs.85%29.aspx?f=255&MSPPError=-2147217396

Xim or Xlib must have text rendering APIs equivalent to Window's. The idea is to render to a system buffer and transfer the pixels to an OpenGL texture. It lets the system use its own (native) font-management strategies. I've needed to do something like this for a while to render Win32's rich-text data to a texture. I can try to get the X version working with Cygwin. But it's good to have GLUT to fall back on until all platforms can be vetted and determined reliable text services.

[nigels-com]

A few comments:

I think the callback registration workflow is tedious

Would you mind on that comment? Are you referring to the internals of the implementation, or the usage of callbacks as an application?

GLUI can even be improved while simplifying at the same time

Agreed. A few considerations from a modernisation point of view:

• C++11 language features - lambdas, standard containers, std::string
• Use of override for virtual method overrides
• utf-8 assumption for text

there is actual interest in not using GLUT with GLUI

I'd agree with that. Ideally there can be other backends for GLFW or Qt? Is there a rationale for dropping support for GLUT?

more practical to render text labels to OpenGL textures and display the textures to refresh

This raises the next question - how to render the text in a cross-platform manner. There are some libraries now available for text rendering via freetype - or pre-baked textures. Some options

• https://github.com/tlorach/OpenGLText
• https://github.com/julienr/libfont
• https://github.com/cdave1/ftgles
• http://oglft.sourceforge.net/

And so on...

Real-time text entry might be better done with strategically positioned native input boxes.

That lacks appeal from a cross-platform standpoint.

[m-7761]

I'm unfamiliar with Qt but one of the main reasons I'm interested in GLUI is that it's not Qt. I assume Qt is very expansive; but I could be absolutely wrong about that. It's just usually UI "toolkits" of its notoriety are.

My philosophy is to avoid using external libraries unless they solve an exact problem and do not have baggage and are not hard to maintain. I think there's no way to do text input justice without using the system APIs. Getting away from GLUT means taking on some of its responsibilities, like occasionally touching the supported systems. By using GLX and basic Windows APIs I think it's completely doable. And I want to see Mac OS fully supported, and that will mean touching its "objective-C" stuff one way or another. Some points for me are matters of pride. I need a cross-platform solution that actually reaches people, and doesn't discriminate as a matter of principle. Input and text are so difficult and system APIs don't require third party solutions and are generally very straightforward to configure around and well supported because the system relies on them. No surprises.

I realize that handheld devices might be another story, which is why maybe the GLUT approach, or some bare bones ASCII or extended Latin approach needs to be kept around. If developers need system support they can pitch in. The freeglut code would not be hard to copy if no one minds.

Generally I don't like these font solutions. I recognize Tlorach from their alternative to Nvidia's Cg API. But their fonts look too much like the standard approach. I just can't understand why the fonts can't work at the text level instead of the character level myself. They are just textures like anything. GLUI is just UI. Even if it was a screen reader the text can be pulled in on the fly. It shouldn't have to manage letters.

C++11 makes me nervous outside of its very basic features that were already in use before its adoption. I stay with C++98/03 unless I'm writing application software. MSVC2010 is the last to work on Windows XP. I find testing code on Windows XP always turns up interesting bugs. It has some C++11 features.

I prefer nullptr because it lights up as a keyword and I've always dreaded NULL and while I don't mind 0 I feel like nullptr is a compromise. Honestly I have a hard time reading much of GLUI's code because its written for very linear thinkers who read code like its a book. A lot of coding style grows out of how code is presented in two column papers inside of text, where it works well, but that's not where code naturally lives, and anyway, I would feel much more comfortable if I can reformat the code to be highly readable, and document it. (I realize this is a contentious topic. I don't understand why coding style isn't applied like fonts in this day and age.)

Finally, for callback registration, I mean something like better to set the GLUI_CB at the window or subwindow level and let the controls inherit that callback if they do not have their own. If they have their own then the callee can be responsible for subclassing the parent callbacks. The callbacks receive a plethora of information about the controls. It's a lot of bookkeeping to write the callbacks and parcel them out, that leads to very long lines of code that have to be typed an read and for no real reason, because there shouldn't be so many callbacks floating around. Many applications are probably fine with having one callback.

I have a lot of experience with Win32 design. I think MFC and ATL that Microsoft produced as layers on top of Win32 were miserable to work with and have gone by the wayside. Win32 is very good. But it's not cross-platform. I would like to take many principles from it. Microsoft is making its documentation hard to find these days even though it's as much as the core of Windows as ever. I would not even mind developing a layer or module of GLUI that uses it to preserve it. At some point. I really need a way to organize my UI interests so I hope GLUI works out for me. I'm a constant developer. That's a good thing to have that shouldn't be taken for granted :)

The funny thing about Windows these days is almost no applications use the same UIs. It no longer seems to have a unified system UI. They just all use about the same color schemes and dimensions. GLUI would do good to detect system dimensions and color schemes, or at least on Windows it should use the system colors (which are classic common controls colors, the same as its draw style.)

I've written a lot, but I don't mind hashing things out. I'm glad to find you here Nigel. I have been using GLUI all through the evening. I "forked" it before checking out. Just in case that's a help. I wanted to try it out even though in the immediate term I need to be working on BSD sockets and a ZIP version of COLLADA called ZAE. I'm fashioning a location bar for remote viewing with GLUI using the CommandLine control until I can connect a button attached to a drop-down menu :)

[josch]

Thanks for opening an issue here instead of writing privately. I agree with what nigel said already and am also curious about your answer to the questions raised. But maybe this discussion would be a bit more structured if you could break down the individual things you want to tackle into multiple issues? Otherwise it's easy to loose track of what you actually want to achieve.

[nigels-com]

Agreed. I'm more of an incrementalist. I'm interested in the scale of ambition on the part of the maintainer, but if there are any obvious and smallish jobs that result from the discussion, I'm more up for that.

[m-7761]

Well, I think I have some basic ideas; but to be honest, I've implemented a working location-bar, using the CommandLine control, and a subwindow, and this touched on a surprising amount of the objects. I experimented to get different results. It all took a very long time to get right.

I think there is a good library here. My sense is its usage is to create the controls and do not interact with them further, except to handle change-notifications...

If that's the correct use case, then is it safe to assume that nearly all class methods are private implementation internals (shorthand for the programmers that implemented them?) And if so is it possible to audit them and place an underscore (_) prefix in front of the broken/buggy/unusable ones?

This would give users a better idea of what methods are available, and it would radically shorten the suggestions in auto-complete pop-menus (there a quite a lot of methods to choose from.)

In any case, I can't realistically work with the library as it is. I will either have to fork it for my own use, or have consent to fully audit it. I think that would be a good improvement. But it would certainly not be backward compatible to clients that tinker with the class internals. They would have to fix their code or stay with earlier versions.

In short: I like the library, but I have to give it a quality-assurance makeover before I can do anything with it. I hope this sounds like a good contribution and not overly scary. We could talk about what should stay. Of course if a method works in the client context it can stay, and if it doesn't work, maybe a new method that would accomplish the task (with the same name) can be added.

I'm accustomed to going over code from top to bottom, but I have to reformat it in advance so I can easily pore over it. It's not very incremental, but on the other hand it's probably long overdue maintenance.

I would like to do 2 things at the same time:

1) Put the classes in a namespace, and define some global back compatibility stuff that can be disabled.

2) Get direct references to GLUT out of the classes, and document where they interact with GLUT. GLUT constants can use GLUI equivalents. (That are equal to GLUT's.) glui.h probably shouldn't include glut.h. It can be annoying if the user is including glut.h from somewhere else...

Putting '#ifndef GLUT_API_VERSION' around the part of glui.h that includes GLUT's headers is a simple fix if anyone wants to. I am not comfortable making small changes like this until I can figure out my overall disposition. I think a more-or-less rewrite-and-review effort would get everyone who's interested on the same page. I'm volunteering to do the heavy lifting. I don't see a way forward without a heavy lift.

[josch]

Depending on what you call "tinker[ing] with class internals" I'm not opposed to what you call a "quality-assurance makeover" but I can only tell you more of my opinion once you either describe more of what exactly you plan to do or show us a patch of your changes. Especially backwards compatibility would be something very important for me. I agree that internal class members which are not supposed to be used from the outside could be hidden instead of presenting them as public API.

[m-7761]

Well, I think the problems are too great to discuss them. I'm at the end of my rope for a UI solution, so I plan to rework the whole thing, and if a review doesn't work out, then I will just do a classic fork for the code. I'm trying to gauge willingness to accept change (and interest) so I can decide how much effort I should make to discuss and get input on the changes I will make.

By "tinkering" I mean calling the methods of the objects--especially controls. I found that most methods did not achieve satisfactory outcomes, and once I reviewed their implementation it became clear that there was also no way to achieve a satisfactory outcome by working around their implementation: either because members that would be required to do so are protected, or because behaviors are hard-coded and inconsistent with the logically expected behavior.

I would be surprised if any code calls on the control methods outside of a handful, because doing generally doesn't accomplish anything and was frustrating and unrewarding. You soon get the impressions that that methods aren't really there for the users; and that really most of them should either be private-methods, or everything should be public-methods so at least users can (precariously) hack their way toward more than basic use cases.

My experience was that in the libraries present state it's not meant to be expressive at all. That's not necessarily a bad thing, but with so many methods available it gives the wrong idea that it is expressive. If it's not expressive then the methods should reflect that. I think that's the case, and that an audit would probably eliminate (hide) upward of 90 to 99% of the methods that exist.

I think the debris need to be cleared away before renovations can proceed.

This process is also a good time to think about removing anything that's been marked as deprecated for a very long time. Of course if you don't have to you don't want to study them for consideration and maintenance and if the names are being changed (I generally think if it's not public it needs an _ prefix, if only because this sorts them to the bottom of auto-completion lists) users won't be able to use them in their original states.

I think if there are _ prefixes use of private and protected can be relaxed; especially if there's going to be a metamorphosis of some kind.

P.S. I wrote Nigel privately about an idea I had to use wxWidgets as an alternative backend. It sounds crazy, but I think it's a solid idea, because it can be a text-service for Unicode that uses system APIs and so is very light. It has next to no external dependencies and so what GLUI would expose of its feature set would be very lightweight to link against. I still want to do more research into using it for generating Unicode textures. That is copying from system bitmaps to OpenGL textures. Which is especially touchy on X11. But wxWidgets would satisfy Nigel's input portability concerns as much as anything could, so I think it's the logical backend, and it can let users flip a switch to get system native UI elements without any sweat off the GLUI effort's back. I want to use GLUI for very ambitious things, so I like the idea of such a straightforward solution to the most work intensives problems :)

An audit and simple 1-to-1 rewrite can pave the way for this kind of development. I don't like the wxWidgets API. So I see GLUI as a way to hide it and to create a very minimal and well defined subset of its features. So even while it may seem like a lot, the goal can still be simplification. I would think wxWidgets like talking to a virtual system. There's no avoiding systems, but they can be abstracted as if they are 1 system.

[nigels-com]

In any case, I can't realistically work with the library as it is. I will either have to fork it for my own use, or have consent to fully audit it. I think that would be a good improvement. But it would certainly not be backward compatible to clients that tinker with the class internals. They would have to fix their code or stay with earlier versions.

For what it's worth, I'm comfortable with a fairly aggressive and speculative branch or fork - rather than a step-by-step sort of approach. Let's leave room for GLUI 2.x bugfix releases.

But wxWidgets would satisfy Nigel's input portability concerns as much as anything could, so I think it's the logical backend

In principle I do also see the appeal, providing that wxWidgets is sufficiently cross-platform, and doesn't "leak" into the header as a compile-time dependency.

I think if there are prefixes use of private and protected can be relaxed_

I'd probably lean towards tightening up private and protected rather than prefixing. Auto-completion is a secondary consideration for me.

Can you push some commits to your clone of the repo, so I can get more of an impression of what you're working on? (More code, less talk)

[m-7761]

At the moment, I am trying to implement an Internet connected COLLADA viewer.

When it comes time to edit GLUI (top to bottom) it will be a major project for me that will take upward of a month to complete.

This is something I'm comfortable doing and have done many times before. I like to take raw code that is promising and overhaul it to my satisfaction, because it imposes creative constraints and affords a lattice work to work in that I think makes development more productive.

I am looking at editing the EditText control slightly to add clipboard semantics and some input enhancements. I don't know if that will be worth a pull-request ahead of a rewrite though. I know it's valuable but at most I can do Win32 and if I get to a Cygwin build I will see about Xlib copy-paste. There's no benefit to a Cygwin build since it's Windows too, but it demonstrates broader portability until a proper back-end arrangement can be had. (I was just looking at Xlib code for this.)

Auto-completion is a secondary consideration for me.

I think this is primary. There's no reason to choose C++ unless it's to leverage namespacing or overloading. Namespacing can avoid collisions, but it's more important for providing menus of identifiers to choose from, because without that you are procedural programming in a single namespace, and this->x is equivalent to x(this).

I'd probably lean towards tightening up private and protected rather than prefixing.

This just ties client's hands in a pinch. It only makes sense if you believe that all functionality is fully exposed. I think only very mature libraries with well defined problem spaces should ever do this. Using _ is the only acceptable prefix in my book. If client code accessing these members it's immediately clear that they are "hacking" to some end (probably to make their UI appear polished) and it neatly divide the auto-completion menu. That menu can be very effective for navigating code, especially when it provides documentation, because we cannot memorize APIs and we need to be able to rely on the methods to behave intuitively, and their documentation to describe unintuitive behavior.

Can you push some commits to your clone of the repo, so I can get more of an impression of what you're working on? (More code, less talk)

In terms of code to talk this talk will be 0.0000001% of my work. It helps to gauge the happiness of parties.

I'm not going anywhere. I have goals to meet and I'm finished with my UI research phase. But there are things I must do before I can schedule a full audit of GLUI. I will complete the work and then present it. If it's liked I will be glad. If not, I will carry on independent of GLUI. I will transform the code base without making functional changes. I will document each member including deficiencies.

I want working with GLUI to be an efficient use of my time before my work with it progresses from demos with a location bar to full blown editing software.

[nigels-com]

Uh, so are you intending to test on Linux and Mac, or is this purely a Windows effort?

[m-7761]

Uh, so are you intending to test on Linux and Mac, or is this purely a Windows effort?

I'm not a one man laboratory. I'm just one user. But my goal is cross-platform or I would not be shopping for cross-platform solutions. I have two headless Linux machines that might turn on, but I don't know if it's worth my time to. I don't have Apple products or any other platform. I'm more of a developer than tester.

I have a colleague working on COLLADA stuff that uses Apple, and they can basically help us at least to say if feature-x works on Apple systems or not. Anyway, the changes we are discussing in the near term are either non-system or would be bonus features (copy and paste) that users would otherwise not have.

The little bit of UI work I'm doing right now will probably all be outmoded if we agree to attach wxWindows. But that's going to be a while. Only copy and paste requires changing GLUI's code, and only because it does not have input hooks. Maybe I gave the impression that I was adding download and webcaching services to GLUI as part of a (new) location-bar control. We can definitely talk about doing that, but I'm just barely using GLUI for the edit box, and am just barely able to make the box span the width of the window as is. (I have it set up to automatically hide away, which only works because I don't want the viewport dimensions to change when it's hidden--as a subwindow--so it's just drawing on top of the window. I'm used to having finer control over the UI than GLUI affords. There will be time to debate these things in the future though. As long as anyone is interested in discussion.)

I'm probably starting on copy-paste this evening. I'm thinking there should be a singleton GLUI_Clipboard object so that whatever functionality it provides can be used independent of the GLUI_EditText. I'm going to add that and change the control's keyboard handler to include it, and also to advance the cursor from the end of the selection when pressing the -> arrow keys. Inputting URLs by hand is impractical, so copy-paste it is.

[nigels-com]

Well, I tend to be hands-on with Linux and Mac most of the time. Perhaps a good start for cross-platform would be to roll out some Travis coverage for GLUI so it's clearer when the Linux and/or Mac build gets broken.

[m-7761]

What I should have said, is that Cygwin is a single platform, and POSIX is a plethora of platforms. And to be honest when I develop with Cygwin in order to deploy to Linux boxes (servers) I never run into problems. I'm not sure about the ins and outs but I think POSIX is mostly straightforward. In hindsight I don't think it should require further testing.

I think we should be glad to have a fervent Windows user instead of worrying about compatibility bumps in the road. I don't know what "Travis" is; and I'm trying to figure out how to include build-environments in open source packages but I'm having a hard time making heads or tails of it.

[nigels-com]

I'm not suggesting you ought to be conversant across platforms in order to contribute to GLUI. I'm pointing out that we can put some infrastructure in place to notify us when there are compiler and link issues that creep in. Cygwin is good as a sanity check, but Travis can offer more coverage for a modest setup investment. I'll take a stab at that over the weekend.

To expand on that slightly, modern Mac builds are clang and Linux is gcc 6. Both of these tend to be pickier than MS compilers, generally speaking. We agreed that GLUI shall remain C++98 and enforce that, or should we allow C++11 also?

[m-7761]

What I'm up for is C++98 but with nullptr but not in its template-capacity; such that #define nullptr 0 always gets C++98 clients over that hurdle.

What that means is setting GCC (for example) to a hard C++98 is probably out of the cards and I really prefer to not have to write >> as > > to instantiate a template. Usually C++03 is synonymous with 98. The follow up versions only address things left out of the previous. I don't think any compilers truly implement C++98 even. Microsoft's certainly doesn't.

P.S. Like I say, I'm not against C++11. It's just I'm more comfortable squarely outlining which features of it will be used. I guess with CMake it can generate configuration files to tell you what features are available. C++11 doesn't have any native way to detect the presence of its features. Visual Studio is missing a lot.

[m-7761]

Bonus reply:

To expand on that slightly, modern Mac builds are clang and Linux is gcc 6. Both of these tend to be pickier than MS compilers, generally speaking.

I'm not sure, but I don't know if Visual Studio works as a Cygwin compiler. I have to use Netbeans or something when I use Cygwin. I try to use Visual Studio as much as possible because it's simply a tremendous time saver compared to any other editor (IDE) I've ever come across. Visual Studio has a "Disable All Extensions" option, which is very good, but it can't be used if "windows.h" is included. (There's no technical reason for Microsoft's headers to depend on extensions--they just do.)

[nigels-com]

Concerning C++98 and/or C++03 and/or nullptr I'll take advice from the maintainer. I'd rather a project specify the language variant and have Travis coverage aligned with that, if possible.

The advantage of having automatic builds is to enable portability by lessening the burden on any particular developer to actively build on a variety of targets. I wouldn't rely on Visual Studio as any kind of guide to what clang or gcc will be happy about.