Does Harman expect to transition from OpenGL to Vulkan or Metal for ios in the near future?

[Dimensionscape]

With apple depreciating opengl on their operating systems, and presumably dropping it completely in the very near future, does Harman intend to transition from OpenGL to Vulkan or Metal? If so, what is the timetable for this?

[ajwfrost]

We've been waiting to hear the schedule from Apple about the deprecation/removal of the OpenGL ES APIs. The preparatory work for this has been completed already, the expectation is that we will use an intermediate library from MoltenGL in order to provide a runtime conversion from OpenGL ES to Metal.

For anyone not using Stage3D, the alternative option is that we'll provide a version without OpenGL ES usage, which means that they can continue to have applications that work without needing MoltenGL's library..

A slightly open point is whether the use of MoltenGL will mean that part of the development needs to be done on a Mac. I've just looked and we've got a large number of people still using iOS development on Windows, so the goal would be to set this up so it's possible on either platform - but this is then a little dependent upon an appropriate agreement between us and Molten to allow us to distribute their binary..

Hope that helps...

[FliplineStudios]

@ajwfrost Great to hear there are some options here! Could you clarify if this MoltenGL solution would hypothetically work for vector-graphics-based "GPU" render modes as well, or would it be limited to only Stage3D?

[ajwfrost]

@FliplineStudios it would work for GPU rendering I believe - but I'm not sure whether that's actually something that happens now. The below link shows that GPU falls back to Direct mode on desktop, but I have a feeling this also happens for mobile.. I'll check this!! https://help.adobe.com/en_US/as3/mobile/WS901d38e593cd1bac-3d719af412b2b394529-8000.html

[FliplineStudios]

There's definitely a big performance difference in our tests between CPU and GPU and Direct on mobile, with GPU being vastly better on both Android and iOS for vector graphics, so I'm assuming it's still doing something different these days. Newer iOS devices are at least more powerful, but on anything before iPhone X and nearly every Android device it's a noticeable difference.

I do remember a blog article from years ago from one of the Adobe engineers who worked on the GPU Mode rendering specifically for iOS because it was built differently from the GPU mode used on Android, but I can't find the link again for the life of me :)

[hardcoremore]

We've been waiting to hear the schedule from Apple about the deprecation/removal of the OpenGL ES APIs. The preparatory work for this has been completed already, the expectation is that we will use an intermediate library from MoltenGL in order to provide a runtime conversion from OpenGL ES to Metal.

I was just reading about MoltenGL on their website and they are only mentioning Open GL ES 2.0. Do they support Open GL ES 3.0?

My application would be unusable if there is no support for Opetn GL ES 3.

Regards,

Caslav

[zwetan]

but I can't find the link again for the life of me :)

maybe this post from Tinic Uro Why Starling (or any other 2D framework on top of Stage3D)?

eg.

Approach #2: On mobile, which includes Android and iOS, we have ‘renderMode=gpu’. Unlike the ‘wmode=gpu’ on the desktop this mode renders vector graphics directly in the GPU using OpenGLES2. This mode is still available to you today on Android and iOS and we see some content using it. Content which is using ‘renderMode=gpu’ successfully sticks to a very small subset of the classic display list APIs which looks eerily close to the subset Starling provides. And yet there is a higher cost overall in the Flash Player than if you would just be using Starling due to the many layers involved to emulate some classic display list features. In short: You are likely better off using Starling going forward for new content.

[Dimensionscape]

@ajwfrost

For anyone not using Stage3D, the alternative option is that we'll provide a version without OpenGL ES usage, which means that they can continue to have applications that work without needing MoltenGL's library..

Does this mean that potentially, in the worst case, the solution will be to just release a SDK without being able to use stage3D for ios/macos?

[FliplineStudios]

but I can't find the link again for the life of me :)

maybe this post from Tinic Uro Why Starling (or any other 2D framework on top of Stage3D)?

Thanks, that could be it, I swear there was another that went into detail about all the work Adobe did on GPU render mode for iOS and how exactly the vector graphics were turned into triangles and meshes and updated on the GPU under the hood, I'll have to check the other articles from that blog on wayback machine. I know GPU rendering with vector graphics isn't as common of a use case as Starling/stage3D with direct, but as long as you're mindful of its constraints it still is surprisingly powerful.

[teotigraphix]

@FliplineStudios

but as long as you're mindful of its constraints it still is surprisingly powerful.

Is this pertaining to the nativeOverlay in Starling (gpu vector graphics)?

[FliplineStudios]

@teotigraphix Sorry for the confusion, I was talking about using only the regular Flash Display List with vector graphics and bitmaps running with GPU render mode (not using Starling or stage3D at all), and wondering if the MoltenGL solution would cover that as well.

[Dimensionscape]

We've been waiting to hear the schedule from Apple about the deprecation/removal of the OpenGL ES APIs.

Also, Apple heavily implied that Catalina and iOS 13 would be the last platforms to support opengl at their WWDC event last year. First minute into their talk: https://developer.apple.com/videos/play/wwdc2019/611/

They issued a strong warning to developers that "now" was the time to move, and that was a year ago. Please look out for any possible announcement regarding the future of opengl on succeeding ios 14+ and 10.16+ (arriving in Q4) operating systems next week(beginning on Monday June 22) at this years event.

[zwetan]

hi @ajwfrost

few questions for clarity sake

The preparatory work for this has been completed already, the expectation is that we will use an intermediate library from MoltenGL in order to provide a runtime conversion from OpenGL ES to Metal.

with this I understand that AS3 dev could just keep using Stage3D for macOS and iOS and they will not have to change their AS3 code?

and

For anyone not using Stage3D, the alternative option is that we'll provide a version without OpenGL ES usage, which means that they can continue to have applications that work without needing MoltenGL's library..

this means for AS3 dev only using the classic display list and not caring about Stage3D, they will have the option to not use/include that MoltenGL dependency?

thanks

[ajwfrost]

@hardcoremore I will double-check on OGLES3 but my understanding was that all the APIs of OpenGL ES that AIR currently uses are supported by MoltenGL..

@Dimensionscape yes that would be a worst case scenario though!

And yes, the preparation to move away from OpenGL ES was done a while ago, but we don't want to trigger this until we have to - will be watching WWDC with interest!

@zwetan yes this will all be completely transparent from the application's perspective, just keep using Stage3D without changing the AS3 code. And if you don't use Stage3D (and possibly GPU mode) then you'd not need to include the MoltenGL library, the auto/cpu modes would not call any OpenGL ES APIs.

thanks

[hardcoremore]

@ajwfrost do you expect performance impact of using MoltenGL lib. I read on their site that their API is faster than OpenGL in many cases. But if you use MoltenGL lib is that another layer on top of Stage3D. Do you expect performance penalty or performance improvement when using MoltenGL?

And what is the biggest reason behind even using MoltenGL. Why are you unable to map Stage3D API to Metal API like it was done for DirectX11?

[denisgl7]

Are you planning to switch to metal for iOS & Mac?

[zwetan]

the next version of macOS Big Sur or macOS 11.0 will supports OpenGL and OpenCL on both Intel and ARM CPU architecture.

while OpenGL is considered deprecated it is not removed.

[denisgl7]

zwetan What was said about iOS?

[JohnBlackburne]

zwetan What was said about iOS?

I have looked out for any news on this, in the WWDC presentations and documentation, but have found nothing. The beta release notes in particular do not mention OpenGL. This all strongly suggests to me that there are no changes to OpenGL - except for the obvious one that it is falling further and further behind Metal which sees more improvements.

[hardcoremore]

except for the obvious one that it is falling further and further behind Metal which sees more improvements.

I don't know much about those lower level GPU APIs but what would be the main features/improvements that are missing from Open GL 3 and are implemented in Metal from which we will benefit greatly from?

[JohnBlackburne]

There was one in a video I watched yesterday, Build GPU binaries with Metal which addresses a particular cost of rendering: shader compilation. Often shaders are delivered to the user as source code or in an intermediate format, then compiled on a user's machine to target their particular GPU. But this can take time, leading to long app launches, or at least first-runs. The new feature lets you compile the shaders in advance, then distribute them in binary form to users. The cost saving can be dramatic for an app with a lot of shaders, e.g. a lot of materials.

That's just one example, there are many more. It's a bit harder to find things this year as there are far more session videos. Apple seem to have taken advantage of the unique circumstances to produce much more of them, not being constrained to what can fit in a limited number of conference halls.

[DrNeroCF]

@FliplineStudios. Hey! It's been awhile!

I've been asking Adobe about triangulating vectors for over a decade now, haha.

Scaleform did it, The Behemoth's custom software for Alien Hominid and Castle Crashers did it, and now Unity can do it with SVG graphics. Adobe, however, wrote a blog post about how it was difficult to get the same results as CPU rendering on the GPU or something like that.

Not to derail the thread, but unless I'm mistaken, the mobile GPU mode always cached everything as bitmaps and rendered those on the GPU (and cacheAsBitmap is like, 100x faster than draw(). If there's anything faster about iOS's GPU mode, I'd be willing to bet it's because of the OoT compiler on iOS and not on Android.

Would be curious about the performance implications of MoltenGL, if that's still faster than OpenGL, and what the speedup looks like on Android (though I'd still appreciate the ability to limit the resolution on all those fairly weak 4k Android phones out there, if that's possible).

[JohnBlackburne]

Well, yes, it would have been very difficult to implement the classic API on the GPU. Large parts of it would have been impossible due to the fundamental differences of GPU and CPU rendering. Few existing apps would work without large amounts of rewriting. In the end they did not go down that route. Instead they implemented a low level API, and supported different approaches to building on top of it. Starling, a 2D API which matches the classic API as much as possible; Away3D for 3D rendering.

It seems any deadline due to OpenGL being removed has been put off for at least another year. By the time AIR switches away from OpenGL I am sure it will perform at least as well - I would not like to speculate about any performance improvements (I have read the claims on the MoltenGL site but am sceptical).

[DrNeroCF]

I mean I'm sure it wouldn't be easy, but like I said, Autodesk did it, The Behemoth did it, Guacumelee draws everything animated with vectors, Unity has a great SVG to triangle pipeline. AIR / Flash is the only thing in existence that's still CPU accelerated.

Oh yeah, and AwayJS can run everything triangulated on the browser as html5, yet another thing that Adobe won't do.

I would believe that would be difficult if Scaleform didn't exist. Almost no code rewritten, everything triangulated. I would have used it for more games if it was better supported, and hitTest wasn't so slow on the devices that were out at the time.

I did Super Fancy Pants almost entirely with Starling. Main character was still vector, because, well, there's thousands of frames of animation, and that's AIR's strongest suit. Developer's are literally unable to utilize the best aspect of Animate / AIR above a resolution of 800x600. That's just nuts.

[hardcoremore]

Hi @ajwfrost,

What is the status of this?

Is it known that iOS 14 and Mac OS Big Sur supports OpenGL 100% and there is no worries?

Thanks

[JohnBlackburne]

Is it known that iOS 14 and Mac OS Big Sur supports OpenGL 100% and there is no worries?

Every indication is that OpenGL and apps that use it will keep working. Apple would definitely have announced it, if OpenGL were being removed from iOS or macOS.

[marchbold]

Not sure if this is any consolation but we have been testing our apps on iOS 14 and haven't seen any issues rendering so far.