progress during cubeface creations

[jordyno]

Is there a way to check on progress during calculations? Tried to implement it myself, but the DOM freezes until the calculations are complete. The only progress I am able to output is to the console. This becomes an issue with large files as the browser just freezes and user is unable to check on face or line being calculated.

[thomcc]

No, there isn't. If they're static images, I recommend doing it offline.

Adding that would require waiting for the event loop periodically, which would add a ton of overhead. It would also force all callers to be asynchronous, which is really not acceptable for some usecases (e.g. I wouldn't use a lib like this that forced it on me).

If you can ignore non-modern browsers, I could probably show you the easy way of what you'd want to do in a gist or something, but I don't think I'd merge it because of these reasons.

There are other options (Running the script in a worker) but they suck for other reasons. Progress bars for CPU-bound code on the client are not easy in JS, due to it's completely single-threaded nature.

[jordyno]

At best would be if I could do it online, it will be part of an online tool. How does Marzipano Tool do this then? The algorithm seems faster and also nice progress bar shows current face and tile being processed. However, it does not handle transparent PNGs just like your script flawlessly does!

I don't mind ignoring non-modern browsers, what is your idea?

[thomcc]

I'd guess they use the webworker solution. It's by far the best solution, in terms of both flexibility, and performance. But it's also by far the most work, and requires the code be both separated out and asynchronous. They might not use that, and do the additional work to make the asynchronous solution not slow. It's hard for me to say without reading their code (which I don't really care to do). In general though, stuff will often be more flexible to your needs if you build it for-purpose for your project, instead of using an off the shelf library.

Regarding being faster, I'd also guess they've spent more time optimizing theirs. I wrote this library in a weekend when I was between jobs and haven't looked at it meaningfully since. The amount of time spent optimizing it was about as much as was required to satisfy me at the time, but I also know a lot more about JS engines now than I did when I wrote it

They also might not be gamma-correct, mine is, but that comes at a nontrivial cost. I'd guess that if they don't handle alpha properly, they probably don't do their blending in linear space, like you need to do to be gamma correct. Or they might not even blend multiple pixels and just choose nearest neighbor (this would probably be very fast, but look somewhat worse).

This is mostly speculation though. I can't say for certain.

I'll come up with an example of how you could add progress information in the way I meant after I'm completely done with work today.

[jordyno]

Many thanks for your effort! I checked their code and it looks like they are using web workers just like you say. And no, they actually don't handle alpha at all, it's JPG only. But the resulting panoramas are of very high quality. So you wrote this in a weekend? I'd probably need at least 2 weeks! :D

[thomcc]

I pushed a new version, it should be about 2x faster. But more importantly for you, it also added a new function that you can use to implement some sort of progress, there is a new example that shows roughly what you'd do.

Any faster would have to not blend colors I think, and it's possible they do that. For a large image you probably wouldn't be able to tell.

Please close this bug if this works for you.

[jordyno]

just so you know what I am doing: I wrapped your code around my functions to first generate cubes from the loaded equirectangular panorama (usually 15000x7500 px) to start with 4096 or 8192px tiles. After the high res cubes are generated I use my functions to resize and split the tiles using canvas to 512px partial tiles of each cube by level. That is, top level cube of 4096px has 4x4 512px tiles, next smaller cube is 2048px and has 2x2 512px tiles, and so on.

[jordyno]

wow thanks so much! will check it out!!! you are awesome! :)

[thomcc]

Err, why not do it smallest first? Then you would have something to display while you compute the larger ones. That said, I think the only way you'll be able to generate 4096px or 8192px cubemaps without freezing the browser is if you do it in a worker.

I guess I probably could get the work to make it work in a worker done over a weekend when I don't have anything else planned, but that would have to be the weekend after this one.

Although, TBH, I wouldn't sweat freezing the browser in initial proof of concept apps.

[jordyno]

well, what I exactly do is, that I generate small 256px cubefaces for a preview first, which is pretty fast. And after that I go to the top level and start with the biggest 4096 / 8192px cubefaces, and after that I just resize the canvas every time I go for a lower level as that consumes less CPU than generating cubefaces from the bottom up, am I right? Will test the new version tomorrow and provide you with feedback, many thanks again!

[thomcc]

just resize the canvas every time I go for a lower level as that consumes less CPU than generating cubefaces from the bottom up, am I right?

It's only faster if you actually use the big ones. But if you do, then yes that would be much faster.

[jordyno]

2x faster, really? JPG 6000x3000px; old code: 2m40s, new: under 15s! PNG 10000x5000px; old code: 16m, new: under 20s!!! I have no words, just WOW! Did not look at the exact changes you made yet, but the performance gain in my scenario is just phenomenal! Had to test it twice to believe it! Great work!!! Will try to use this with a worker and see how that comes out, but also the transformSingleFace function is extremely handy. Is there a way to break the face computing loop in case a user cancels the action, or is this achievable via worker?

[thomcc]

Ah yeah, it was 2x faster for my test image. It makes sense that bigger images would get a bigger speedup, though.

No, there's no way to break the face computing loop. If you have a worker you can terminate() the worker, though.

Will try to use this with a worker and see how that comes out

You might be able to just importScripts('path/to/equirect-to-cubemap-faces/index.js') inside the worker, and then equirectToCubemapFaces.transformSingleFace should work in a worker. The main equirectToCubemapFaces function won't.

Getting the data in and out of the worker is a pain though, and usually pretty slow (transferables can help). What you can do inside a worker is also pretty limited (can't touch HTML elements, IIRC you can't even log to the console), and debugging them sucks. Overall it's not my favorite API.

It's worth noting that sending enormous images to webgl won't work on a large number of machines (see http://webglstats.com/webgl/parameter/MAX_TEXTURE_SIZE -- in practice 4096 is safe but if you want to use higher you should query the MAX_TEXTURE_SIZE first)

[jordyno]

Many thanks Thom!!!

[jordyno]

Further testing revealed that while your code is still slower than Marzipano Tool, the color representation of the cubes is much more accurate - and that might be as you said - because theirs is not gamma-correct. My implementation of your code is still overall faster than Marzipano Tool though, because I just generate 256px faces and the 4096px ones and then resize them by resolution level, while Marzipano Tool goes from the bottom up. So the speed is not an issue anymore, it's quite fast. The only problem I am having now is trying to generate 8192px faces. Both Chrome and Opera crash at some point during the process - I would guess that at least one face gets computet though. Any idea how to overcome this limitation? Maybe computing each face separately? I am still using the original equirectToCubemapFaces function.

[thomcc]

Does Firefox crash too? I'm in a better position to help you diagnose a Firefox crash than a Chrome/Opera crash (my day job is working on Firefox).

I'd imagine it's out of memory? 8192px 8192px 4bytes per pixel * 6 faces is about 1.6GB of memory for all the faces. Which is a lot. Computing each face separately would probably help, since it would allow us to avoid creating redundant canvas objects for each step, so long as you compute the face, then upload to WebGL, then compute the next face, etc.

You should strongly consider if you need 8192px faces though, 1.6GB of RAM is a lot of ram to use if you can avoid it. Even 4096px sounds like overkill to me. Keep in mind the resolution these will be displayed at when they're actually on screen.

If it's crashing during upload to WebGL, then you're probably surpassing the MAX_TEXTURE_SIZE for your GPU, as I mentioned before.

[jordyno]

I really need the 8192px to provide high resolution panoramas creation for regular users with better PCs. And if Marzipano can do it, I am sure your code can too! :) PNGs could be good enough with 4096px as they provide overlays only. This is interesting though: Firefox - 23000px x 11500px JPG (8192px faces) worked Firefox - 11500px x 6750px JPG (8192px faces) worked Firefox - 10000px x 5000px 32bit PNG (8192 faces) worked (will test at higher res later) Chrome - crashed with all above, limited to 4096px faces with both PNG and JPG. Marzipano tool has no problems with 8192px faces (JPG only) in any modern browser though, I assume (tested only FF and Chrome).

However, the color quality of your code ran inside FF compared to Marzipano Tool results is nearly invisible. There is only very slight color shift (just a bit more vibrance). Going back to Chrome and 4096px faces showed significantly higher quality image with your code. To make this even stranger, I ran Marzipano Tool on FF and Chrome with the same file. FF showed more vibrant colors and Chrome a bit faded ones. It's weird, because that means: Your code on Chrome (4096px only though) - better colors Your code on FF - normal, a bit faded colors Marzipano on Chrome - normal, a bit faded colors nearly the same as above Marzipano on FF - better colors - I would guess the same as your code on Chrome

Now, my implementation of Marzipano Viewer with the resulting faces from your code displayed in FF shows something like vibrance to the second power. It's kind of too much. But could be also the way I save and zip tiles from resulting cubefaces via canvas? Maybe something I am missing?

I can post screenshots if you would like to see the color shift.

By the way, how to find out if the code is crashing during upload to WebGL? The VRAM should not be the problem in this case, I've got 3GB (Geforce GTX 770M).

[thomcc]

Screenshots would help.

[jordyno]

www.powderia.com/capturechrome.png www.powderia.com/captureff.png

[thomcc]

That's very strange. Can you give me a copy of the source image? I'll try and look into it.

[jordyno]

and this is what it looks inside my viewer with your code's resulting cubes

[jordyno]

weird, the last PNG link I supplied did not look so vibrant when opened in FF nor Chrome...

here is the pano: pano1full.jpg

[thomcc]

It sounds like there's a color space issue but I'll have to look into it later, thanks, this should be enough for me to be able to investigate.

I should be able to give you a version to try that skips the bilinear blend step (and thus avoids needing to worry about gamma correctness). In general this has worse results, but for enormous output sizes like yours, it is unlikely to matter.

On Thu, Oct 19, 2017 at 3:38 PM, jordyno notifications@github.com wrote:

weird, the last PNG link I supplied did not look so vibrant when opened in FF nor Chrome...

here is the pano: pano1full.jpg http://powderia.com/pano1full.jpg

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/thomcc/equirect-to-cubemap-faces/issues/12#issuecomment-338014513, or mute the thread https://github.com/notifications/unsubscribe-auth/AA0h-WGVhT7Fw53vIeNco4XuLtNeDQlPks5st6U5gaJpZM4P8oIg .

[jordyno]

thank you!

Maybe I could use the bilinear blend step for lower-res cubefaces and go without at higher-res ones.

Will continue with tests. Even Adobe Illustrator has issues outputting PNGs bigger than 10 000 x 5000 px ;)

One more interesting note that might help you: When generating transparent PNG tiles and faces with your code, there are some faces and tiles which contain no graphics at all. Inside FF, your code gives a clean, only 1096 byte sized PNG output, while Chrome gives always 5.617 byte PNG. Quite a difference - even though not an issue for me - will try to make the viewer ignore "empty" tiles like these. Samples are here: Chrome Firefox

but this might also be a browser issue that occurs at the point when I am saving the canvas (after your code generates the cubes)

[thomcc]

When generating transparent PNG tiles and faces with your code, there are some faces and tiles which contain no graphics at all

Could you file a separate bug for this? Also with the image that was used to generate the empty tile (and which tile it was, ideally).

[jordyno]

will do :)

[thomcc]

I pushed a version that allows you to use nearest-neighbor interpolation, which should be slightly faster, at the cost of looking slightly worse. If you provide { interpolation: "nearest" } as an option, you'll get this behavior.

Using nearest-neighbor should also avoid any colorspace conversion inside my library (e.g. we don't blend at all, so there's no reason we need to convert colorspaces to blend without losing gamma-correctness), so if the color vibrancy difference is happening to your images, it's happening outside of my library. Sadly, I expect this to be the root of your issue with vibrancy.

createImageBitmap offers a way to opt out of the browser doing implementation-dependent color space stuff, with colorSpaceConversion: "none", but it's very much a hassle to use in a case like this (and IMO createImageBitmap/ImageBitmap is a kind of bad API in a lot of ways...). I don't know how good the support is for this option is.

[jordyno]

thanks! will also try with different source files - with/without ICC profile, etc.