Browser dies on large raster files

[MetalMusings]

Open a large image, like a HD photo in jpg, png. Generate raster and keep track of the browser's memory usage - it will grow with GBs when the path rendering is created. For a large enough photo the browser will crash.

I assume that the problem is in the area of number of THREE lines, a 1000x1000 photo may need 1 million lines with bad luck, usually less, but a lot of lines anyway.

I have no working experience with THREE but I assume it would help to group the lines into lines with many vertices with different colors.

I intend to look into this the coming days.

[ghost]

Quickest way of ruling out whether it is three.js or not, is to comment out https://github.com/openhardwarecoza/LaserWeb2/blob/gh-pages/js/raster.js#L247

(then it wont generate a preview of the code file) - that should tell us whether its in the paperjs/raster work, or in the threejs work.

Three has a "mesh" function that you can feed a bunch of coordinates. However, the "lines" is a critical part of showing actual machine moves. If we are starting to show a preview of a "supposedly what the gcode looks like" instead "what the gcode looks like" - we might as well just NOT generate the gcode preview at all (and only show the user the image as loaded into the threejs texture)

Coming up with a optimisation for keeping the existing geometry, would be a better win. And would help @chilipeppr with their 5mb file size limit too (which also causes the browser to crash out) (in the chilipeppr project, theres even been talk of showing sections of the file at a time as the job progresses)

asking for help in the Three.js stack overflow http://stackoverflow.com/tags/three.js/info usually gets a clever and quick reply.

[ghost]

PS: only seeing about 101Mb of RAM on Chrome with a 1000x1000px jpg, the gradient is not the worst worst though, so feel free to forward sample files. I tried a few high resolution wallpapers lying around on my machine, and worst was around 120Mb (took about 6 minutes to raster + preview gcode)

[ghost]

sorry, clicked wrong button (:

[MetalMusings]

Yes, the movements will be optimized in that jpg, I guess it will only be like ten lines per row. Take random cat photo and test with.

I did, kind of, what you suggested, but I commented out the addSegment call in lib/gcode-viewer.js for G0 and G1. No lines generated. The G-code parser need like 100 MB for its work and that is all. I strongly suspect all the three.line. I will test by grouping the lines into three.linesegments and see if that has any effect.

[MetalMusings]

Grouping many vertices definitely has a positive impact. I think this is the way to go. Instead of 51 THREE.lines with two vertices each, grouping 51 vertices in one THREE.line reduced the memory consumption from 2.7 GB to 700 MB. Still a lot, but this can be extended further. For all I know the whole path can be one line. I read somewhere of 65536 but not sure if that was relevant for this.

The fix is easy in principle. Just delay the execution of this code in gcode-parser.js var line = new THREE.Line( geometry, material ); gcodeObj = line; }

gcodeObj.userData.p2 = p2; gcodeObj.userData.args = args; new3dObj.add(gcodeObj); until it is time to close the line so to say.

[ghost]

Awesome... Keep at it... @johnlauer ^ see above... Optimising gcodeviewer

On Mon, May 23, 2016 at 5:43 PM, HakanBastedt notifications@github.com wrote:

Grouping many vertices definitely has a positive impact. I think this is the way to go. Instead of 51 THREE.lines with two vertices each, grouping 51 vertices in one THREE.line reduced the memory consumption from 2.7 GB to 700 MB. Still a lot, but this can be extended further. For all I know the whole path can be one line. I read somewhere of 65536 but not sure if that was relevant for this.

The fix is easy in principle. Just delay the execution of this code in gcode-parser.js var line = new THREE.Line( geometry, material ); gcodeObj = line; }

gcodeObj.userData.p2 = p2; gcodeObj.userData.args = args; new3dObj.add(gcodeObj); until it is time to close the line so to say.

— You are receiving this because you modified the open/close state. Reply to this email directly or view it on GitHub https://github.com/openhardwarecoza/LaserWeb2/issues/53#issuecomment-221014087

[johnlauer]

Well, the problem there is the 3d viewer has one THREEE.Line per line of gcode so you can inspect it and see what’s going on. The collapse those into one long THREE.Line would remove that capability and throw off tons of stuff, i.e. there would not be one three object per gcode line. That is a major reworking of everything.

From: Peter van der Walt [mailto:notifications@github.com] Sent: Monday, May 23, 2016 9:15 AM To: openhardwarecoza/LaserWeb2 LaserWeb2@noreply.github.com Cc: John Lauer jlauer@zipwhip.com; Mention mention@noreply.github.com Subject: Re: [openhardwarecoza/LaserWeb2] Browser dies on large raster files (#53)

Awesome... Keep at it... @johnlauer ^ see above... Optimising gcodeviewer

On Mon, May 23, 2016 at 5:43 PM, HakanBastedt notifications@github.com<mailto:notifications@github.com> wrote:

Grouping many vertices definitely has a positive impact. I think this is the way to go. Instead of 51 THREE.lines with two vertices each, grouping 51 vertices in one THREE.line reduced the memory consumption from 2.7 GB to 700 MB. Still a lot, but this can be extended further. For all I know the whole path can be one line. I read somewhere of 65536 but not sure if that was relevant for this.

The fix is easy in principle. Just delay the execution of this code in gcode-parser.js var line = new THREE.Line( geometry, material ); gcodeObj = line; }

gcodeObj.userData.p2 = p2; gcodeObj.userData.args = args; new3dObj.add(gcodeObj); until it is time to close the line so to say.

— You are receiving this because you modified the open/close state. Reply to this email directly or view it on GitHub https://github.com/openhardwarecoza/LaserWeb2/issues/53#issuecomment-221014087

— You are receiving this because you were mentioned. Reply to this email directly or view it on GitHubhttps://github.com/openhardwarecoza/LaserWeb2/issues/53#issuecomment-221020668

[ghost]

Oh yeah. I forgot. You need the segments for the simulation! Darn, thought we could be helpful :( sorry. On 23 May 2016 18:18, "John Lauer" notifications@github.com wrote:

Well, the problem there is the 3d viewer has one THREEE.Line per line of gcode so you can inspect it and see what’s going on. The collapse those into one long THREE.Line would remove that capability and throw off tons of stuff, i.e. there would not be one three object per gcode line. That is a major reworking of everything.

From: Peter van der Walt [mailto:notifications@github.com] Sent: Monday, May 23, 2016 9:15 AM To: openhardwarecoza/LaserWeb2 LaserWeb2@noreply.github.com Cc: John Lauer jlauer@zipwhip.com; Mention mention@noreply.github.com Subject: Re: [openhardwarecoza/LaserWeb2] Browser dies on large raster files (#53)

Awesome... Keep at it... @johnlauer ^ see above... Optimising gcodeviewer

On Mon, May 23, 2016 at 5:43 PM, HakanBastedt <notifications@github.com mailto:notifications@github.com> wrote:

Grouping many vertices definitely has a positive impact. I think this is the way to go. Instead of 51 THREE.lines with two vertices each, grouping 51 vertices in one THREE.line reduced the memory consumption from 2.7 GB to 700 MB. Still a lot, but this can be extended further. For all I know the whole path can be one line. I read somewhere of 65536 but not sure if that was relevant for this.

The fix is easy in principle. Just delay the execution of this code in gcode-parser.js var line = new THREE.Line( geometry, material ); gcodeObj = line; }

gcodeObj.userData.p2 = p2; gcodeObj.userData.args = args; new3dObj.add(gcodeObj); until it is time to close the line so to say.

— You are receiving this because you modified the open/close state. Reply to this email directly or view it on GitHub < https://github.com/openhardwarecoza/LaserWeb2/issues/53#issuecomment-221014087>

— You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub< https://github.com/openhardwarecoza/LaserWeb2/issues/53#issuecomment-221020668>

— You are receiving this because you modified the open/close state. Reply to this email directly or view it on GitHub https://github.com/openhardwarecoza/LaserWeb2/issues/53#issuecomment-221021602

[MetalMusings]

Aah.

[ghost]

@HakanBastedt no problem for us, LaserWeb2 doesnt simulate the segments, just means sadly we cant upstream that as an improvement to Chilipeppr (which would have been nice). You can thus continue your investigation of it. On 23 May 2016 19:31, "HakanBastedt" notifications@github.com wrote:

Aah.

— You are receiving this because you modified the open/close state. Reply to this email directly or view it on GitHub https://github.com/openhardwarecoza/LaserWeb2/issues/53#issuecomment-221039283

[chilipeppr]

What you could do is create a fallback mode for the 3d viewer that uses the lightest techniques possible for rendering. That would include:

• Use BufferGeometry like made cuz it is 1/3 the memory. I already do this but not everywhere. It makes managing the 3d objects wayyyy harder, but worth the memory.
• Use three.line's in extended mode but turn off all simulation and object inspecting, i.e. dim out the buttons
• Don't try to attach gcode lines to each three object thus dim out any features that rely on that including if the gcode viewer tries to ask the 3d viewer to jump to a line just give up and throw up a flash message that you're in fallback mode
• Try to even use more aggressive techniques to push the three objects into the GPU. i don't have any ideas but i know that todd from jscut pushed his 3d objects into the gpu using c code in a filter in the browser

On Mon, May 23, 2016 at 10:31 AM, HakanBastedt notifications@github.com wrote:

Aah.

— You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub https://github.com/openhardwarecoza/LaserWeb2/issues/53#issuecomment-221039283

[ghost]

Mind if I tag @tbfleming - just always interesting to hear about the various tricks. @HaganBastedt already sped up our raster generation by a factor of 50 so, lets make sure he has some tools and knowledge of whats been tried and what worked/didnt - perhaps he comes up with fixes that benefits us all (:

On Mon, May 23, 2016 at 7:38 PM, John Lauer notifications@github.com wrote:

What you could do is create a fallback mode for the 3d viewer that uses the lightest techniques possible for rendering. That would include:

• Use BufferGeometry like made cuz it is 1/3 the memory. I already do this but not everywhere. It makes managing the 3d objects wayyyy harder, but worth the memory.
• Use three.line's in extended mode but turn off all simulation and object inspecting, i.e. dim out the buttons
• Don't try to attach gcode lines to each three object thus dim out any features that rely on that including if the gcode viewer tries to ask the 3d viewer to jump to a line just give up and throw up a flash message that you're in fallback mode
• Try to even use more aggressive techniques to push the three objects into the GPU. i don't have any ideas but i know that todd from jscut pushed his 3d objects into the gpu using c code in a filter in the browser

On Mon, May 23, 2016 at 10:31 AM, HakanBastedt notifications@github.com wrote:

Aah.

— You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub < https://github.com/openhardwarecoza/LaserWeb2/issues/53#issuecomment-221039283

— You are receiving this because you modified the open/close state. Reply to this email directly or view it on GitHub https://github.com/openhardwarecoza/LaserWeb2/issues/53#issuecomment-221040567

[tbfleming]

I agree with the following:

• Use BufferGeometry for the lines. The fewer BufferGeometry objects the better.
• Push as much as possible onto the GPU

Mapping lines in BufferGeometry to gcode:

• Use an array
• array[vertex index] = index into gcode

Ray casting:

• THREE.Raycaster returns vertex index
• Use the index to get the gcode position from the array

[MetalMusings]

I will look into the proposals. I am not ready for a massive rewrite though.

I took the mind set to not touch the three.lines and in that case the number of lines has to be reduced. I experimented with reducing number of grayscale levels of the lines and joining neighboring lines with the same grayscale levels much like it is done in laserraster.js when generating g-code. That is a possible way, even as low as ten levels of gray look ok- not brilliant, but ok. That is for the path on the screen. A 1600x1200 cat photo pumps up firefox with 800MB but it is workable. Without the grayscale level reduction firefox crashes after reaching 3GB. It is a delicate balance and for sure there will be cases that will not work so well. Here is the photo i tested with

http://4.bp.blogspot.com/-HHGgyi01vlI/Tn6juVxafkI/AAAAAAAACWs/IogIWAdhVrQ/s1600/994-animals_cat_cats_wallpaper.jpg

I also did a test to put the whole path with 1 million small lines into one three.line and a million vertices. That worked fine, memory increased by 500MB without any grayscale tweaking versus over 2GB with a million three.line. This is a more reliable method I think. I can do some experiments with the proposed methods and see what it can give.

[MetalMusings]

Looked around a bit more today. Here is an example with 10000 lines http://threejs.org/examples/#webgl_buffergeometry_lines Even modifying the example to 10 million lines and another coloring make the view appear after five seconds and memory consumption is 50 MB. This can be the performance target, Three.js does not have to be a bottleneck.

Parsing the G-code is quick, a few seconds, no worries there. But there are many items that are stored. addSegment() create a line object and keep it and the line is later recreated in drawobject() where also the the new line is kept. I have removed most graphics but still a lot of memory is used. BufferGeometry is used as far as I can tell, but not in the same way as is in the example. I see your points now @chilipeppr and @tbfleming and they make sense. @openhardwarecoza what are you going to do with the path more than just showing it on the screen? Are some of the userdata used somewhere else? From what I see so far the path-from-gcode generation can be much more efficient if the path is only used for visualization. If more is needed, then it depends of course.

[ghost]

Honestly from my point of view: nothing.

Coming from history -> The viewer was originally written by https://github.com/joewalnes/gcode-viewer , then John did a lot of work to add more moves, like G2, etc as well as a whole "simulation" on top... Couple months later, it was "simplified" by Andrew hodel in https://github.com/andrewhodel/grblweb/tree/master/i/gcode-viewer - whereafter I took it and changed the feel quite a lot, as well as added some fixes of my own (if I recall G3 wasnt working, I did a lot fo work to support 3D printing gcode, added dualstrusion viewer, etc, added the opacity based on the S value of the g1 lines, etc... will need to go look at https://github.com/openhardwarecoza/LaserWeb/commits/master/i/gcode-viewer/gcode-parser.js and https://github.com/openhardwarecoza/LaserWeb/commits/master/i/gcode-viewer/renderer.js and https://github.com/openhardwarecoza/LaserWeb/commits/master/i/gcode-viewer/ui.js for the full history

However, I didnt do any more work on removing features we didnt need.

For LaserWeb we can gladly remove those addsegmetns, etc - the goal with LaserWeb has always been a more stripped down viewer anyway (I can't run chilipeppr in the garage, the old GPU in there gets 2fps lol)

[MetalMusings]

Sounds good, then I will do some surgery in the code in the weekend. I'll keep all features ad try to speed it up and lower the memory footprint.

[MetalMusings]

Good progress here. A question, what is needed for the bounding box of the path? Where is that actually calculated?

[johnlauer]

There is a helper function that three.js gives you to calculate that. Just search for it in the chilipeppr 3d viewer to find example code. Not sure if peter has that in his version or not.

From: HakanBastedt [mailto:notifications@github.com] Sent: Friday, May 27, 2016 5:50 PM To: openhardwarecoza/LaserWeb2 LaserWeb2@noreply.github.com Cc: John Lauer jlauer@zipwhip.com; Mention mention@noreply.github.com Subject: Re: [openhardwarecoza/LaserWeb2] Browser dies on large raster files (#53)

Good progress here. A question, what is needed for the bounding box of the path? Where is that actually calculated?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/openhardwarecoza/LaserWeb2/issues/53#issuecomment-222280927, or mute the threadhttps://github.com/notifications/unsubscribe/AA8jlU_u6Alg5yPUVtRu5eI1Z_ja_Oo2ks5qF5FOgaJpZM4IkSgo.

[MetalMusings]

Thanks for that. I have made a test version at http://hakanbastedt.github.io/index2.html. The cat figure now needs approximately 200-300 MB. The path is not lightning fast to rotate and so, it is after all 2 million small lines. But it works also on really big images now. Small images are quite nice to work with.

[ghost]

I had a go on the test instance, looking good! Can't pickup any issues, feel free to send a pull request and I'll merge. Mind you, would you like Push access? I'll give you push access to the main repo

[MetalMusings]

I think you already gave me some access. Not comfortable working directly in this repo though. There is some tidying up I think, before submitting a PR.

One thing I see when testing is that sometimes the image that is read in is not displayed. Can not find fov it says. Don't think it is me doing that, but could be.

So what I have done is

• Not use addSegment for lines (only lines),
• Use THREE.Line-s with 10000 vertices each (arbitrary selected)
• Vertices are (p1 p2 p2 p3 p3 p4 ...) they are double but don't see how I can get rid of that. Using VertexColor, FaceColor could perhaps solve it, but not successful in googling an example.
• Only the graphics are stored in the "object" not anything else
• Colors are different now. Have to use varying intensity of the color (not the opacity as was done before) so a unique color can be set for every vertex.

The performance for displaying the path is now pretty much decided by the size of the G-code that need to be processed, the cat generates 30 MB G-code, and it takes time to process.

[ghost]

That FOV error comes up if it cant find the object just loaded, in a global var. On 28 May 2016 15:34, "HakanBastedt" notifications@github.com wrote:

I think you already gave me some access. Not comfortable working directly in this repo though. There is some tidying up I think, before submitting a PR.

One thing I see when testing is that sometimes the image that is read in is not displayed. Can not find fov it says. Don't think it is me doing that, but could be.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/openhardwarecoza/LaserWeb2/issues/53#issuecomment-222308861, or mute the thread https://github.com/notifications/unsubscribe/AHVr27eZ-WXwf_-807JHkS7bCClMjl8vks5qGERSgaJpZM4IkSgo .

[MetalMusings]

This is fixed now.

[ghost]

A massive thanks! @HakanBastedt , you are officially welcome to add your details to https://github.com/openhardwarecoza/LaserWeb2/wiki/Sponsors#paypal-links so the community can show you some gratitude!