Api for gcode creation

[rlarge]

Anyone interested in api for milling and laser options. Like feed a json with properties of pocket and a svg file to http://localhost:8000/mill/pocket and it would create then respond with g code so you could programmatically create different shapes or text from a python script or node. Right now i'm creating gift tags for x-mas and it would have been nice to just write a script to automate it. Don't really know anything about react but if people are interested i could learn :)

[jorgerobles]

Welcome aboard!

It's a nice idea, indeed, there's a feature wish floating up there to include a gallery of shapes into LW. Maybe we could mix the idea.

There are some considerations about the API. Currently LW generates the gcode from the client frontend (pure js), so there's no real endpoint to be attached to. To do so, we can do several things (ideas welcomed)

1. Moving all the gcode generation to the server: Unlikely - Chrome frontend offers us some features that could not be done easily on Node (as far as I know -- @tbfleming, @cprezzi )

2. Implementing a client API, via URL hashbangs: Possible but limited - I don't know current limits of URL length, was 1024 a while ago. We have to check if that's enough to pass the parameters. Surely insufficient to pass the vector data.

3. Inverse service API: As LW needs vector data (from SVG, DXF or native JSON data) and operation parameters, LW could use your service as a plugin, where LW can ask for the data from your parameters scripts, a la Tinkercad/Customizer generators. LW could expose a gallery section using the API, asking for a generator passing parameters, and then receive the vector data (not gcode) and optional operations data if needed.

What do you think?

[tbfleming]

Maybe a new server that includes the CAM code? The main client wouldn't benefit; it'd just be for new scripts and apps. The CAM code doesn't depend on React. You'd probably have to bypass the webworker code.

[tbfleming]

2: Probably have to keep the URL simple and use http post.

I forgot about SVG conversion. The SVG parser relies on the browser to do much of the work. The CAM code never sees SVG or DXF.

[jorgerobles]

Yep.. including cam on server could benefit from developing c code for node, and so on. Also getting post vars would be handled by server, as usual.

[rlarge]

Hello everyone nice to meet you guys... I thought about it a bit more and I thank a plugin system might be more appropriate. Instead of creating another code base just allow contributors design plugins without the need for pull requests. @jorgerobles How difficult would this structure be to implement. https://www.nylas.com/blog/react-plugins/ Given my 1 day knowledge of react i am fairly certain i could create a plugin for shapes and text rather quickly.

[tbfleming]

We only use react client-side. It wouldn't help make a gcode-generation API.

[rlarge]

Ya i get that..... i recently wrote a js script that imported a svg placed it on canvas added text from a list and exported the svg (fabricjs). Now imagine a addon or plugins system that could use the available commands like engrave without changing the backend. I could make a simple shapes addon in about 30 min or less. But you wouldn't want addons crashing the whole program... hence the structure question..

[rlarge]

I've changed my mind about an api lol @tbfleming

[jorgerobles]

@rlarge LW is built upon an architecture similar to https://www.nylas.com/blog/react-plugins/, but done with React-Redux. No server side.

The way I see, should exists 2 sides:

1. LW side, a react component (gallery) that previews, lists and allows to ask for the SVG once the parameters are filled. An specific generator will be shown on iframe with CORS authentication allowing LW to interact with it.
2. (your) API/JS side that could standalone generate the SVG. LW will be allowed to get the SVG document once finished editing.

How it works:

LW should implement a way to store your SVG and parameters URL in LW native format, allowing an UI button to popup and recall your generator (IE: user makes a 5 point star, places on LW canvas. Later he wishes a 6 point star, so clicks the smart shape button, the popup appear with the shape and parameters, changes to six. Popup is closed, and the shape is changed on LW.

This architecture is free enough to allow you to develop the generators on your own way, without worry or interference of LW development and viceversa.

Thoughts?

[rlarge]

Ya i believe were on the same page. I come from an AngularJS world so i'm just learning react right now lol. The way i thought about it was to have a folder /addons out side the scope of LW. And then have LW dynamically inject links of the addons creating a dropdown menu with addons. When addon was selected it could be opened in a iframe popup as long as it would have access to current selection and object properties. Like i import a svg tag design of a simple square with a milled design. I apply a cutout for the outside cut of 3mill and engrave for the design. I then proceed to click on the addon "Add multi-text" a popup appears that has access to the selected objects and paths with cut properties. In the popup i have a text area that i could type names separated by commas with a canvas that shows the position of the text adjustable location wise and other things. Also at this point i would add the milling options for the dynamic text. Now you would click done and my addon with clone the original apply text with milling options and feed each one back to LW with all the properties. From there you generate the g-code the normal way.

Also it could be one way with a simple undo function on LW. Kinda like gimp does with its addon/plugins.

same page, what you thank? @jorgerobles

[tbfleming]

Something to watch out for: we don't hold on to any SVGs after importing. We only hold on to our json data. Use Save Workspace to get an idea of what it looks like.

[jorgerobles]

Fine :)

@rlarge Some questions/caveats:

1. Seeing your described workflow, should LW pass the current selection vector data to the addon? if so SVG is not an interchange format, but the native one (@tbfleming could help with current vector format explanation) and so the access to operations (or the whole redux store).

2. As you explains addons folder, I understand the addons will not be an online service, but a compile-time repository (Possibly a bunch of webworkers). We have done this with materials and machines, but I thought It was better an online service (just for the sake of independent development, you know any change in addons will force the LW recompilation).

A possible solution is to that files to be served from.. well, lw.comm-server or another parallel server.

[jorgerobles]

@tbfleming cross-post, sorry!

[tbfleming]

It'd be nice to do something like this:

• User enters URLs of plugins they want
• We embed the plugins using iframes with locked-down permissions. We have to lock it down since electron gives the UI direct access to the filesystem (we just haven't been using those functions so far)
• We SendMessage() data back and forth, also for security. Since it's in an iframe, the plugin doesn't have to be react.

This would allow anyone to host their own plugin on gh-pages or other hosting site.

The SVG machine controller (unfinished) does this.

[tbfleming]

The hard part: we'd have to aggressively check data coming back from the plugin. Bad data can create a security issue. e.g. a malformed data uri in an image, or a file uri instead of the data uri, or a http uri, or ...

[rlarge]

That's exactly what i was thinking. But also be able to run local versions of addons with less restrictions (with warnings of course) so you could save locally with addons (with time). It would open a whole ecosystem of options. While also not complicating LW and improving workflow. Imagine how easy it would be to create a nesting addon with SVGnest.

[jorgerobles]

I propose a fix the addons to be hosted on a LW repo for the moment. @rlarge will get access and publish freely, of course. Any other candidate should do a PR to publish the addons. It's a little price for security. I'm fine with local addons AFTER we include a discharge of responsability, of course :)

@rlarge, you have touched my heart with the nesting feature. If you can do that the first addon, you will get my soul :D

[rlarge]

Ya security would be something that would be needed to look at. Addons could be only be served locally to start. How hard is it start up a python simple httpserver and only allow local host.

[rlarge]

@jorgerobles i'll see what i can do, gonna have some time over the holidays and now that i don't need to totally learn react.

[rlarge]

@tbfleming I'm gonna look at the SVG machine controller

[rlarge]

@tbfleming Where is this famed unfinished SVG machine controller lol

[tbfleming]

https://github.com/LaserWeb/lw.controller

[rlarge]

Ok so after looking through some stuff i got a couple questions. The raw paths from the saved work space what do you use to convert them back for the fronted. @tbfleming I would like to use the same code. Also is there anyway to get the work space size added to the file when saved. That would help with positioning stuff correctly with the packing problem per sheet. As of right now SVGnest just ignores the ones that don't fit. I got ideas but the size is hard coded now. Also is there a better forum that you guys use? Thanks

[tbfleming]

https://github.com/LaserWeb/LaserWeb4/blob/dev-es6/src/lib/mesh.js has a bunch of conversion functions.

• triangulateRawPaths converts to triangles; the webgl display uses this to fill the shapes.
• The SVG-related functions only handle part of the SVG conversion; it's split between mesh.js, lw-svg.parser, and reducers/document.js
• Most of the other conversions are used by the CAM.

You're at the right place; we do most development discussion in issues. End user support (for people who aren't writing code) is at https://plus.google.com/communities/115879488566665599508 .

[tbfleming]

We could send the settings to plugins. To see what they look like: settings -> tools -> settings -> save.

[rlarge]

Okay last night i tried to put together a little poof of concept and i ran into a couple of snags...lol

1. One was how little i know about cam data and the way the data is saved. After i did a bit of research i found the whole cam/cad ecosystem has no idea also. Seems there are a few ways to save and all come with incomplete documentation lots of exceptions. Am i right about that?

2. I looked a mesh.js and see where the svg is converted to points. I lightly grasp the idea but have no idea how to reverse the process. @tbfleming If someone smarter then me would kindly provide a small library detached from LW that could convert a svg to rawData like LW already does and then rawData back to svg "D" format. Pretty Please lol

3. I don't really have a background in this so any direction or correction in my understanding of things please let me know. Also i have a 4ft by 3ft mill for wood working if you would need any testing help.

Thanks, Roger

[jorgerobles]

Hi Roger:

1. All the data is stored on a Redux store: You can install the react extension to chrome and will see the store dictionary. There, inside documents, you will see rawpaths that holds the vector data. And yes.. LW is a poorly documented b*tch. The CAM parameters are store in Operations, holding a reference of the paths.

2. The library that converts SVG to rawpaths is https://github.com/LaserWeb/lw.svg-parser, but currently it's results has been postprocessed, partly for improvements and partly as the library is unmaintained by it's original creator. Rawpaths are used by a Webgl engine so no reversal method has been made.

3. Ok, build a bartop arcade and send me to Spain. Will not help on development but will be fun 🤣 (just joking)

[tbfleming]

2. File import starts here. This is in response to a file input's OnChange handler.

https://github.com/LaserWeb/LaserWeb4/blob/36bb7a3d0415cd4da0c5cc09fab3e9cf69c8b88c/src/components/cam.js#L240

[tbfleming]

2. I forgot to answer about the reverse direction. We have no reverse direction; we never convert to SVG.

[rlarge]

@tbfleming My bad I've only used svg with LW i didn't even know you could import other types. Now i know why you said never convert it instead of save it. Sorry about that. So rawData (2d or 3d mesh) from the work space is used by the cam and during creation of the g code.

I propose that plugins be separated by import "TYPE". Creating a plugin that could nest every type of input is out of my league. But lets say a plug-in for g code that edits the corners for a drag knife would be easier.

So here is what i imagine: Plugins --> G-code Svg ------> Nesting Have the ones that are not currently available grayed out. Click on nesting temp saves the work space and opens up a iframe then uses web sockets to connect to a remote plugin. Sends the work space json or maybe right now just send all the data that is available. It does its thing and sends the workspace back to LW. LW then checks the data for changes to rawData. If changed or not present LW creates it and updates the work space. If error happens undo would just change the work space to previous state.

Issues:

1. You could get malformed Json... so you would have to check structure. Bad revert workspace
2. You could get wrong settings for operations.
3. And all the other problems with untrusted Json, JavaScript Injection, Cross Site Scripting and more. Not really a problem there are librarys that will clean json.

Assuming LW would save the svg in the workspace a simple plugin design would be this. Plugin Name: Fill Cut Area Plugin Type: SVG

Plugin would take workspace that contains a 10mm square that has a ouside mill operation clone the svg to fill area giving each 10mm square a new uid and adding it to the operations of its parent. Return the workspace back to LW it creates rawData for each new svg from the json and applies the changes to the view.

Ideas?

@jorgerobles Building one is on my list of things to do... just got a 32 inch flat screen for free a couple weeks ago. Have a pie sitting next to it...lol

[tbfleming]

I don't want to complicate the workspace by adding formats to it, since it impacts the entire app. If we add plugins before CAM, then they'll have to deal with rawpaths.

SVGnest doesn't directly use SVG; it converts it to and from its own format. Maybe you could write a converter between rawpaths and SVGnest's format.

[rlarge]

It would only get save when you insert a svg into the workspace. You wouldn't covert anything else into a svg. You already save the Name, type, number of paths, parents and if its a child of another path. You literally save everything besides "D" in a svg lol

Converting to svgnest format negates that whole point of use. You want a simple shape. As of right now you would have to go to inkscape or gimp create a circle export it and import to LW. Then select it create a operation. click gcode and cut. With just saving the data you could go to plugins click on simple shapes click on a circle change size with slider. Click export on the bottom your back at LW and LW checks the returned workplace json that rawpath is present if not creates it from the json svg data and then updates the dom.

None of that changes anything that isn't already happening when you load a work space.

[tbfleming]

It changes a lot.

• There has to be some visual indicator on the document view that there's an SVG file tied to onscreen objects, since it affects what plugins a user can use with it.
• Editing will cause the SVG file to get out of sync.
  • Editing = moving, scaling, rotating, and deleting objects that were in the file.
  • Option 1: delete the SVG file when the user makes edits. They'll get annoyed.
  • Option 2: leave the SVG file as is. Users will get confused when the plugin's output isn't related to what they saw onscreen before running it.
  • Option 3: edit the SVG file. Nope. Editing redundant data invites massive bugs.
• Workspace files will be bigger, just to support something that not all users need.

[rlarge]

Ya out of sync would be an big issue. My workflow i create all the paths in gimp and just use LW for g-code. I don't think i moved anything yet lol. Gotta thank about other people sometimes.... I'm reinstalling my dev environment now and if i can get ubuntu 16.04lts and nvidia drivers to ever play nice i go through the conversion process to rawData and webgl. This might take a bit..lol Thanks for the help

[jorgerobles]

I dug svgnest a couple times, and it uses a same/very similar format, clipperjs. Its differences are basically the svg parsers.

El 20 dic. 2017 5:05 a. m., "rlarge" notifications@github.com escribió:

Ya out of sync would be an big issue. My workflow i create all the paths in gimp and just use LW for g-code. I don't think i moved anything yet lol. Gotta thank about other people sometimes.... I'm reinstalling my dev environment now and if i can get ubuntu 16.04lts and nvidia drivers to ever play nice i go through the conversion process to rawData and webgl. This might take a bit..lol Thanks for the help

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/LaserWeb/LaserWeb4/issues/443#issuecomment-352959269, or mute the thread https://github.com/notifications/unsubscribe-auth/ABoIYMZB6KofkpOGsckda3HUneW5hrQqks5tCId_gaJpZM4REgeP .

[msurguy]

Hello! I am really impressed with the speed of SVG -> GCode conversion in LW. I think it's state of the art comparing to many other things I've tried so far, including Glowforge's conversion algorithms used by thousands of people.

I am very interested in taking this jewel of code (SVG -> GCODE parser) and making it into a modular system that could be used with other SVG generators on the web.

Can someone describe to me in a few words (or a hand drawn chart) how the system works together after you drop the svg into the workspace and move it to desired position?

@tbfleming @cprezzi I am pretty proficient in taking stuff that usually runs in the browser and make it work on a server. Could you please let me know what specific limitations are there with making the conversion work on a server? There's a lot of recent advances in running Chrome in a headless mode, for example via https://github.com/GoogleChrome/puppeteer that could run WebGL in headless mode on MacOS and possibly on Linux.

Even if running it completely headless is not possible, I would like to come up with a way to have the SVG -> GCode converter work as a self-contained module of some sort and any info on how to do that would be beneficial to move this forward.

Thanks for your hard work on this!

[tbfleming]

@msurguy We use DOM SVG objects to parse the file. We then convert it to our own JSON format, which lives in a Redux store. When the user changes something, UI events dispatch actions to the store. All transforms, except image manipulation, are pure computation that doesn't involve any DOM objects or WebGL. DOM and WebGL just provide the UI. Image manipulation: we do use DOM objects to help with this part.

Be careful putting any of our code on a server; it's Affero GPL, which places requirements on the server code.

[msurguy]

@tbfleming thank you for explaining that info in detail!

@jorgerobles @tbfleming the SVG -> GCODE converter in LaserWeb is basically state of the art and very rare solution that can work well in a browser. I was wondering whether there may be an option to open source or sell just the SVG->GCODE converter part itself as it can push a whole new range of possibilities forward. Please let me know what the possibilities are to be able to use this converter in another open source project or a part of a project.