search

smartin015 / continuousprint

Octoprint plugin to allow the creation of a print queue that prints, clears the bed and then prints again
40 stars 38 forks source link

Integration with inventory management #57

Open alanpuccinelli opened 2 years ago

alanpuccinelli commented 2 years ago

While one can always set an incredibly high number of copies might be nice to have a repeat till cancelled option. Maybe just setting a "-1" value for copies or a checkmark option?

smartin015 commented 2 years ago

Hey Alan - thanks for the feature request. Can you elaborate on when you'd want to print something indefinitely? I'm hesitant to add this as a feature as it has the potential to be quite wasteful.

I also suspect an "infinite loop" setting might be a coarse way to specify something that could be better done with a specific signal (e.g. "print enough plastic beads to fill a WxHxD container" may be a better intent than "print plastic beads until the heat death of the universe").

alanpuccinelli commented 2 years ago

Probably just what I'm most used to. We run thousands of parts for weeks at a time on many of our farm printers and I've just been using M808 for that up until now. Its obviously a nice to have but not critical. Frankly I don't see how it would be any more wasteful than just setting a really high number of copies.

smartin015 commented 2 years ago

Ah, gotcha. What are some reasons for stopping your print run, after you've started? Is timing a factor, or is it more of a general sense of having enough inventory?

(I'm also kind of curious - what kind of things are you printing thousands of?)

alanpuccinelli commented 2 years ago

It fully depends on product demand at any given time. We're still somewhat unsophisticated on inventory control so yes a lot of it can be a guess or a higher priority job comes in and bumps it.

Alan Puccinelli Founder & Owner Repkord LLC

https://www.repkord.com

On Fri, Apr 29, 2022 at 2:05 PM Scott Martin @.***> wrote:

Ah, gotcha. What are some reasons for stopping your print run? Is timing a factor, or is it more of a guess at having enough inventory?

— Reply to this email directly, view it on GitHub https://github.com/smartin015/continuousprint/issues/57#issuecomment-1113730665, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAFI4FXWN6A75JURF7ESJQLVHRFIJANCNFSM5UTLKCBQ . You are receiving this because you authored the thread.Message ID: @.***>

smartin015 commented 2 years ago

Ah, I was wondering why your name sounded familiar - I was just watching the 3DPN youtube demo of your CR30 upgrades a couple days ago :) cool stuff.

Now I'm imagining an integration with some (pre-existing) inventory management service that allows you to issue commands like "print until I have X amount in stock", with additional logic that adjusts the print priority based on whatever's currently the least in-stock. WDYT?

I have some work ongoing that allows for multiple queues with different queue behaviors, which would open up the ability to have a "manual" linear/job-order queue and a separate "smart" queue such as above, where the printer can pull work from any of them based on some predetermined rules (e.g. "do stock replenishment, but preempt that with anything in the manual queue"). Also planning support for multiple printers managing these multiple queues (#35) in a many-to-many sort of way. This could be a potential extension/adaptation of that work.

alanpuccinelli commented 2 years ago

WDIT? I think that sounds incredible and incredibly useful to us here. Tell me how I can help.

On Apr 29, 2022, at 2:20 PM, Scott Martin @.***> wrote:

 Ah, I was wondering why your name sounded familiar - I was just watching the 3DPN youtube demo of your CR30 upgrades a couple days ago :) cool stuff.

Now I'm imagining an integration with some (pre-existing) inventory management service that allows you to issue commands like "print until I have X amount in stock", with additional logic that adjusts the print priority based on whatever's currently the least in-stock. WDYT?

I have some work ongoing that allows for multiple queues with different queue behaviors, which would open up the ability to have a "manual" linear/job-order queue and a separate "smart" queue such as above, where the printer can pull work from any of them based on some predetermined rules (e.g. "do stock replenishment, but preempt that with anything in the manual queue"). Also planning support for multiple printers managing these multiple queues (#35) in a many-to-many sort of way. This could be a potential extension/adaptation of that work.

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.

smartin015 commented 2 years ago

The first and best way for you to help is with the specifics - it appears you're using Shopify for the e-commerce part of your site, which I haven't used in the past. Do they have built-in inventory management? Is it sufficient, or do you think a third-party IMS would work better? (IMO it'd be great to have an option not tied to a specific sales service, but convenience trumps that)

The overall design can be somewhat generic: a way to match SKUs to CPQ jobs, plus an extensible way to sync inventory state via polling/pushing with the IMS, in addition to the queue logic. But it's helpful to have an target integration in mind.

It's probably going to be another month or so before I've worked through what's already planned for CPQ and start working on this. If you're interested in faster delivery or in helping further, we can chat about contribution / incentives.

alanpuccinelli commented 2 years ago

We do use shopify inventory management is minimal. I believe there is an API but I haven't tried it out yet and frankly most of our stuff is parts for larger kits so we wouldn't monitor specific stock parts for direct sales on the eshop this way. Feel free to reach out to me via email to discuss further when you're ready.

Alan

Alan Puccinelli Founder & Owner Repkord LLC

https://www.repkord.com

On Sat, Apr 30, 2022 at 9:58 AM Scott Martin @.***> wrote:

The first and best way for you to help is with the specifics - it appears you're using Shopify for the e-commerce part of your site, which I haven't used in the past. Do they have built-in inventory management? Is it sufficient, or do you think a third-party IMS would work better?

The overall design can be somewhat generic: a way to match SKUs to CPQ jobs, plus an extensible way to sync inventory state via polling/pushing with the IMS, in addition to the queue logic. But it's helpful to have an target integration in mind.

It's probably going to be another month or so before I've worked through what's already planned for CPQ and start working on this - if you're interested in faster delivery or in helping further, we can chat about contribution / incentives.

— Reply to this email directly, view it on GitHub https://github.com/smartin015/continuousprint/issues/57#issuecomment-1114019601, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAFI4FUJROJWCGBGMC425S3VHVREJANCNFSM5UTLKCBQ . You are receiving this because you authored the thread.Message ID: @.***>

alanpuccinelli commented 2 years ago

Hey Scott, question for you. How hard would it be to have an option for continuous print to output the entirety of the queue as gcode at any given point. Reason being if I wanted to create a loop sequence but don't have enough pis for all my machines at the moment it would be super handy to just copy that gcode to an SD card so "dumb" machines can match up what CP is doing. Good idea? Want me to throw it in a feature request ticket?

Pooch

Alan Puccinelli Founder & Owner Repkord LLC

https://www.repkord.com

On Sat, Apr 30, 2022 at 9:58 AM Scott Martin @.***> wrote:

The first and best way for you to help is with the specifics - it appears you're using Shopify for the e-commerce part of your site, which I haven't used in the past. Do they have built-in inventory management? Is it sufficient, or do you think a third-party IMS would work better?

The overall design can be somewhat generic: a way to match SKUs to CPQ jobs, plus an extensible way to sync inventory state via polling/pushing with the IMS, in addition to the queue logic. But it's helpful to have an target integration in mind.

It's probably going to be another month or so before I've worked through what's already planned for CPQ and start working on this - if you're interested in faster delivery or in helping further, we can chat about contribution / incentives.

— Reply to this email directly, view it on GitHub https://github.com/smartin015/continuousprint/issues/57#issuecomment-1114019601, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAFI4FUJROJWCGBGMC425S3VHVREJANCNFSM5UTLKCBQ . You are receiving this because you authored the thread.Message ID: @.***>

smartin015 commented 2 years ago

Hey, sorry for not handling your question earlier! I somehow lost it in the flow of things.

That does sound interesting, sort of a legacy/compat option. Please do file a feature request and we can chat about what makes sense to implement there :)

smartin015 commented 2 years ago

Hey @alanpuccinelli - I haven't forgotten about our conversation here. I sent a request via your contact form - we can follow up using the email I submitted there.

smartin015 commented 1 year ago

High-level feedback

Ideas:

nickgabbard commented 1 year ago

Weighing in as requested.

We have 18 printers running 2 shifts a day. We use shopify for sales and orders. Shopify inventory management kind of sucks, so we use an in-house PostgreSQL database to handle inventory. Our tech stack also has an automation platform called N8N running in the shop. It's similar to a self-hosted IFTTT that does webhooks and http requests. It's currently used for file management when creating new listings, but we are trying to come up with a way to integrate that into the printing process as well.

For clarity in the rest of the way I'm going to explain things, here's some more background. We refer to a sales unit as an "object." If it's for sale on the website, it's an object. There are also packages of multiple objects for sale. We treat that as a completely new object. Objects are made up of parts. An object can have 1 or more parts. Each part is 1 print file (either a .gcode if we print it with an FDM printer, or a .stl file if we print it with a resin printer).

We don't track inventory on objects (which is why shopify's inventory management doesn't work for us). We do track inventory on parts.

Our catalog is huge. We usually only have less than 10% of parts in stock at any given time.

So, we are essentially a print-on-demand shop, but we need a check step to see if we happen to have one laying around first.

Current thought process on a workflow is:

  1. Get notified of order from shopify trigger to n8n.
  2. Based on "skus" transmitted from shopify (these match up with objects) determine what "'parts" need to be shipped. (probably can be done with Postgres trigger by creating and writing to an "orders" table or something)
  3. Based on the sum of current parts in inventory AND the sum of parts in currently unfulfilled orders AND the sum of parts that are currently in the "to be printed list" (not the continuous print queue, btw) , generate a list of parts that need to be printed and add them to a "to be printed list" (some postgres table)
  4. The "to be printed list" should have a flag that indicates if it has been added to a CP queue or not, and some sort of indicator that the print is actually complete/good.
  5. Iterate through the "to be printed list" and identify items that haven't been sent to CP and use n8n and the CP API to add them to a print queue. It would be very handy at this point to be able to move them directly to a network queue, assuming that the automatic printer profile detection is working properly. Additionally, since we are shop that only runs one specific model of FDM printer, a global configuration option in CP that defaults all jobs to a specific printer profile would be awesome.
  6. and this is where my train of thought breaks down......

At this point, physical objects start printing. We would need to come up with a way to link those physical objects back to items on the "to be printed list" to indicate that they are finished, but we have to have a human QC point in there. Everything that comes out of the print room should be added to inventory, so orders would always be filled from inventory, instead of straight from production.

Honestly, I don't know if any of this adds to the conversation or just muddies the water.

smartin015 commented 1 year ago

Thanks, this is exactly what I was hoping for! Also hadn't heard of N8N before - that's a great alternative to IFTTT, Zapier etc.

Objects are made up of parts. An object can have 1 or more parts. Each part is 1 print file... We don't track inventory on objects (which is why shopify's inventory management doesn't work for us). We do track inventory on parts.

This lines up perfectly with what I've heard about other inventory systems - that extra indirection between sale items and parts means a custom solution is needed every time. It'd be great if there was a "common" custom solution, which is where I get interested in the details :)

Current thought process on a workflow is:

  1. Get notified of order from shopify trigger to n8n.
  2. Based on "skus" transmitted from shopify (these match up with objects) determine what "'parts" need to be shipped. (probably can be done with Postgres trigger by creating and writing to an "orders" table or something)
  3. Based on the sum of current parts in inventory AND the sum of parts in currently unfulfilled orders AND the sum of parts that are currently in the "to be printed list" (not the continuous print queue, btw) , generate a list of parts that need to be printed and add them to a "to be printed list" (some postgres table)
  4. The "to be printed list" should have a flag that indicates if it has been added to a CP queue or not, and some sort of indicator that the print is actually complete/good.

Although I'd initially thought about hosting inventory data in CPQ, I like the idea of an "authoritative" external DB that feeds CPQ. That means fewer integration points needed on the plugin side and less "util columns" for holding opaque user data.

  1. Iterate through the "to be printed list" and identify items that haven't been sent to CP and use n8n and the CP API to add them to a print queue. It would be very handy at this point to be able to move them directly to a network queue, assuming that the automatic printer profile detection is working properly.

215 sounds like what you're looking for. I'd definitely want the same validation checks as with moving from local to network queues. Which slicer are you using? At this point, Kiri:Moto, Prusa slicer, and Simplify3D are supported. If your gcode files already have printer metadata, a new slicer can pretty easily be added in.

On the flip side, you could always pass STL files around and slice them on the fly. That doesn't require profile inference, but does require a slicing service to be set up. Maybe not worth it if the files being printed don't change regularly, and there's no caching as of yet (see #225).

Additionally, since we are shop that only runs one specific model of FDM printer, a global configuration option in CP that defaults all jobs to a specific printer profile would be awesome.

I've heard this request before, but I'm afraid to "defeat" the gcode validation step like that if it's not actively causing any problems. That being said... I don't think it's been done before, but /set/add should allow passing the printer profile in the request body. If you want to assert your files are for a certain printer, it seems fine to do it that way.

We would need to come up with a way to link those physical objects back to items on the "to be printed list" to indicate that they are finished, but we have to have a human QC point in there.

Does https://github.com/jneilliii/OctoPrint-BedReady/issues/16 seem useful to you at all? Not saying it would eliminate all QC, but how much work do you think could be eliminated if you could auto-reject based on image differences?

Re: linking back to the "to be printed list", I'm imagining a couple options:

  1. Create a custom at-command that allows making a web request to a given URL with info about the currently printed file, state of CPQ etc. This would be run at script execution time, so response data couldn't easily be used.
  2. Add a function to the preprocessor's symbol table that permits sending a web request. Benefit over (1) is that the response could be used to affect the next executed GCODE (e.g. send "it's done" to a server, receive "push it this way or that way off the bed", and move accordingly). This could risk a jailbreak from ASTEval if an exploit is discovered, but IMO the risk is pretty low (would have to be an exploit to do with assigning a string to a variable, and the most obvious one - buffer overflow - can be pretty easily mitigated).
  3. Formal integration of "send a web request on completion" to CPQ's printing state machine. This gives the most opportunity for changes in CPQ behavior, but locks changes into the release cycle and complicates the system overall.
  4. External service polls for changes in the queue and diffs against prior state. Requires no changes to CPQ, but does seem kind of wasteful and overly custom.

WDYT?

nickgabbard commented 1 year ago

Objects are made up of parts. An object can have 1 or more parts. Each part is 1 print file... We don't track inventory on objects (which is why shopify's inventory management doesn't work for us). We do track inventory on parts.

This lines up perfectly with what I've heard about other inventory systems - that extra indirection between sale items and parts means a custom solution is needed every time. It'd be great if there was a "common" custom solution, which is where I get interested in the details :)

If there is a standard I've not found it.

Although I'd initially thought about hosting inventory data in CPQ, I like the idea of an "authoritative" external DB that feeds CPQ. That means fewer integration points needed on the plugin side and less "util columns" for holding opaque user data.

Having it in CPQ I feel like that could get messy really fast, and would make it harder to integrate into anything else.

  1. Iterate through the "to be printed list" and identify items that haven't been sent to CP and use n8n and the CP API to add them to a print queue. It would be very handy at this point to be able to move them directly to a network queue, assuming that the automatic printer profile detection is working properly.

215 sounds like what you're looking for. I'd definitely want the same validation checks as with moving from local to network queues. Which slicer are you using? At this point, Kiri:Moto, Prusa slicer, and Simplify3D are supported. If your gcode files already have printer metadata, a new slicer can pretty easily be added in.

We use Prusa Slicer, but automatic detection doesn't work for a majority of our older gcode files. I've not really dug into why yet, because it looks from the code like it should.

On the flip side, you could always pass STL files around and slice them on the fly. That doesn't require profile inference, but does require a slicing service to be set up. Maybe not worth it if the files being printed don't change regularly, and there's no caching as of yet (see #225). The file being printed does not change once we get a good working print. A lot of these things are pretty fiddly and need custom supports. There's also a few thousand different gcodes in the library this point, so that's also another reason I don't want to re-slice them all to attempt to get profile inference to work.

Additionally, since we are shop that only runs one specific model of FDM printer, a global configuration option in CP that defaults all jobs to a specific printer profile would be awesome.

I've heard this request before, but I'm afraid to "defeat" the gcode validation step like that if it's not actively causing any problems. That being said... I don't think it's been done before, but /set/add should allow passing the printer profile in the request body. If you want to assert your files are for a certain printer, it seems fine to do it that way.

I full intend to try passing it thru /set/add later today.

We would need to come up with a way to link those physical objects back to items on the "to be printed list" to indicate that they are finished, but we have to have a human QC point in there.

Does jneilliii/OctoPrint-BedReady#16 seem useful to you at all? Not saying it would eliminate all QC, but how much work do you think could be eliminated if you could auto-reject based on image differences?

It might, but we don't run webcams. Actually, we don't run individual hardware for each printer. All 18 printers are connected via USB to an HP server and we run 19 different Octoprint docker containers. We leave one of them not connected to a printer and use it sort of like a master control to add new sets/files/etc. Adding USB webcams to that mix would tank the USB throughput and cause a host of other issues. Maybe when rpis aren't selling at black market prices.

Re: linking back to the "to be printed list", I'm imagining a couple options:

  1. Create a custom at-command that allows making a web request to a given URL with info about the currently printed file, state of CPQ etc. This would be run at script execution time, so response data couldn't easily be used.
  2. Add a function to the preprocessor's symbol table that permits sending a web request. Benefit over (1) is that the response could be used to affect the next executed GCODE (e.g. send "it's done" to a server, receive "push it this way or that way off the bed", and move accordingly). This could risk a jailbreak from ASTEval if an exploit is discovered, but IMO the risk is pretty low (would have to be an exploit to do with assigning a string to a variable, and the most obvious one - buffer overflow - can be pretty easily mitigated).
  3. Formal integration of "send a web request on completion" to CPQ's printing state machine. This gives the most opportunity for changes in CPQ behavior, but locks changes into the release cycle and complicates the system overall.
  4. External service polls for changes in the queue and diffs against prior state. Requires no changes to CPQ, but does seem kind of wasteful and overly custom.

WDYT?

Option 1 or 2 seems like the most immediately useful, however N8N is probably capable of doing Option 4 now with no changes.

smartin015 commented 1 year ago

Having it in CPQ I feel like that could get messy really fast, and would make it harder to integrate into anything else.

Agreed.

We use Prusa Slicer, but automatic detection doesn't work for a majority of our older gcode files. I've not really dug into why yet, because it looks from the code like it should.

If you want to toss an example file my way, I could take a look and see where the soft matching breaks down :)

The file being printed does not change once we get a good working print. A lot of these things are pretty fiddly and need custom supports. There's also a few thousand different gcodes in the library this point, so that's also another reason I don't want to re-slice them all to attempt to get profile inference to work.

Makes sense.

It might, but we don't run webcams. Actually, we don't run individual hardware for each printer. All 18 printers are connected via USB to an HP server and we run 19 different Octoprint docker containers. We leave one of them not connected to a printer and use it sort of like a master control to add new sets/files/etc. Adding USB webcams to that mix would tank the USB throughput and cause a host of other issues. Maybe when rpis aren't selling at black market prices.

Yeah, rpi's are quite expensive right now. I started thinking about a lower-cost option involving cheaper embedded devices. That's likely to be several months away.

Do you have any special hardware on your HP server to handle 18 independent serial connections? I had another user on #213 experiencing bandwidth issues, and I wonder if there's enough info around that it'd be worth writing a guide in the docs for a single-host, many-printer hardware setup.

Option 1 or 2 seems like the most immediately useful, however N8N is probably capable of doing Option 4 now with no changes.

Agreed - between the two, I might tackle option 2 in the future as it has the most potential. If you find a good solution with option 4 in the meantime, I'd be interested in documenting it so others have a starting point for their integrations.

smartin015 commented 1 year ago

I've split off the computer vision feature discussion to #229. May also work on option 2 as part of it.