search

inventree / InvenTree

Open Source Inventory Management System
https://docs.inventree.org
MIT License
4.22k stars 759 forks source link

[Roadmap] Implement proper revision system #5028

Closed SchrodingersGat closed 3 months ago

SchrodingersGat commented 1 year ago

Currently the "revision" field is a simple text field, does not have any functional meaning.

Future development should focus on implementing proper "revision" for Part:

### Tasks
- [x] Prevent deletion of part which is used in an assembly -https://github.com/inventree/InvenTree/pull/7260
- [x] Add "locked" attribute to Part model - https://github.com/inventree/InvenTree/pull/7527
- [x] Prevent update of BOM items / parameters / etc for locked parts - https://github.com/inventree/InvenTree/pull/7527
- [x] Add "revision_of" attribute to Part model - https://github.com/inventree/InvenTree/pull/7585
- [x] Implement validation rules for variants / revisions - https://github.com/inventree/InvenTree/pull/7585
- [x] Add revision selection to PUI - https://github.com/inventree/InvenTree/pull/7585

Upvote & Fund

Fund with Polar

matmair commented 1 year ago

This will be a killer feature but there are a bunch of things to consider; especially how it factors into the API.

wolflu05 commented 1 year ago

What about returning the active data (I think we discussed that there should be a active/latest marked revision) for the current requests (an new rev parameter is obmitted) so it's downwards compatible. And if there is an rev parameter we search for the particular revision. Can this be implemented with an Mixin? Wouldn't that remove so much work for the future if we do it properly?

martonmiklos commented 1 year ago

Uhm I do not want to do feature creeping, but PLM like change managment features would be very useful in some small business use cases. I am thinking mainly about enforced approval processes of the revision changes. For e.g. if someone wants to make a new revision active then an approval process needs to be followed in order to make the change. This could contain approvals from various parties involved in the products lifecycle (for e.g. test engineer, contract manufacturer, etc..) If all approval done the new revision is made active automatically.

matmair commented 1 year ago

I think after we focus on cleaning up a bunch of things this would be really great; although I would prefer to get #4826 across first.

bberg115 commented 5 months ago

We were discussing how revision control is the biggest feature gap for our use in Inventree in a recent team meeting. I know the MES effort is the current goal from this post, but I was wondering what sort of effort is required for the revision re-write?

Specifically, we'd love for the revisions to cover the three bullets in the OP and have the ability to create new revisions for any part that the revised part is "used in". That way any revision on a BOM could revise the actualy master part as well.

I am sure I can convince my employer to donate funds/coffee to this effort, but it would be beneficial to have some numbers before asking for them to become a sponsor.

SchrodingersGat commented 5 months ago

@bberg115 automatically generating new revisions for assemblies would be a much tougher ask - how can we account for the particular requirements of any given company who might handle variants differently?

I would like to come up with a core framework for defining revisions, etc, and then the plugin system could be used to write a plugin that does what you want, and even be tailored to your company's particular requirements.

Otherwise the different options and edge cases would become unsustainable.

That being said, this issue is still very important for me, and any funding towards its development would be very much appreciated.

Additionally any insights into how people would use it or like to see it structured would be helpful too.

bberg115 commented 5 months ago

@SchrodingersGat thanks for the follow up. Your comment makes sense, especially regarding the plugin feature.

Having fundamental revision control tied to the tool would go a long way for our company as we're considering a paid variant instead due to the overhead of keeping revisions sync'ed between PDM/CAD and Inventree.

What is your best guess is for the timeline of this request? Is there some metric or standard practice for donations/sponsors? I would love to bring this up with leadership but I do not have a good reference for the dollar amount or associated timeline.

martonmiklos commented 5 months ago

I will try to collect my fading memories on the topic which might be helpful defining the concept/goals.

I have used two PLM (product lifecycle managment) systems in the past: Agile (from Oracle) and Arena (now part of PTC). I used the Agile in a contributor role at a bigger company and the Arena as something like an 'admin' (I set it up for the company, put together processes etc.) at a very small company.

They both shared similar concept on revisioning what I am trying to remember and describe below.

Each revision is basically a capture of all aspects of the part: BoM, parameters and properties.

The revisions had a status which could be:

The states above only make sense if further constrains are made on the Active and Superseded state:

The revision changes were implemented by change orders in each systems. In Arena I have been able to "look behind the courtains": these processes were flexible, user definable with approvals, etc.

When the change order was submitted a validation had been run on checking the BoM's of all affected items for non-active BoM items. After the validation is passed the change got "submitted" and all affected parts to "Change order submitted" state. After the change process finished all affected part revisions moved to active.

matmair commented 5 months ago

When implementing inventree_approval I found it very useful to have a clear set of the imagined workflow in steps and conditions that might influence steps. Could you provide something like that @bberg115.

Most of the company/industry-specific complexity like states, approvals, (change) orders and sync with external systems could be handled by plugins like @SchrodingersGat - especially if we hook into extendable surfaces like generic states (see https://github.com/inventree/InvenTree/issues/4289 - pretty sure this will land in 0.16.0), state transition hooks, metadata and events.

A few challenges:

matmair commented 5 months ago

@bberg115 we/specific devs have gotten sponsorships for similar complex things in the past but I am not sure if the sponsoring company wants this info publicly available - please reach out to me (mail-in GH profile) and support@inventree.org to get a ballpark. We always appreciate sponsorships as the project is a lot of work - especially for Oliver - and having your company maybe sponsor publicly would also make questions like this easier to answer. We can provide invoices via polar.sh if needed.

bberg115 commented 5 months ago

@matmair that's a good point. Here's a very high level logic diagram of what the MVP plugin would do for us.

image

This would only make sense if the "revision" was tracked like "batch code" by being tied to the actual stock. Our current work around consists of connecting date codes (via Batch Code Template) and an external database that can translate revisions of stock.

I have sent you an email. Thanks for sharing that info.

matmair commented 5 months ago

Ok I think that simple of a flow should be part of the core system and be reasonably simple to implement. Thank you for sharing that. I will wait a few days for a response from Oliver and Lukas but I can see a possible outline developing.

SchrodingersGat commented 5 months ago

To add some context here, as this has been a problem we have been considering for quite some time:

Revisions as Parts

I think the most sensible way to handle creation of "revisions" is to add a new field revision_of which points to the base part. Each time you make a new "revision" of a part, it is simply a new part which points back to the original part. Each part has its own unique BOM, parameters, etc.

revision

Revision Field (Existing)

Each Part has an existing revision field - this is just decorative and has no functional business logic behind it currently. When we move to a proper relational revision system, we can either keep this field (and use it to denote the "name" of the revision) or we remove it and use some other mechansim. I would vote to keep it, but then we have the issue of how we migrate existing systems where users have already filled in this revision field. Do we (can we) automatically create a new "revision" for each part that has a revision field specified? Or do we require the users manage this manually (somehow)?

Part Locking

I would suggest we add in a new part attribute locked (or similar) which indicates that the part definition is fixed and cannot be edited. If set, then part parameters and BOM entries cannot be adjusted. This would be important for parts which have been checked and are ready for production. Any new part (or revision) is by default not locked.

BOM Protections

Currently, BOM items can be created, edited or deleted without any particular protecitons. I would suggest that we add in (at least) the following:

Any other suggested protections here?

Revision Restrictions

The following restrictions would be in place (feel free to suggest others):

I will continue to ponder on this and would appreciate feedback on the ideas above. It is important that we get this right!

wolflu05 commented 5 months ago

I like the idea from @SchrodingersGat of how part revisions should work, (although I currently have not a real need for it, its more a nice to have feature for me).

SchrodingersGat commented 5 months ago

And is a new panel on the parts page which just lists all revisions in a table and highlights the current just enough for now for navigating between the revisions?

I would also suggest that on the part detail page, if the part has "revisions" there is a dropdown menu (with the displayed revision selected) - the user can open that and select a different revision to display.

We can use the existing "active" part flag to show which revisions are in use, but how do we determine which is the "most up to date" revision? Just using the "revision" attribute field? Note that some companies may want to have multiple active revisions at any one time.

wolflu05 commented 5 months ago

Maybe we can assume all active parts are the used/latest revisions, and highlight the inactive in light grey.

And just an idea, not sure how much switching between variants is necessary, but maybe we could also show a revision drop down selector right to the breadcrumb to easily switch between revisions.

SchrodingersGat commented 5 months ago

A selector to the right of the breadcrumbs seems like a clean implementation to me.

SchrodingersGat commented 5 months ago

Regarding "change management" - I would like to push this off to a plugin entirely. Maybe we offer a "first class" plugin, but we should not enforce approvals, etc, at this level.

bberg115 commented 5 months ago

I think both @SchrodingersGat and @wolflu05 have valid points with respect to locking. The thought of it being a plugin is nice, but I am not familiar how a plugin would stop someone from making changes if the plugin enforces approvals.

With respect to this reply - https://github.com/inventree/InvenTree/issues/5028#issuecomment-2119087615

Revision Field (Existing)

  1. I think this is a little tricky - as you've noted - to have two different revision fields. Could we consider an option in Inventree settings to which style to utilize (maybe default to classic)?
  2. I do think it would be safest to require users to manually input the original revision as you could reference an external part revision which may not start at the first index of rev (aka - I buy part X from Digikey and it's already rev 22, I don't want to rev 22 versions to sync up reality). This could complicate my next thought below (3) if an external revision uses a different index for revision control.
  3. Another important aspect that would be nice (sadly, it may be best to be a plugin) where you could specify the revision index array. Example, we currently use two types of revisions - prototypes get a Rev ## (eg. Rev 01 is revised to 02) and production parts get Rev X (Rev A gets revised to Rev B). I am picturing an array where you get to list out the revisions (A, B, C, D, E... ). I think it's reasonable for our users to have to specify a production rev when revising a prototype part (Rev 03 gets revised to Rev A)

Part Locking I love this idea. This is how most PDM software work and having inventree use a similar concept would minimize confusion. I think it would be nice to take this a little farther and adding some approval metric - maybe a feature that marks a part "controlled" which may require a seperate user to review and approve the change before unlocking. An example is technical drawings - these typically note what user drew the document, who reviewed it, and who approved it.

BOM Protections Agreed - part locking should unlock or block the BOM (as @SchrodingersGat noted). I'd go one step further in blocking or notifying the user that the BOM change will effect open build orders. We currently run into issues where a user "fixes" a BOM but any open build order does not get updated. Could it be possible to have an extra "validate BOM" callout for the entire BOM once all part have been validated? Stretch goal 1 - it would be nice if this effort could address open Build Orders that are effected by the BOM change Stretch goal 2 - changing BOM would kick off a new revision for the main part

Parts cannot be deleted if there are BOM entries which reference them. The underlying parts must be deleted first
BOM items cannot be deleted if they point to a locked assembly
BOM items cannot be edited if they point to a locked assembly
BOM items cannot be created for a locked assembly
Any other suggested protections here?

Nothing else comes to mind - it may be nice to not be able to "deactivate" a part with the same logic.

Revision Restrictions

Cannot have a revision of a revision. All revisions must point to a part which is itself not a revision. @SchrodingersGat - Can you explain this thought some more? My assumption is that revisions are typically a change to the latest rev and not normally the "orginal" revision. Example: Rev 10 is closer to Rev 09 than it is Rev 01.

I think a majority of these topics may be best as boolean options from settings. I can picture users having different preferences for locking, protections, and restrictions. This could be a future solution, as I am sure it adds scope. Maybe the original effort would turn them all on or off?

bberg115 commented 5 months ago

Additional information from @matmair 's reply -

Which system are you using? We are currently using Solidworks Vault for PDM. We are also reviewing PLMs (and some ERPs) that can help support revision control. Honestly, I think a future version of Inventree could do what we want but the lack of state control is one the biggest issues we have with the current version.

Examples:

  1. if someone changes a part (revision, name, BOM, etc.), it's not easy / possible to stop or track the difference within inventree.
  2. revisions - we are constantly making new revisions for build orders. However, we cannot easily look at a Sales Order from the past and know what revision was shipped with out an external database linking revisions to dates.

Would you code the integration or is this something we should do

I could see this going either way, but I do think it may be be best as a plugin. I think the MVP of this effort is some sort of background worker that identifies differences (revision mismatch, missing parts, etc) between the two systems. It would be nice if the PDM could link to the part file in inventree to push any changes - even if it is a manual process. Even something as simple as a plugin that highlights changes in the previous 24 hours (possibly a report?).

Which system is the leading one (ie in a conflict which is the source of truth)

This is always a difficult internal discussion - we end up using PDM as source of truth because it's a controlled source (you can lock out files from being edited and require review to "revise"). However, I think we would lean on Inventree if this could be done in the tool itself as a majority of our team interacts with inventree and only a view have access to PDM (due to cost/training).

What would you like to sync and would this be a static selection or do you require dynamic controls like depending on parameters, responsible groups, weighted rules

I think it would be nice, but not required anytime soon. Personally, I think there is value in users comparing the two databases for mismatch as there's a lot of stuff to review and not just a straight forward comparison. In addition, our PDM only covers mechanical stuff and we have to identify a solution for our other platforms - software and PCBAs.

Do you see PDM integration as part of this effort or a separate one

I think it should be considered as a way shaping future versions on inventree, but I assume it will be best to be seperate them as there are so many flavors of PDM/PLM.

matmair commented 5 months ago

I think we have the requirements on the table - maybe it would be best to now create an epic with a short outline of the target function, python/REST APIs and a tasklists for the required changes.

Backend

To keep this generic I would suggest adding 2 models:

The business logic for parts would need the adoption mentioned above by @SchrodingersGat - maybe we could also call state transitions when deleting / deactivating parts to make that also available to plugins. One interesting question: are IPNs still enforced to be unique between revisions or does the check allow parts with revisions to share IPN - should that be configurable?

REST API changes:

Frontend

I like the idea of a dropdown in the breadcrumbs, maybe we can also add an admotion or alert if an old revision is viewed.

matmair commented 5 months ago

@bberg115 thank you for your inputs - I think we should split out PDM integration into it's own issue as there is only loose (business logic) coupling in this area. Do you agree?

bberg115 commented 5 months ago

One interesting question: are IPNs still enforced to be unique between revisions or does the check allow parts with revisions to share IPN - should that be configurable?

REST API changes:

@matmair I think this is tricky. I'd suggest that "unique" is IPN+Revision.

An example would be for part 10005 - you can have 10005 (original part, per Oliver's reply above), 10005-01 (first revision), 10005-02 (2nd revision), etc.. Therefore, Inventree would see 3+ unique IPNs and allow for new revisions without allowing for duplicate revisions.

@bberg115 thank you for your inputs - I think we should split out PDM integration into it's own issue as there is only loose (business logic) coupling in this area. Do you agree?

Agreed. They both effect each other but do not need to be tied together.

martonmiklos commented 5 months ago

Frontend

I like the idea of a dropdown in the breadcrumbs, maybe we can also add an admotion or alert if an old revision is viewed.

I would like add one important thing to the frontend wishlist: revision diffing tool to being able to quickly see what BoM items/parameters/properties had been changed.

motusRP commented 4 months ago

I'd like to add some thoughts to this as i'm implementing Inventree at my current company and am looking for ways to document changes in the interim before there is an "automated" way to do it within Inventree.

Every company i've worked for over the last 25 years has had the concept of a major and a minor change. When it comes to assemblies, what does that mean?

So if you have the following assembly, where Major revisions are letters and Minor revisions are numbers:

If you make a minor change to 22-00002-003 to raise it to A1, the new BOM becomes:

And if you made a major change to 22-00002-003 to raise it to B0, the new BOM would become:

There would be no recursion higher than this, otherwise you'd be revising every level of your BOM every time any part on any level of the BOM were updated.

As an MVP system, i'm looking at using the ability to duplicate a part to approximate a revision control system that could retain some of the history of the overall BOM. I'm not quite to a complete solution, but this might give some ideas how to approach a more integrated solution.

The plan: Background Setup:

Using the part tree from above as an example:

For a MINOR revision of a part (eg. 22-00002-003 Rev A0)

For a MAJOR revision of a part (eg. 22-00002-003 Rev A0)

This could be done fairly easily manually, but it would be nice if it could eventually be automated in some way. I've worked through it a few times and I don't think there's anything amiss... Old BOM's end up marked inactive, but still available to view if you need/want to. No, there's no "control" on this, so even though the parts are inactive, they can still be edited, unless there's a permission somewhere that would prevent that for most users.

I've been looking at this for a week now and this is the first time i've written it all out as a process so apologies in advance if i've overlooked something fundamental or if it's not clear... Happy to add more detail or work on a flowchart to show it more clearly at some point. :)

gunstr commented 3 months ago

I came across this discussion and just as a referece regarding the IPN uniqueness I described my usecase last year in issue #4618.

To summarize, I want to validate that each new base part is assigned with a unique IPN. If a part has multiple revisions the same IPN should be used for all revisions of the same base part. As this validation scheme is not supported by the current core code I have done such validation with a plugin since last year. It is also worth to mention that the assigned serial numbers are unique for each base part irrespective of revisions which is good.

So my suggestion for the question from @matmair

One interesting question: are IPNs still enforced to be unique between revisions or does the check allow parts with revisions to share IPN - should that be configurable?

is to allow parts with revisions to share IPN, but still it should be possible to validate that the IPN is unique for the base part

SchrodingersGat commented 3 months ago

@gunstr that could be implemented as a custom validation plugin, but it does sound like it would be useful as a first class feature, perhaps behind a configurable setting.

bberg115 commented 3 months ago

@motusRP I think you have a great summary, sorry for my delayed response.

I think the only thing I would add is that major/minor revision "list" should be customisable as we all have different means noting major/minor revisions.