Feature request: Option to specify a Part IPN (IPN) to set on imported parts

[cmidgley]

Currently, when imports create new parts the Part IPN is not set. I have a use case where setting it to the search term (SKU/MPN) would be helpful.

Use case:

My schematic uses SKU (supplier part numbers) to identify components, not MPN. This is done for a couple of reasons, one being that I use JLCPCB assembly where I use LCSC part numbers to auto-match the BOM to their assembly process, and another is it defines a specific part to use. When I import these into Inventree, the MPN is used for the Part ID and the LCSC part number is on the Supplier Part, which makes a lot of sense and is great when then running --update-recursive because it pulls in matching parts from other suppliers (fantastic for prototype parts and when not using JLCPCB assembly).

However, when I use my BOM to import parts into an assembly, none of my parts match. This is because the Part ID is MPN and the Part IPN is blank, so the BOM's use of Supplier (LCSC SKU) IDs doesn't match anything.

Proposed feature:

I propose a new option to set the Part IPN (only when creating a new part) to the SKU/MPN supplied on the import.

inventree_part_import --ipn -o lcsc C12345

Assuming that part C12345 (MPN STM32F407VGT6) hasn't been imported yet, this would set the Part IPN to "C12345" (and the part ID to "STM32F407VGT6"). This will allow a BOM import will work with either "STM32F407VGT6" (as always) as well as "C12345".

[30350n]

Thanks for the detailed description of your use case and the PR!

inventree_part_import currently doesn't support IPNs because I don't really use them, nor see the point of using them, so nice to see an actual use case from someone else. I do wonder though how useful the exact behavior from your implementation would be for other people though.

If I remember correctly, Ki-nTree allows for setting IPNs based on a user defined pattern, so you could name parts automatically for example based on category. (For example all your Resistors would have IPNs starting with RES- or stuff like that). Such a system would make more sense imo and could also incorporate your exact use case (if there'd be a pattern option for the search term). In it's current form I'd argue that only having the option to set the IPN to the search term is kind of confusing though.

Also I feel like the actual issue is on InvenTree's side here 😅 Your main problem seems to be that the BOM import in InvenTree doesn't support SKUs, right? Which should also be relatively easy to fix (I don't really see a reason why that shouldn't work atleast).

So to conclude, in this current form I find the feature too confusing to include. So I'd propose we 1. try to get the BOM import in InvenTree fixed and 2. think about extending this PR to allow the user to set the IPN based on a pattern. What do you think?

[cmidgley]

I do agree that there is room for improvement on the InvenTree side with BOM management, including more flexibility in the BOM part matching system. If that were to support matching to supplier number it may solve the BOM import part of this problem.

Even still, there is still an important issue here related to IPN with inventree-part-import. Let's take your example of using names like "R_1K_0402" in the schematic. We would need to provide inventree_part_import with two values for each part - a part name (MPN/SKU, same as currently done) and a custom IPN. This would allow a BOM import to use an abstract IPN (such as "R_10K_0402") to match parts, and take advantage of InvenTree to map parts to suppliers on a per-build basis.

A summary of changes based on this proposal are:

• [ ] Add a new CLI option such as --set-ipn false|true|overwrite , where false is default, true is enable IPN support but do not overwrite existing IPN, and overwrite will always overwrite the IPN on matching parts
• [ ] Extend CLI so that Part Name values be paired with Part IPN when the option is set

  inventree_part_import --set-ipn CRCW04021K00FKED R_1K_402 
  inventree_part_import --set-ipn CRCW04021K00FKED R_1K_402 CRCW04021K10FKED R_1K1_402

• [ ] Extend file import to support multi-columns, one for Part ID and one for Part IPN. It would prompt for the second field name similar to the current single field. If the file is not multi-column the operation fails.
• [ ] Add the code to set the IPN based on the business logic above.

[30350n]

Even still, there is still an important issue here related to IPN with inventree-part-import. Let's take your example of using names like "R_1K_0402" in the schematic. We would need to provide inventree_part_import with two values for each part - a part name (MPN/SKU, same as currently done) and a custom IPN. This would allow a BOM import to use an abstract IPN (such as "R_10K_0402") to match parts, and take advantage of InvenTree to map parts to suppliers on a per-build basis.

This kind of opens a whole other can of worms and is already beeing discussed in #32 and #20. Assigning the same IPN to multiple parts (I think that's what you are suggesting?) is not possible, as IPNs have to be unique. The ideal approach would be to have multiple ManufactureParts per Part (atleast I think), but that also wouldn't be perfect.

We should also consider how all of this integrates with inventree_kicad as that'll probably be the best way to interface between InvenTree and KiCad.

[cmidgley]

Thanks for the feedback and background. I briefly looked at #32 and #20, and if I understand correctly, they are addressing a different problem - which is trying to find a way to link various compatible supplier and manufacturer imported parts (potentially with a different MPN) to a single part.

I'm talking about something very different. I do not want to link multiple parts together, rather I want the ability to set the Part IPN to a custom value, as intended by the InvenTree schema. Currently, for those who use IPN and inventree_part_import, every imported part must be hand-edited only to fill in a value on the Part IPN field.

As for unique IPN, yes that is a requirement (and correct behavior) and would require graceful handling to ensure no two parts share the same IPN. The solution is not to merge parts, but rather what the import does when it finds a duplicate IPN (such as an option to specify if duplicates cause the part import to be skipped, or just the setting/changing of the IPN is skipped).

[30350n]

Thanks for the feedback and background. I briefly looked at https://github.com/30350n/inventree_part_import/issues/32 and https://github.com/30350n/inventree_part_import/pull/20, and if I understand correctly, they are addressing a different problem - which is trying to find a way to link various compatible supplier and manufacturer imported parts (potentially with a different MPN) to a single part.

Ah okay then I guess I missunderstood that, sorry.

Regarding your proposed changes:

Add a new CLI option such as --set-ipn false|true|overwrite , where false is default, true is enable IPN support but do not overwrite existing IPN, and overwrite will always overwrite the IPN on matching parts

I think I'd prefer a simple --ipn flag option as proposed by your original PR. Not overwriting seems like a reasonable default behavior and the extra overwrite use case is not really in scope for this project imo. The idea of inventree_part_import is to fully automate importing many supplier parts. Stuff like having to manually overwrite a single field shouldn't come up often and can easily be done via the web interface.

Extend CLI so that Part Name values be paired with Part IPN when the option is set inventree_part_import --set-ipn CRCW04021K00FKED R_1K_402 inventree_part_import --set-ipn CRCW04021K00FKED R_1K_402 CRCW04021K10FKED R_1K1_402

Similar thing here, the whole idea is to go from a single MPN to a complete part definition, I'd much prefer offering the tools to automatically generate the IPN based on the available part data. We could add another python file in the configuration directory for this, similar to the hooks.py. It'd have to contain a generate_ipn function which takes inventree_api, category, api_part. We could also add a _meta field to the category definitions in categories.yaml so one could add things like a ipn_prefix there which could then be retrieved in the generate_ipn function.

[cmidgley]

Let me try another attempt at describing a use case, because I feel like we may be discussing a solution without a common understanding of the problem (and I now have a better understanding of the problem and how InvenTree is designed to address it):

In my world, I start with a schematic and components in my electronics design tool. While I refer to InvenTree during design, I don't update the database until the board is done. Because I use a board assembler for production that requires I use their parts (JLCPCB/LCSC), I add a SKU attribute to all my components (I can dive deeper into why SKU over MPN if interested, but it's quite intentional). When done with design, I export an InvenTree-standard BOM.

The BOM contains two key ingredients - some identifier that links the BOM entry to my schematic (such as "R_1K_402"), and some SKU that identifies a specific part. I take this BOM and have inventree_part_import import it which makes a Part, a Manufacturer Part, and multiple Supplier Parts for each entry. It can even map my vendor-specific SKU to an MPN and locate all matching parts from other suppliers with the --update-recursive option. Totally awesome!

Now with all my parts created, I import my BOM into InvenTree... but it fails to match up to any of my new parts. This is because my BOM part has SKU as the Part ID but the database part has an MPN (as it should).

What is needed is a common key between the schematic and the part. This is exactly the intended use case for IPN in InvenTree. Anytime a new part is created, and the use case depends on internal part numbers, the IPN must be set to link them together or the part will not be found when importing a BOM. What is needed is for inventree_part_import to attach an IPN from the BOM to the newly created part.

Hopefully, given this description, you can see why it's a simple and appropriate task for import to handle setting this single field (and why automatically generating an IPN would break everything). Also, I hope you can see why using the UI to update these parts is challenging, as there may be hundreds of parts where even a tiny data entry mistake can result in errors or incorrect parts being selected in the BOM.

Does this help narrow the focus and clarify the need?

[cmidgley]

After a bit of thought and reviewing your feedback earlier, I have a new proposal.

• New ipn_template field in config.yaml and _ipn_template in categories.yaml, which are Jinja2 templates that are used to define an IPN value. The field in config.yaml is the default (if no other templates located), and categories.yaml is specific to the category (and it's children, unless overridden).
• Support multiple context variables, including parameters, SKU, MPN, Vendor and part_id (which provides a unique ID number for IPN's such as ID-293 or RES-293).
• New CLI --ipn never|new|always option to specify when an IPN is allowed to be changed (where new is default, and only changes the IPN when it isn't already set)

This approach has several benefits:

• More flexible control, including at a category level, on IPN naming
• Able to extend this in the future, especially once this package becomes a plugin to InvenTree, as that would allow access to template tags, models, etc.
• No change in current behavior unless templates are defined in config files.

For my use case, I have a simple default template of {{ SKU }}. Others may wish a more detailed IPN such as RES-{{ part_id }} on the Resistor category and CAP-{{ part_id }} on Capacitor, or even RES-{{ parameter.Resistance }}-{{ part_id }}.

Pull request (#46) implements this proposal.

[cmidgley]

I've been using the code in my pull request (#46) and I've also implemented a new supplier (BoltDepot, I'll be doing a pull request on that after I've finished importing all my parts to ensure it's fully working). BoltDepot doesn't share manufacturer information, so in their case MPN is the same as SKU. So my global template for IPN is now {% if MPN != SKU %}{{ SKU }}{% endif %}. This only sets the IPN to the SKU if it is different than the MPN. Pretty cool...

[30350n]

Hi! Sorry for not answering your previous post (I didn't have a lot of time).

This new proposal is much more along the lines of what I'd imagined for this. I like it a lot, great job!

Able to extend this in the future, especially once this package becomes a plugin to InvenTree, as that would allow access to template tags, models, etc.

I've thought about directly integrating into InvenTree a bunch more and I think if that were to be done, it could be done in a way more elegant way than this. So if I'm ever going to attempt that, I'd most likely start from scratch.

BoltDepot doesn't share manufacturer information, so in their case MPN is the same as SKU. So my global template for IPN is now {% if MPN != SKU %}{{ SKU }}{% endif %}. This only sets the IPN to the SKU if it is different than the MPN.

We need both a SKU and a MPN in InvenTree anyways, so you should probably just set mpn = sku in the supplier class. I already do similar things for some TME parts that don't specify a MPN/Manufacturer: https://github.com/30350n/inventree_part_import/blob/d9182cc5ff56dff396ff142aa040276f25afe8b0/inventree_part_import/suppliers/supplier_tme.py#L86-L88

Thanks a lot for the PR, I'll leave some more comments regarding the implementation there.