30350n / inventree_part_import

CLI to import parts from suppliers like DigiKey, LCSC, Mouser, etc. to InvenTree
MIT License
24 stars 8 forks source link

[FEATURE[S]] Add Datasheet and HTTP-LIB Support #1

Closed Nudelsalad closed 6 months ago

Nudelsalad commented 6 months ago

Hi @30350n, awesome work! I already managed to adopt your categories/parameters structure. Very straight forward! I would love to have a feature of a datasheet getting pulled and pushed to inventree (as urls often change and may be broke in future) but I can live with it if inventree_part_import could manage to pull the URL. I suggest you to add the Operating Temperature parameter to every part as this is sth crucial in my opinion missing in the template so far. The HTTP-Lib support is also something in my opinion that should be considered. Ki-nTree issue With your tool this is possible put 2 parameters for every part separately maybe kicad_sym and kicad_foot and link them with the kicad http plugin of inventree. But @konkers doing some extra magic defining for generous types (Diode, 0603 Resistor) the kicad symbol and footprint and linking them there. Even more userfriendly not defining for every type every time!

These are just some thoughts with existing workarounds(except for the datasheet). I would love to see your software as plugin inside inventree and would be interested in a Roadmap for this piece of software! Regards!

30350n commented 6 months ago

Hi, thanks for checking out the project! :)

I would love to have a feature of a datasheet getting pulled and pushed to inventree (as urls often change and may be broke in future) but I can live with it if inventree_part_import could manage to pull the URL.

This is something I've thought about, but haven't really needed myself, but it should be fairly easy to implement. I'll take a look at it the next days ^^

I suggest you to add the Operating Temperature parameter to every part as this is sth crucial in my opinion missing in the template so far.

For my usecases, it really doesn't matter that's why I don't have it included in the default config nor my personal config. I think I'd like to keep the default config realitively simple, so I think I'd keep it how it is. You should just be able to add it to your personal config though. You can also use version control on it, my personal config for example is available here.

The HTTP-Lib support is also something in my opinion that should be considered.

Already working on it ^^, though my approach is a little different: Instead of adding a kicad footprint parameter for every part, I thought it'd be cleaner to just have a mapping from the already present Footprint Type -> Kicad Footprint name.

I haven't really thought much about schematic symbols though tbh ... For resistors, caps etc. it's relatively easy, you can pretty much just set it for the whole category. But especially for ICs, I guess you'd need specific symbols for each one pretty much 🤔 Maybe direct integration into inventree_part_import would be useful there.

I would love to see your software as plugin inside inventree and would be interested in a Roadmap for this piece of software!

Me too 😅 ... not sure when or if that's happening though. There has been some work on building a base for this in #3791 but atm the InvenTree dev team is focused on the new PUI frontend, so there's little movement on that front (afaik). The reason this exists in the form it does right now is that I needed something that just works, now. And this seemed like the easiest option for that.

konkers commented 6 months ago

https://github.com/afkiwers/inventree_kicad/pull/44, though my approach is a little different: Instead of adding a kicad footprint parameter for every part, I thought it'd be cleaner to just have a mapping from the already present Footprint Type -> Kicad Footprint name

I like the idea of a separate package to footprint mapping. However I still think it's useful to have some sort of normalization step before part creation so that you don't end up with multiple variations of 0402R in your database. KiCad Footprint is probably not the best normalization form so a translation between vendor name and canonical name for Package Type at import time out be useful.

There's also automatic category detection to think about.

For schematic symbol, this is where Ki-ntree's interactive approach has an advantage. It gives you an opportunity to manually specify the symbol between querying the vendor and part creation. Going back and adding the symbol through Inventree is more time consuming and more clicks. I wonder if a hybrid workflow where inventree_part_import creates an intermediate CSV file and allows the user to edit/annotate it before batch part creation would be a reasonable solution.

30350n commented 6 months ago

There's also automatic category detection to think about.

inventree_part_import already has this.

For schematic symbol, this is where Ki-ntree's interactive approach has an advantage. It gives you an opportunity to manually specify the symbol between querying the vendor and part creation.

inventree_part_import has an interactive mode now to automatically select unmatched categories and automatically specify unmatched parameters. So this is trivial to "implement", you'd really only have to add the symbol parameter. (This is also functionally equivalent to the hybrid workflow you propose I'd say, if not better).

Nudelsalad commented 6 months ago

For resistors, caps etc. it's relatively easy, you can pretty much just set it for the whole category. But especially for ICs, I guess you'd need specific symbols for each one pretty much

And that's ok! You cannot specify every symbol possible in advance. I am speaking of generous ones like in here: Template

https://github.com/afkiwers/inventree_kicad/pull/44, though my approach is a little different: Instead of adding a kicad footprint parameter for every part, I thought it'd be cleaner to just have a mapping from the already present Footprint Type -> Kicad Footprint name.

I really like this approach and I am stunned on how you are actually contributing to inventree and their plugins. This is also something that I noticed and this pr is a very nice workaround. I also fiddled arround a bit with the newes partdb. In partdb there exist several pages for a part where you can add datasheets, etc, where the developer added a separate Kicad EDA page. There he used a helper script to generate a simple text file where kicad standard symbols and footprints are in. To have a drop down menu to choose which one of the standard symbols you want. For a category one can specify the same symbol if wanted image and inside a part you can specify both if wanted: image

Just mentioning this here cause you linked the pr.

To continue, the operating Temperature was fairly easy to implement, also a good idea to version control this... The datasheet can sadly not be found via a parameter but I may also have a look into it. Also wondering where the datasheet mapping in the kicad inventree plugin is?

Thanks so far for your work!

30350n commented 6 months ago

There he used a helper script to generate a simple text file where kicad standard symbols and footprints are in. To have a drop down menu to choose which one of the standard symbols you want. For a category one can specify the same symbol if wanted.

This is interesting. Thought about something similar, mainly for symbols yesterday, but I think I'd prefer if that was queried "live". That way you could maybe even include custom user symbols/footprints in the selection (if inventree_part_import is running on the same machine you have KiCad installed on).

Also wondering where the datasheet mapping in the kicad inventree plugin is?

Here. Seems like they are just checking for an attachement named datasheet.

30350n commented 6 months ago

@Nudelsalad datasheets are supported on master now. I've also added the "datasheet" comment to them, so this should just work with inventree_kicad.

Would be great if you could give this a quick test and see if everything works as it should ^^

Nudelsalad commented 6 months ago

Hi @30350n,

but I think I'd prefer if that was queried "live".

Haven't thought of this, but would be even cooler! Specifying the symbol library and footprint library path in the configuration routine and then having them linked for a helper script. Maybe these scripts help, but like mentioned in the repo description not very flexible written with regex

Would be great if you could give this a quick test and see if everything works as it should ^^

Tried it out, worked out of the box for me. The Datasheet gets correctly shown in inventree under the attachements category. Thanks for your quick implementation!

Nudelsalad commented 6 months ago

This is interesting. Thought about something similar, mainly for symbols yesterday, but I think I'd prefer if that was queried "live". That way you could maybe even include custom user symbols/footprints in the selection (if inventree_part_import is running on the same machine you have KiCad installed on). (https://github.com/afkiwers/inventree_kicad/blob/252ef57f3446cbd3200ebfd5017bebd2785e2bb7/inventree_kicad/serializers.py#L173C20-L173C20) Seems like they are just checking for an attachement named datasheet.

Hi @30350n, I guess I can close this, as Issue #44 will bring in the footprint mapping which is even more awesome. Thank you for your contribution so far i guess i will heavily use these tools and maybe contribute in near future! Thanks.