yaqwsx / jlcparts

Better parametric search for components available for JLC PCB assembly
https://yaqwsx.github.io/jlcparts/
MIT License
572 stars 52 forks source link

buildtables overwrites files on (case-insensitive?) category collision #140

Open ducky64 opened 3 months ago

ducky64 commented 3 months ago
An example is with inductors, which have category entries id category subcategory
12 Inductors/Coils/Transformers Inductors (SMD)
695 inductors/coils/transformers Inductors (SMD)

On Windows at least, both write to InductorsakaCoilsakaTransformersInductors_SMD.json.gz, so in practice the latter one overwrites the former. The latter one is also completely useless, containing just one part.

There's a bunch more of these cases.

Quick and dirty way to not get it to overwrite: in jlcparts/datatables.py, add:

  def saveJson(object, filename, hash=False, pretty=False, compress=False):
+     if os.path.isfile(filename):
+         print(f"**** dropping {filename}")
+         return

A cleaner solution may be to append the tables instead of overwriting, treating it as one giant file. Which doesn't adhere to the input data as strictly (especially in terms of category ID), but maybe is closer to the spirit of the data?

Actually, category name standardization (across case, / vs. & vs. ,) could probably clean up the data quite a bit