Question: best approach to modify the project to accept generic files (*.xy, *.bom, front.svg, back.svg)

[antonysigma]

I tried out the demo on the project website. It is such an eye candy! Thank you for sharing this project.

The only catch is that I am not using Kicad to design my PCB. I wonder how can I modify your code to support the following command line interface?

python3 path/to/InteractiveHtmlBom/generate_interactive_bom.py \
    --xy board.xy --bom board.bom\
    --front board-front.svg --back board-back.svg\
    --width-dpi ${width} --height-dpi ${height}\
    --output interactive.html 

My EDA software (PCB-RND) can export the following format:

• pick-n-place coordinate file: filename.xy;
• bill of materials: filename.bom;
• photorealistic shot of the pcb, front side; and
• photorealistic shot of the pcb, back side.

Some features that I am willing to give up

I am fine to trade a few advanced UX in InteractiveHtmlBom in return of the eye-catching BOM table and the zoomable front-back PCB images in the interactive html.

For example, the coordinate file filename.xy does not carry the silkscreen information, so the html output will not being able to "highlight" the silkscreen of the footprint in red bounding boxes. Instead, a red rounded "cursor" of ~50mil radius is sufficient for me to locate the corresponding element on the front-back image of the pcb.

Where I got the idea from

What I am using right now is the interactive pick-and-place helper at https://gist.github.com/russdill/3081369, where I can execute the following commands to get me an interactive session

dimensions=$(grep '^PCB' $(filename).pcb |sed -e 's/[^0-9 ]//g'|awk '{print "-w" $$1/1e6 " -h" $$2/1e6}')
pcb -x bom --bomfile /dev/null --xyfile $(filename).xy --xy-unit mm $(filename).pcb
gnetlist -g bom2 -Oattribs=footprint,value $(filename).sch -o $(filename).bom
pcb -x png --outfile $(filename)-front.png --dpi 600 --use-alpha --photo-mode $(filename).pcb
pcb -x png --outfile $(filename)-back.png --dpi 600 --use-alpha --photo-mode --photo-flip-x $(filename).pcb
python ../pnp.py --xy Arduino_LED.xy --bom $(filename).bom \
    --front $(filename)-front.png --back $(filename)-back.png -s footprint,refdes\
    ${dimensions}

References: http://support.seeedstudio.com/knowledgebase/articles/1911202-how-do-i-export-pcb-pick-and-place-xy-files-for

[antonysigma]

Update: my usecase is very similar to https://github.com/openscopeproject/InteractiveHtmlBom/issues/101, except that I have more data other than the positional files and Gerber.

[qu1ck]

Hi, thanks for reaching out. I took a look at pcb-rnd, it's an interesting project, but best of all, open source.

What you described above can be done but will require some brute force to shoehorn a square peg into a round hole.

Much better approach: go with existing support for ecad parsers, take a look at EcadParser class and it's implementation for EasyEDA. You will also need to understand internal ibom format which is what gets injected into generated html page: dataformat.

So my suggestion would be to add an EcadParser implementation for pcb-rnd format. You can go about it in 2 ways with varying amount of complexity in different fields:

1. Write native lihata parser in python and parse pcb-rnd's objects from the file directly.
2. Write an intermediate format exporter as pcb-rnd's plugin or script (http://repo.hu/projects/pcb-rnd/user/06_feature/scripting/intro.html). Then write a close to trivial EcadParser that will read the intermediate format.

...Or you can skip all of that and just save your board into kicad format right from pcb-rnd and run ibom from kicad :)

[qu1ck]

There is also option 3. Write an EcadParser for specctra SES/DSN files. Pcb-rnd and many other pcb design software supports them so you will kill many birds with one stone.

[antonysigma]

Thank you very much for pointing out the EcadParser and the dataformat documentation.

In the spirit of avoiding vendor lock-in, I prefer coding a GenericZipParser that extracts photo-realistic front/back pcb images and the pick-n-place coordinate files, and then render the interactive html as is.

diff --git a/InteractiveHtmlBom/ecad/__init__.py b/InteractiveHtmlBom/ecad/__init__.py
index d937418..a836896 100644
--- a/InteractiveHtmlBom/ecad/__init__.py
+++ b/InteractiveHtmlBom/ecad/__init__.py
@@ -7,6 +7,8 @@ def get_parser_by_extension(file_name, config, logger):
         return get_kicad_parser(file_name, config, logger)
     elif ext == '.json':
         return get_easyeda_parser(file_name, config, logger)
+    elif ext == '.zip':
+        return get_generic_parser(file_name, config, logger)
     else:
         return None

@@ -19,3 +21,7 @@ def get_kicad_parser(file_name, config, logger, board=None):
 def get_easyeda_parser(file_name, config, logger):
     from .easyeda import EasyEdaParser
     return EasyEdaParser(file_name, config, logger)
+
+def get_generic_parser(file_name, config, logger):
+    from .generic import GenericParser
+    return GenericParser(file_name, config, logger)

[antonysigma]

I also created a skeleton pcbformat demo code to simulate the output of the GenericZipParser. All things will be rendered in pcbdata.silkscreen layer, and pcbdata.modules will generate red-colored bounding boxes of individual components.

I am still working out how to tweak the pcbdata structure to render raster images in the canvas. As a last resort, I can code a routine rasterImageToRects to convert a binary front/back pcb image to "pixels" in the silkscreen layer.

var pcbdata = {
    edges_bbox: [
        [0, 0],
        [100, 100]
    ],
    bom: {
        B: [],
        F: [],
        both: [
            [1, "220u", "C200", [
                    ["C2", 0],
                    ["C3", 0]
                ],
                [], null
            ],
            [1, "50u", "C400", [
                    ["C4", 1]
                ],
                [], null
            ]
        ],
        skipped: []
    },
    edges: [{
        type: "rect",
        start: [10, 10],
        end: [90, 90],
    }],
    silkscreen: {
        F: [{
            start: [15, 15],
            end: [35, 35],
            type: "segment",
            width: 5
        }, {
            start: [15, 35],
            end: [35, 15],
            type: "segment",
            width: 5
        }, {
            start: [100 - 15, 100 - 35],
            end: [100 - 35, 100 - 15],
            type: "segment",
            width: 5
        }, {
            start: [100 - 35, 100 - 35],
            end: [100 - 15, 100 - 15],
            type: "segment",
            width: 5
        }],
        B: [],
    },
    fabrication: {
        F: [],
        B: [],
    },
    modules: [{
            center: [25, 25],
            bbox: {
                angle: 0,
                pos: [25, 25],
                relpos: [-10, -10],
                size: [20, 20]
            },
            pads: [],
            drawings: [],
            layer: "F",
            ref: "generic_bbox"
        },
        {
            center: [100 - 25, 100 - 25],
            bbox: {
                angle: 0,
                pos: [100 - 25, 100 - 25],
                relpos: [-10, -10],
                size: [20, 20]
            },
            pads: [],
            drawings: [],
            layer: "F",
            ref: "generic_bbox"
        }
    ],
    metadata: {
        title: "unamed"
    },
    font_data: {},
    ibom_version: "v2.3-31-g6c70↵"
};

[qu1ck]

You mentioned avoiding vendor lock in but you are choosing a route that requires user to generate a very specific zip with very specific data that roughly 0 programs support out of the box and only select few would support by the way of scripting. That doesn't sound very generic to me.

If you want to cover as much ground as possible, writing a parser for specctra .dsn file is your best bet. This format is considered industrial standard and it's specifications are open.

This will also give you the best results without crippling ibom functionality.

[antonysigma]

Yes you kind of pointed out the rift between the monolithic and microlithic design approach in the EDA community. For the gEDA-GAF + PCB-RND project, it is quite common to daisy chain individual scripts to design the PCBs. They seemed to adhere the so called UNIX philosophy: write one script to do one thing and do it well.

For example, we use the djboxsym script to generate custom schematic symbols, which is then linked into the gschem for schematic capture, and gattrib to define component footprint and values. Then, run gnetlist program to export the netlist, which is then imported into pcb-rnd to connect the pins and vias with copper traces. They even have a separate app (gerbv) to preview gerber file.

Using such scripting approach for the past 10 year, I simply grew too accustomed to it. Indeed your project, being a plugin of KiCad, deserves a much tighter integration with pcb-rnd or the Specctra .dsn file. I will definitely look into it.

By the way, the pick-n-place / centroid (*.xy) file is so far the most widely supported format in the PCB Assembly service. Perhaps one day they will favor Speectra or KiCad, but as of today the centroid file format remains the industry practice.

References: https://www.pcbway.com/helpcenter/technical_support/How_to_create_a_Centroid_File_from_Eagle__Altium_Sprint_Layout_and_ORCAD_.html http://support.seeedstudio.com/knowledgebase/articles/1911202-how-do-i-export-pcb-pick-and-place-xy-files-for https://support.jlcpcb.com/article/84-how-to-generate-the-bom-and-centroid-file-from-kicad

[antonysigma]

Here is a mock up image of what I was trying to achieve. Of course so much of the UX of the iBom would have to be disabled, I may as well add a banner "Pick-n-Place helper mode only" as a warning.

[qu1ck]

By the way, the pick-n-place / centroid (*.xy) file is so far the most widely supported format in the PCB Assembly service. Perhaps one day they will favor Speectra or KiCad, but as of today the centroid file format remains the industry practice.

They likely won't because specctra files and pnp files have different purpose. First describes the whole board, including component shapes, netlist, tracks, etc. Second is just an input to pnp machine which only cares about which value and package goes where and at what angle.

For ibom .xy file just doesn't have enough information. Being a tool designed as a visual aid for manual assembly it benefits from detailed drawing of the board. Silkscreen, pad locations and bounding box highlighting is the major part of that. Then there are tracks/zones and related debug functionality but it's secondary.

Having a raster image in the background is not bad but absence of proper component bounding boxes will take away a lot from UX. This prevents reverse lookups too (clicking on board finds the component under cursor and highlights the corresponding bom row).

[antonysigma]

After spending a week to review the specctra file, I have to admit defeat for the complexity of the format. I will open a separate Github issue in case I have more free time.

A few DevOps questions:

• I plan to avoid reinventing the wheels by importing existing Specctra parser library. What is your criteria on accepting external dependencies in your project?

• Similarly, do you permit a dependency on a Python-binder interface of a C/C++ library (via Pybind11)?

  • Any coding style I should be aware of? Could you provide a YAPF style config in the project?

In the meantime, I am almost finished tweaking the project to parse the generic centroid file format. I know it breaks a lot of UX and it is unlikely to be merged upstream, but still I would like to hear your comments on it: https://github.com/openscopeproject/InteractiveHtmlBom/pull/191

[qu1ck]

I plan to avoid reinventing the wheels by importing existing Specctra parser library. What is your criteria on accepting external dependencies in your project? Similarly, do you permit a dependency on a Python-binder interface of a C/C++ library (via Pybind11)?

My main criteria are: maintaining support on all 3 platforms (win/linux/mac), keeping ibom installation easy, and complying with licensing requirements of pulled dependencies.

To expand: having a binding to c/c++ would make cross platform support such a hell that it's not worth it at all. Really, it's easier to rewrite whatever c++ does in python. Keeping ibom easy to install means ideally having everything packaged in the project itself. For anything that is in the "critical path" for this plugin to work in KiCad this is a hard requirement. For everything else like support for other CADs I can give a bit of leeway in form of requiring to run "pip install X". But if installing the dependency is more involved than that, then probably it's a no go. Licensing requirements are mostly only relevant if the dependency is redistributed with the project. GPL or MIT code is fine, something more restrictive must be reviewed on a case by case basis.

Any coding style I should be aware of? Could you provide a YAPF style config in the project?

For now I'm happy with what intellij/pycharm autoformatter with default settings yields. As long as the code is pep8 compliant, has sensible method and variable naming, and is consistent it should be fine. This project doesn't have that large volume of merge requests from outside contributors that formatting would become an issue.

I have to admit defeat for the complexity of the format. I will open a separate Github issue in case I have more free time.

It's totally fine if you don't want to tackle this at the moment. I just filed #192 to keep track of this, I'll probably look into it later as well.

Few tips if you do find yourself working on it:

• You don't need a full featured parser. Only enough to generate the pcbdata.
• It's ok to cut some extra functionality like zone, netlist support. Focus on bom table, board outline, silkscreen, and component pads.
• You can use open source parser implementations as reference. freerouting (java) has one.