Improve interface, plot display, and textual diff

[Gasman2014]

The current paradigm relies on selection of commits ante hoc and the script has to be rerun from scratch to compare different revisions.

A more useful real world scenario would be to review the sequential changes between arbitrary commits, the whole interface being delivered within a web-browser. It would probably be easiest to use a git (or fossil) commit-hook to generate the artefacts after every commit so the artefacts were already embedded in the repo. This does slightly go against the philosophy that the board is 'source code' and the images are 'compiled' so should not pollute the repo but would simplify & speed up the plotting process.

Options to select multiple layers would be helpful. The DOM would have to be redesigned to support javascript selection of commits/layers and to explore the actual diff changes.

The initial launch script could be an 'Action script' from within pcbnew.

Ultimately, it would be very useful to support schematic diffs (programatic plotting of schematics may be available in KiCad 6) and to be able to launch this from within the Project Manager window.

A very rough initial mockup --->

[leoheck]

See you have now fixed the legend too; something I had meant to do too. Oh, good, my bad hehe.

It is really cool to be able to scroll through both sequential and arbitrary commits and all the layers! Looking really good on Firefox but images still don’t seem to load in Safari. I’ll take a look tomorrow.

Indeed, it is pretty cool to be able to review all these images pretty fast going back and forth easily.

Please, check Safari development tools, especially a console to look for javascript issues so we can investigate this better. I found this: If you don’t see the Develop menu in the menu bar, choose Safari > Preferences, click Advanced, then select “Show Develop menu in the menu bar.”

I recently started to have another look at this and did think that a more meaningful diff could be generated if the layouts were optimized for diffing i.e. generate a more useful format directly from KiCad.

We could have another script that generates this info passing the commit hash, then we could insert this into the mockup page easily to evaluate how this would work.

[leoheck]

Hey John, do you how to handle when an image is missing? I would like to use a blank.svg instead of showing a big funny image. I tried to add onerror tag but it doesn't work. If you want to try, the path of this blank.svg is "../blank.svg"

[leoheck]

Found this answer now. Seth has answered my question. Do you have any idea on how to start by checking if we can export sch as .svg?

[leoheck]

Hey @Gasman2014 I added a -w flag so you can open it with another viewer. You can set Firefox by doing this -w firefox It is -w because -v was not working, haha. What a strange thing.

I think gitplotsch is messing with the repository. For some reason, it goes back to master branch always. Do you have the same issue? Could you try to run this tool on a different branch, and check if it goes back to master after the execution?

[Gasman2014]

Don’t know very much (i.e. zero!) about Opam but do you need to ‘pin’ a version?

opam pin

https://opam.ocaml.org/doc/man/opam-pin.html

On 26 Mar 2021, at 13:38, Leandro Heck @.***> wrote:

Hey @Gasman2014 https://github.com/Gasman2014 I added a -w flag so you can open it with another viewer. You can set Firefox by doing this -w firefox It is -w because -v was not working, haha. What a strange thing.

I think gitplotsch is messing with the repository. For some reason, it goes back to master branch always. Do you have the same issue? Could you try to run this tool on a different branch, and check if it goes back to master after the execution?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Gasman2014/KiCad-Diff/issues/36#issuecomment-808226689, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACADFOPBAOXEO4MXDB2XJILTFSE35ANCNFSM4XYNIEMA.

[leoheck]

This pin-thing is for installing/testing. Opam/Opam-language are like hieroglyphs for me. This is so hard. Also, the name of plotgitsch is also hard for me to even remember haha.

I was talking more about when I am using plotgitsch inside this kdiff script, something is not good. And makes the Kicad project move to the master branch. This is not a big issue, but I still have to investigate this because annoys me on my tests.

Ah. Now I have a better image refreshing method. Can you try it with Safari? It also improves the way it handles the schematic pages, so you can also test with a multi schematic project.

[leoheck]

@Gasman2014, when you have time, could you check the last changes on kdiff? It is more stable now. It may work better on safari too. I am also interested to check if I haven't added extra bugs on OSX.

• I am moving the layers/pages list to a dropdown list, so they won't use too much space. Then I will try to add kidiff textual diffs as in your original mockup.

[leoheck]

Added a demo video here, https://www.youtube.com/watch?v=PMC0USSsbDE&t=7s

[jnavila]

Hi @leoheck , have you had a look at how it is done on wikipedia? for instance: https://en.wikipedia.org/w/index.php?title=Apple_Inc.&action=history

[leoheck]

Hi, @jnavila I haven't seen this before. This is interesting. I believe you are referring to the way commits are selected, and not how they are displayed. Am I right?

Have you tried the current version of this mockup? Do you think the commits list is not being displayed in a usable way? The idea was not to use the mouse to select commits. The idea is to use the arrow keys to move back and forth. Have you tried to use keyboard shortcuts? On the demo video was changing commits using the mouse sometimes to help the viewer to move the focus to the place I wanted to show, but it is not required.

[jnavila]

The video is cool. The only thing is that you can only select adjacent commits. You cannot compare distant tags or even branches.

[Gasman2014]

You can actually compare ANY arbitrary commits using this interface.

John Pateman

Sent from my iPhone

On 1 Apr 2021, at 20:57, Jean-Noël Avila @.***> wrote:

 The video is cool. The only thing is that you can only select adjacent commits. You cannot compare distant tags or even branches.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

[leoheck]

I see.

Kdiff generates svgs for each commit before displaying the website. By default it excludes commits that do not modify the .kicad_pcb or the .sch files. It also gets all the commits from the current commit until the first commits.

To be able to compare everything as you said @jnavila it will need to generate these artifacts for all the commits or it needs to generate these things on the fly. I haven't tried generate something on the fly. But I think it may make the review process slower. But it can be improved if we have smaller and faster tools to generate svgs for schematics and layout.

Now getting back to the way wikipedia displays commits. I still think it is pretty much the same the way I am doing, except that on wikipedia we have one less step when using the mouse or the trackpad, because you don't need to disselect before selecting something new.

Ok, I will try to make something that displays commits the way wikipedia does and then I will run these tools on the fly to check. Let's see if I can do something like that.

[leoheck]

Ah, @jnavila on kdiff you don't have to select adjacent commits only. You can select freelly on the list using the mouse. You can also move adjacents easily by using arrows up/down and you can move the second selection back and forth using these characters [ and ]

[leoheck]

I was looking at this, it looks like javascript cannot execute an external command.

[leoheck]

Hey, @Gasman2014 would you like to check the current version of the kdiff?

It is better now. It will work even with Plotgitsch or Kicad-Diff missing but it is better to have both of course. It is working on Windows too, I have some instructions in the README.md

I would like to check with you if it is still working on OSx but I believe it is still working.

If you have time for this. You have to upgrade kdiff and my version of plotgitsh since it is a little bit tweaked. Inside my version of plotgitsh you will find a install.sh that you can use to upgrade it easily.

Then, to run the tool you have to pass the project.pro instead of the project.kicad_pcb which differs from Kicad-Diff For instance

kdiff project.pro -V -r

Jnavila is working to make the plotgitsch more resilient when a component is missing, generating an image of the schematic even without that specific part. 90% of an image is always better than nothing, right? So, instead of breaking it, it will generate the image without the missing part, if the user sets a flag for that, of course.

[Gasman2014]

I’ll give it a try tonight.

John Pateman

Sent from my iPhone

On 19 May 2021, at 23:53, Leandro Heck @.***> wrote:

 Hey, @Gasman2014 would you like to check the current version of the kdiff?

It is better now. It will work even with Plotgitsch or Kicad-Diff missing but it is better to have both of course. It is working on Windows too, I have some instructions in the README.md

I would like to check with you if it is still working on OSx but I believe it is still working.

If you have time for this. You have to upgrade kdiff and my version of plotgitsh since it is a little bit tweaked. Inside my version of plotgitsh you will find a install.sh that you can use to upgrade it easily.

Then, to run the tool you have to pass the project.pro instead of the project.kicad_pcb which differs from Kicad-Diff For instance

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

[Gasman2014]

Im having some trouble with your new kdiff script. I’m passing the path to the project reference ie.

kdiff /full/path/to/myboard.pro -r -V

but the script fails silently after maybe half a second - no output at all. I will try and debug when I get some time but there is no error even if I pass a non-existent board reference which is clearly wrong.

I am on the latest checkout.

% git log commit 0a55c42bb831267e35d7c10ed9391007f9d9c6f0 (HEAD -> main, origin/main, origin/HEAD) Author: Leandro Heck @.***> Date: Fri May 21 20:55:03 2021 -0300

Update help, kdiff uses .pro instead of .kicad_pcb file now

On 20 May 2021, at 06:20, John Pateman @.***> wrote:

I’ll give it a try tonight.

John Pateman

Sent from my iPhone

On 19 May 2021, at 23:53, Leandro Heck @.***> wrote:



Hey, @Gasman2014 https://github.com/Gasman2014 would you like to check the current version of the kdiff?

It is better now. It will work even with Plotgitsch or Kicad-Diff missing but it is better to have both of course. It is working on Windows too, I have some instructions in the README.md

I would like to check with you if it is still working on OSx but I believe it is still working.

If you have time for this. You have to upgrade kdiff and my version of plotgitsh since it is a little bit tweaked. Inside my version of plotgitsh you will find a install.sh that you can use to upgrade it easily.

Then, to run the tool you have to pass the project.pro instead of the project.kicad_pcb which differs from Kicad-Diff For instance

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Gasman2014/KiCad-Diff/issues/36#issuecomment-844550174, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACADFOO2LJYND3OC6I4QAQLTOQ6NLANCNFSM4XYNIEMA.

[leoheck]

Ok, I changed the behavior of some tests to check if tools are present in the PATH and this was not right. Now it is fixed. I was also able to run it on a friend's OSx. It just doesn't display images on Safari, yet. But it works on chrome and firefox.

You will have to upgrade plotgitsh too. I had to fix paths on git-imgdiff script there too.

[leoheck]

@Gasman2014 instructions for running it in OSX were updated here. I tested it and it worked. Feel free to experiment one more time, please. https://github.com/leoheck/kdiff#osx-dependencies

[Gasman2014]

This is looking really good! I can confirm that I have it working in macOS incl rendering on Safari (which had previously failed). So good to be able to scroll through commits so easily!

Few initial observations;

Some of the rendering is relatively slow and even with the -V verbose flag, you don’t get any feedback during the ‘- Updating html’ processes.

How about an 'Open File …' selector if no board passed?

Are we selecting the pcb or the project - both seem to work but in the context of schematics & layout, choosing the .pro makes more sense. (.kicad_pcb or .pro)? The Read.Me is slightly unclear (You say .kicad_pcb in the ‘Using’ section and .pro in the Command line help.

On 4 Jun 2021, at 21:52, Leandro Heck @.***> wrote:

@Gasman2014 https://github.com/Gasman2014 instructions for running it in OSX were updated here. I tested it and it worked. Feel free to experiment one more time, please. https://github.com/leoheck/kdiff#osx-dependencies https://github.com/leoheck/kdiff#osx-dependencies — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Gasman2014/KiCad-Diff/issues/36#issuecomment-854991432, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACADFONLQ2MIDKRNXIYBS4LTRE4HRANCNFSM4XYNIEMA.

[leoheck]

Pretty nice, I appreciate our feedback.

So good to be able to scroll through commits so easily!

Isn't it? I am proud of it too. I am looking for more useful features. Maybe it is time to add the textual diff that you mention before. I am also thinking to be able to launch kicad by clicking the version of the commit. I just haven't try to launch external tools from there yet.

Some of the rendering is relatively slow and even with the -V verbose flag, you don’t get any feedback during the ‘- Updating html’ processes.

Totally, I tried to improve this but I could not do it yet. I could not use sed for substitution due to some issues scaping HTML code. I wanted to keep it simples. And then I'm using Bash's variable pattern substitution for that. It is pretty fast on Linux, or at least on my super-powerful computer. But it is really slow on OSx as I could see testing. Thanks for giving me this feedback, I will think of a way to improve this.

How about an 'Open File …' selector if no board passed?

It can be done but I guess this will add extra dependencies. And I can also ask this in the command line, but this is not what you want right?

Are we selecting the pcb or the project - both seem to work but in the context of schematics & layout, choosing the .pro makes more sense. (.kicad_pcb or .pro)? The Read.Me is slightly unclear (You say .kicad_pcb in the ‘Using’ section and .pro in the Command line help.

It was *.kicad_pcb then I replaced with the *.pro file. But now you have mentioned it, it will give the same results since I am just removing the extension and .kicad_pcb when I need the board but I will improve the readme, accordingly.

[leoheck]

I added a GUI file selection to kdiff. I am using zenity for that. You will need brew install zenity. To use it, just run kdiff without the project file and it will launch the file selector. It also doesn't need to be inside the repo anymore.

[leoheck]

I was able to update the text substitution using sed now. It looks faster now even on an old MacBook air. Let me know if you see any issues.

[Gasman2014]

Wow! When you say faster, you really meant was at least two orders of magnitude faster !!! Much, much, much better!

Thats a real success - it did worry me that the file generation was a bit slow and I was considering whether we should track the presence of existing files as a form of caching but with the speed bump its really isn’t necessary.

I’ve also tried out the Zenity file selector - this works very nicely and suits my general working pattern. I presume it fails gracefully if Zenity is not found? The only downside is the ridiculous list of dependencies that it seems to bring in - including gnuplot and gtk3 but I can live with that.

Great work!

John

On 9 Jun 2021, at 03:06, Leandro Heck @.***> wrote:

I was able to update the text substitution using sed now. It looks faster now even on an old MacBook air. Let me know if you see any issues.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Gasman2014/KiCad-Diff/issues/36#issuecomment-857314525, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACADFOMRKJR3PH53E4MPI4TTR3EC7ANCNFSM4XYNIEMA.

[leoheck]

Sure, if zenity is missing it will just exit without too much noise like a gentleman. haha

Yeah, it is not good that it needs to download the whole world to give us a GUI. Do you know any better zenity replacement for OSX? I mean, maybe something native. I am using zenity because it was easier for me.

Ah, this GUI thing gave an idea to integrate it with the Kicad toolbar. It is working there already if you pull the repo one more time. It takes time to give some feedback to see user because I am just calling kdiff directly without much effort. But this is the normal time that kdiff takes to generate everything. But it looks promising. Check how to install it here https://github.com/leoheck/kdiff#kicad-integration The only drawback is that kdiff is currently now considering the current local changes to generate diffs, only the git history. But I will try to include this.

And again. Thanks for testing it and for the feedback.

[leoheck]

I realized that I could use tk to create a file picker. It is already on kdiff repo. If you remove zenity it will use my file_picker.py. It is not beautiful but it works. It needs tk but Kicad-Diff also needs it.