fomu-hardware git directory/branch structure

[ewenmcneill]

On 2019-06-25 on #tomu on IRC, @xobs wrote:

I need to collapse all the branches down to one and make subdirectories. Possibly throwing away history? Or maybe keep the history in the other branches...

The easiest solution is probably to reattach each branch into its own directory in the master branch, creating a new set of git commits that mirror (but aren't the same commits) each branch.

In the past I've done that with git-filter-branch to rewrite the branches into (or out of) a subdirectory, and then merging that rewritten branch onto master (the two branches likely have no common ancestor at that point, but git will still merge that).

If you can describe the "existing branch" to "new location on single shared branch" transformation that you want to achieve, it should be possible to turn that into an appropriate set of git-filter-branch commands and git-merge commands to achieve the result without throwing away the history. (In particular "I want to push everything in this branch down one or two directory levels" is one of the easier "rewrite history" transformations...)

[xobs]

The problem with that is that we would lose tags, as each release of hardware has a separate tag attached to it. One nice property of the current approach is that you can do "git checkout evt3" and get the state of the repository when that was checked in.

[ewenmcneill]

@xobs: I'm unclear on what you want here. Do you (a) want to reassociate the tags with the new commits on the shared branch, (b) keep the old branches, or (c) not rearrange anything because it'll break some other external dependencies.

FWIW, it's perfectly possible to keep the existing branches (and their tags, if you wish), and just stop adding to them. It's also possible to move the tags to somewhere more relevant in the new structure. (But it's not possible to have them in both places :-) ) If the branches are re-applied on top of a central branch (master?) in a sensible order, the moved tags will even be fairly sensibly ordered.

Anyway my point of creating this ticket is that the mechanical part of rearranging things, retaining history is relatively simple. Deciding what you want the new structure to look like, down to the level of "we could write a program to transform this" is complicated.

If you've decided you don't want to rearrange anything after all, because it'll break things, feel free to close this issue :-)

Ewen

[ewenmcneill]

Also FWIW it's literally impossible to change anything in a git repository history without creating new commits. By design. Because it's a chain of hashes. So you either live with what you have, or you make new commits. Just saying.

Ewen

[xobs]

The answer to every problem is "blockchain!"

Users will generally encounter one of three boards: EVT, PVT, and Hacker. I'd like master to have a tree that looks something like:

+- gerber
+----- evt1
+----- hacker
+----- pvt1
+- kicad
+----- evt1
+----- hacker
+----- pvt1
+- reference
+--- schematics for boards and PDF files for components
+- case

Then the full history will be in the various branches.

This gives something handy to link to, in addition to making it more obvious where things are.

[ewenmcneill]

What happens with evt2 and evt3 in that structure? And later production versions? Are they more parallel subdirectories? Or are they later commits on the "evt1" directory? Other than that question, and what happens with the tags (moved, to where; left on branch), it looks a fairly simple transformation to me.

Ewen

PS: I was trying to avoid using the "B" word :-)

[xobs]

In this structure, "evt1", "evt2", etc. are all still in their respective branches, but schematics are pulled out into the reference directory as PDF files, and gerber files are pulled to the gerber directory as parallel directories.

The alternative would be to just include the zip files that I send to the production house, but that's harder to link to.

[ewenmcneill]

So https://github.com/im-tomu/fomu-hardware/tree/evt/hardware/pcb becomes kicad/evt1? And there's only evt1 even though the EVT went through four revisions? (And it's "evt1" not "evt"?)

And:

https://github.com/im-tomu/fomu-hardware/tree/evt/hardware/releases/evt1 https://github.com/im-tomu/fomu-hardware/tree/evt/hardware/releases/evt2 https://github.com/im-tomu/fomu-hardware/tree/evt/hardware/releases/evt3 https://github.com/im-tomu/fomu-hardware/tree/evt/hardware/releases/evt3b

become:

1. gerber/evt1, gerber/evt2, gerber/evt3, gerber/evt3b; or

2. gerber/evt1, gerber/evt1, gerber/evt1, gerber/evt1 commits over the top of each other, with tags to reference which one?

If we can get to the point of a CSV file that has "existing_path,copy_to", then this is automatable, including retaining all history (in new commits) over the transformation. But I'm still not quite clear on the transformation required. (And we can leave the old branches just as they are, or remove them, as makes sense for each one.)

Ewen

PS: FTR, because git is a content store (ie, stored by hash of the content), having multiple copies of a given file in multiple locations / multiple branches doesn't take additional space in the git repo itself. They're all effectively symlinks. (It only potentially takes more space on checkout of a branch with multiple references to the same file.)

[xobs]

Option (1) - the releases go into gerber/evt1, gerber/evt2, gerber/evt3b, etc.

With all of the schematic PDFs going either into reference or into gerber -- I'm not sure which would make more sense.

[mithro]

Please take a look at https://github.com/mithro/fomu-hardware/tree/master/archive -- this is my proposal for how to structure it....