Discussion of branching style

[masinter]

The original note was moved to a new issue #574

One of the interactions between Medley and Git is the handling of deleting a file from the repo.

If I have on my local disk FOO FOO.~1~ and FOO.~2~ and with some git action that updates FOO (checkout of another branch), it recovers OK : git will break the hard link between FOO and FOO.~2~ and Lisp will see FOO;3.

But things aren't so good if the checkout deletes FOO because FOO.~1~ and FOO.~2~ are still around. Lisp will see INFILEP(FOO) finding FOO:2.

At the very least, a github "hook" could warn the user when the checkout deleted a file while there are ~nn~ versions.

[masinter]

Moved to #574

[masinter]

It might be easier if we used the {UNIX} device to avoid the DWIM of the {DSK} device compensating quietly but imperfectly. E.g., if I delete file FOO then INFILEP(FOO) should be NIL.

Another way to do this would be to change maiko DSK code to Ignore versions that don't correspond to unversioned files. Fix up the hard links for files that are same content and make a new versioned file if FOO and FOO.~max~ (max versions aren't same content.

[rmkaplan]

I don't want git by itself to screw around with my local file system at all. I want Medley to sit on top of git and be able to see files in branches and commitments, compare them intelligently, copy files or definitions that look good, etc. And let it do what it automatically does, maintain my local version history.

Separately, why do we fool around with all the overhead of branches? Trying to name them and describe them as if somehow they had conceptually related changes rather than changes that happened to emerge as work on one problem pulled a thread that opened up other issues, the typical Medley style of working in a sysout that can persist for days if not weeks. Commitments maybe (on a per-file basis, as part of CLEANUP?). But what do we get from incremental branches?

I can see why the branch structure would be good for major system-wide changes (5 byte atoms, 64 bit pointers, Unicode replacing XCCS...), but not for what evolves in the usual Medley work process.

[nbriggs]

git clean -dxn and look at what it will do, if you're happy with that, git clean -dxf -- such as:

silversword% cd medley
~/Projects/medley
silversword% git clean -dxn
Would remove library/EXPORTS.ALL
Would remove library/UNIXTELNET.LCOM.~1~
Would remove library/UNIXTELNET.LCOM.~2~
Would remove library/UNIXTELNET.~1~
Would remove library/UNIXTELNET.~2~
Would remove library/VTCHAT.LCOM.~2~
Would remove library/VTCHAT.LCOM.~3~
Would remove library/VTCHAT.~2~
Would remove library/VTCHAT.~3~
Would remove library/VTCHAT.~4~
Would remove library/WHEREIS.HASH
Would remove lispusers/DATE~
Would remove lispusers/LIFE.DFASL
Would remove loadups/init.dribble
Would remove scripts/lsee~
Would remove sources/LLSUBRS.LCOM.~1~
Would remove sources/LLSUBRS.LCOM.~2~
Would remove sources/LLSUBRS.LCOM.~3~
Would remove sources/LLSUBRS.LCOM.~6~
Would remove sources/LLSUBRS.LCOM.~7~
Would remove sources/LLSUBRS.~1~
Would remove sources/LLSUBRS.~2~
Would remove sources/LLSUBRS.~3~
Would remove sources/LLSUBRS.~6~
Would remove sources/LLSUBRS.~7~
Would remove sources/LLSUBRS.~8~
Would remove tmp/

[nbriggs]

This is why I suggested not using the repository as your working directory.

[rmkaplan]

I’m trying not to. I do all my work now in a separate unpublished branch. But then everything gets out of step in a different way—figuring out what and when to copy, in either direction.

On Oct 19, 2021, at 5:04 PM, Nick Briggs @.***> wrote:

This is why I suggested not using the repository as your working directory.

— You are receiving this because you were assigned. Reply to this email directly, view it on GitHub https://github.com/Interlisp/medley/issues/533#issuecomment-947195634, or unsubscribe https://github.com/notifications/unsubscribe-auth/AQSTUJLQU4W3DV5L2EZXDRDUHYBPNANCNFSM5GKE5VMA.

[nbriggs]

I mean not in a branch, not in a directory that git knows anything about... copying the files from your workspace to the git repo directory (on a branch, or not, whatever) by hand when you're all done and want to release them to the world. Pretend git doesn't exist in your regular workflow right up until you want to release things.

[rmkaplan]

And copy changed files when I notice that something might have been updated from git? Manual processes so as not to get screwed by all this super automation. But you are right, at least then git won’t mangle because of conflicts and deletions.

The lightweight thing would be not to have to deal with new branches everytime, just upload a bunch of files to the same place as a single new commitment, whenever it seems exporting for other people to synchronize with.

On Oct 19, 2021, at 8:47 PM, Nick Briggs @.***> wrote:

I mean not in a branch, not in a directory that git knows anything about... copying the files from your workspace to the git repo directory (on a branch, or not, whatever) by hand when you're all done and want to release them to the world. Pretend git doesn't exist in your regular workflow right up until you want to release things.

— You are receiving this because you were assigned. Reply to this email directly, view it on GitHub https://github.com/Interlisp/medley/issues/533#issuecomment-947299820, or unsubscribe https://github.com/notifications/unsubscribe-auth/AQSTUJOMEXM4VF2UZLX2RHDUHY3U3ANCNFSM5GKE5VMA.

[nbriggs]

A branch (for doing a github pull-request) is ephemeral -- the branch will go away once it has been merged, so the name is not important. You could use something formulaic like kaplan-fixes-YYMMDDNN. The descriptions of the changes that you commit are important, those last forever and are the documentation for people following behind to have some idea of what you did and why you did it. There's default text in the PR, from the commit message(s) I guess, but that is also ephemeral (PRs are not a "git" thing, they're a "github" thing and not recorded in the git history)

[masinter]

In the meanwhile, I think the enhancement I want is something like adding a script .git/hooks/post-checkout which scans the repo for version files with no corresponding unversioned

find . -name ".~[1-9]~" -exec warnif {} \;

where warnif checks if (presumably after the checkout action, there wasn't an unnumbered corresponding file. Fancier would be to consolidate the warnings.

I think we should continue to look for ways to make the integration of git and Interlisp.

The GitHub desktop isn't well-done and it can get you into bad states. I think a medley Git client would be much more valuable

[rmkaplan]

On Oct 19, 2021, at 8:47 PM, Nick Briggs @.***> wrote: I mean not in a branch, not in a directory that git knows anything about... copying the files from your workspace to the git repo directory (on a branch, or not, whatever) by hand when you're all done and want to release them to the world. Pretend git doesn't exist in your regular workflow right up until you want to release things.

Maybe this is the least dangerous and simplest way of side-stepping the git problems.

I will clone a new repository, and essentially just treat it as a local image of a remote file server. I’ll then remove all of the .git stuff I can find in my current working branch, so that its connection to the repository and to git operations and conventions is severed. Just like an old Medley DSK, version numbers, no hard-link confusions, etc. (I’ll also delete all of the inactive/archival files from my working directory, since that stuff is unchanging and will always be available in the clone.)

From time to time I’ll do a COMPAREDIRECTORIES of my working directory with the master branch of the clone, use COMPARESOURCES to decide whether to merge files (or maybe just extract partciular definitions, if I have gotten ahead of things), and then do COPYFILES or LOADDEFa to copy over things that are new.

In the other direction, I’ll run COMPAREDIRECTORIES from time to time, to determine what working files have diverged from the Master branch. Then copy the subset of changes that it makes sense to upload into Kaplan-n branches in the clone, construct commits and PR's that try to reflect the useful trajectory of my working history (probably at a somewhat coarse grain-size).

The important point is that I’ll have explicit control inside Medley, git will never screw with my working directory, so it won't get smashed up or mixed up with (un)stashes and text-based conflicts.

You are receiving this because you were assigned. Reply to this email directly, view it on GitHub https://github.com/Interlisp/medley/issues/533#issuecomment-947299820, or unsubscribe https://github.com/notifications/unsubscribe-auth/AQSTUJOMEXM4VF2UZLX2RHDUHY3U3ANCNFSM5GKE5VMA.

[masinter]

this issue morphed into a discussion of #536 while the original "want" was just a simple hook.

[rmkaplan]

I advanced a bit towards a workable configuration: I copied by working clone to someplace else and removed its .git files, then recloned a fresh git. From inside medley I have it set up so that the glt clone is accessed through a new variable GMEDLEYDIR, which is a {UNIX} directory without medley version numbers. I will treat that mostly as a glorified dropbox that happens to have funny upload and download conventions--and that requires running FIXEDITDATE from time to time, so new downloaded files sort to the top.

I also tried to eliminate some of the git-rot in my working {dsk}-medley directory, files with early filecreated dates but later version numbers. I recovered a certain number of lost edits, but I'm sure that more will show up over time.

I will now use only medley tools (COMPARESOURCES, COMPAREDIRECTORIES, COPYFILES...) to move stuff back and forth between the clone and my working configuration. I'll only pull the Master branch of the clone, and only push new branches that contain whatever files I have copied over.

I think this will be a much safer arrangement.

[masinter]

This discussion evolved from a simple feature request to a broader discussion of branching methodology. But in the meanwhile it lost the original request. I'll open a new issue for it.

[masinter]

Maybe it would help to talk about the difference between {DSK} and {UNIX}. Whenever you git pull a branch that deletes or renames something, it can leave behind ~nn~ residue

so even though I deleted the "undecorated" file, the {DSK} device will show it's still there.