Tracking downstream Win32-OpenSSH changes with upstream release tags/branches

[awakecoding]

I am working on an OpenSSH distribution based on Win32-OpenSSH on Windows, and "vanilla" OpenSSH on non-Windows. The problem that I have right now is that I cannot track downstream changes to their original upstream release tags or branches. Right now, there are three repositories:

Win32-OpenSSH, which doesn't contain the openssh-portable source code, but is used for release tags and issue tracking. While the release tags contain source code along side binaries, it does not provide sufficient information to track the changes in the git history.

PowerShell/openssh-portable contains the Win32-OpenSSH downstream changes, but it doesn't have the same tags and branches as the upstream openssh-portable repository.

All changes in the downstream repository appear to be made in a "latestw_all" branch, with no clear history to track how the Win32-OpenSSH changes are rebased on top of which upstream openssh-portable release tag.

Can someone explain to me how a Win32-OpenSSH release like v8.9.1.0 was made such that I could make a diff of the downstream sources against their corresponding upstream sources, and clearly see how the source code was modified? Ideally, I would like to separate the Win32 patches to make them easier to reapply on top of newer versions of OpenSSH, like 9.0 that just came out.

My goal is to have my own cross-platform OpenSSH distribution that correctly incorporates the Win32-OpenSSH changes. I know it is not a goal of Win32-OpenSSH to build and support non-Windows, but just working toward better tracking of downstream changes could help making them much easier to reapply on top of future upstream releases, and reduce the overall maintenance effort required to keep it up to date.

[awakecoding]

@bagajjal @SteveL-MSFT

[bagajjal]

@awakecoding - You can get the commit id associated with the upstream tag. Later you can do a git checkout < commit_id >. Getting upstream tags (147) clutters our repository.

[awakecoding]

@bagajjal I manually imported the V_8_9_P1 upstream tag that should correspond to the downstream v8.9.0.0 tag on my GitHub fork to simplify operations. The downstream changes don't appear to be rebased, but the result of years of "git merge" with upstream, making the history quite messy. The GitHub comparison shows 506 commits, a long a which much older than the upstream tag: https://github.com/awakecoding/openssh-portable/compare/V_8_9_P1...v8.9.0.0

I tried all sorts of ways to crate a usable diff or patch file with git using those two tags, but I failed to get anything that would work with either the Linux patch command or "git apply". Do you have a working procedure to produce a single patch out of all changes such that they can be re-applied on top of the corresponding upstream tag?

I finally gave up doing it with git commands, and did it the hard way on Linux using the diff and patch tools a copy of the upstream and downstream sources in which I have removed the .git directory to avoid it being included in the patch file. Here's my procedure for V_8_9_P1 (upstream) and v8.9.0.0 (downstream):

Clone upstream and downstream repositories, remove .git directory in both:

mkdir /tmp/git && cd /tmp/git
git clone https://github.com/awakecoding/openssh-portable openssh-upstream
cd openssh-upstream && git checkout V_8_9_P1 && rm -rf .git/ && cd ..
git clone https://github.com/awakecoding/openssh-portable openssh-downstream
cd openssh-downstream && git checkout v8.9.0.0 && rm -rf .git/ && cd ..

Generate single patch from upstream and downstream directories:

diff -ruN openssh-upstream/ openssh-downstream/ > Win32-OpenSSH.patch

Clone upstream repository again, create branch + apply patches on top:

git clone https://github.com/awakecoding/openssh-portable openssh-upstream
cd openssh-upstream && git checkout V_8_9_P1
git branch Win32-V_8_9_P1 && git checkout Win32-V_8_9_P1
patch -s -p1 < ../Win32-OpenSSH.patch

Create new commits containing the Win32-OpenSSH changes on top of the upstream tag, separating new files from modified files to see which changes are isolated more easily:

git add $(git ls-files -o --exclude-standard)
git commit -m "Win32-OpenSSH new files"
git add -A
git commit -m "Win32-OpenSSH modified files"

The end result, when pushed on a branch, is the end result I was looking for, but ideally with better tracking through the git history. The technique I've used is the last resort because I couldn't find a proper way to do this with git commands.

Here's the cleaner reconstructed history for the resulting branch compared against the upstream tag it is based on:

https://github.com/awakecoding/openssh-portable/compare/V_8_9_P1...Win32-V_8_9_P1