Running under PowerShell fails to invoke `git` in hooks

AaronLieberman commented 2 years ago

Description of the bug

Moving commit generates a warning and leaves the old commit in place. I'm on Windows 11, in a PowerShell terminal.

⋮
◇ f148352 13d (remote origin/main) Fix up renamed csproj
┃
◯ 777ac1f 8s file1
┃
● 9c3f752 1s file2
D:\Projects\Git\ExpressionTree > git move -s HEAD -d f148352
Attempting rebase in-memory...
[1/1] Committed as: 62e3191 file2
/d/Projects/Git/ExpressionTree/.git/hooks/reference-transaction: line 6: git: command not found
branchless: Failed to process reference transaction!
branchless: Some events (e.g. branch updates) may have been lost.
branchless: This is a bug. Please report it.
/d/Projects/Git/ExpressionTree/.git/hooks/post-rewrite: line 4: git: command not found
branchless: running command: git checkout 62e3191f2ff4bae240bf82e4f9e2532b75b9b43a
Previous HEAD position was 9c3f752 file2
branchless: processing 1 update: ref HEAD
HEAD is now at 62e3191 file2
branchless: processing checkout
branchless: running command: git branchless smartlog
:
O f148352 13d (remote origin/main) Fix up renamed csproj
|\
| o 777ac1f 15s file1
| |
| o 9c3f752 8s file2
|
@ 62e3191 1s file2
In-memory rebase succeeded.

Expected behavior

I expected the HEAD commit to be moved to be a child of origin/main.

Actual behavior

Warning generated about not being able to find git.
Commit cherry-picked to the correct location but original commits are still visible.

Version of `git-branchless`

git-branchless 0.3.12

Version of `git`

git version 2.35.1.windows.2

Version of `rustc`

rustc 1.57.0 (f1edd0429 2021-11-29)

Automated bug report

Software version

git-branchless 0.3.12

Operating system

Windows 6.2.9200

Command-line

git-branchless bug-report

Environment variables

SHELL=<not set>
EDITOR=<not set>

Git version

> git version 
git version 2.35.1.windows.2

Events

Show 5 events

##### Event ID: 151, transaction ID: 288 1. `RefUpdateEvent { timestamp: 1650762474.984679, event_tx_id: EventTransactionId(288), ref_name: "HEAD", old_oid: 0000000000000000000000000000000000000000, new_oid: 62e3191f2ff4bae240bf82e4f9e2532b75b9b43a, message: None }` 1. `RefUpdateEvent { timestamp: 1650762475.0473373, event_tx_id: EventTransactionId(288), ref_name: "HEAD", old_oid: 9c3f7524633c5c38a57c47ec51d91b768557aa5a, new_oid: 62e3191f2ff4bae240bf82e4f9e2532b75b9b43a, message: None }` ``` : O f148352 13d (remote origin/main) xxx xx xxxxxxx xxxxxx |\ | o 777ac1f 5m xxxxx | | | o 9c3f752 5m xxxxx | @ 62e3191 5m xxxxx ``` ##### Event ID: 149, transaction ID: 286 1. `RefUpdateEvent { timestamp: 1650762467.3501792, event_tx_id: EventTransactionId(286), ref_name: "HEAD", old_oid: 777ac1f503f886aa09ea66499a7a993642b726ed, new_oid: 9c3f7524633c5c38a57c47ec51d91b768557aa5a, message: None }` 1. `CommitEvent { timestamp: 1650762467.0, event_tx_id: EventTransactionId(286), commit_oid: NonZeroOid(9c3f7524633c5c38a57c47ec51d91b768557aa5a) }` ``` : O f148352 13d (remote origin/main) xxx xx xxxxxxx xxxxxx |\ | o 777ac1f 5m xxxxx | | | o 9c3f752 5m xxxxx | @ 62e3191 5m xxxxx ``` ##### Event ID: 147, transaction ID: 284 1. `RefUpdateEvent { timestamp: 1650762460.6356707, event_tx_id: EventTransactionId(284), ref_name: "HEAD", old_oid: f148352480d981f078b609f5a49870baa2e6d107, new_oid: 777ac1f503f886aa09ea66499a7a993642b726ed, message: None }` 1. `CommitEvent { timestamp: 1650762460.0, event_tx_id: EventTransactionId(284), commit_oid: NonZeroOid(777ac1f503f886aa09ea66499a7a993642b726ed) }` ``` : O f148352 13d (remote origin/main) xxx xx xxxxxxx xxxxxx |\ | o 777ac1f 5m xxxxx | | | o 9c3f752 5m xxxxx | @ 62e3191 5m xxxxx ``` ##### Event ID: 145, transaction ID: 280 1. `RefUpdateEvent { timestamp: 1650762410.2934084, event_tx_id: EventTransactionId(280), ref_name: "HEAD", old_oid: 0000000000000000000000000000000000000000, new_oid: f148352480d981f078b609f5a49870baa2e6d107, message: None }` 1. `RefUpdateEvent { timestamp: 1650762410.4033375, event_tx_id: EventTransactionId(280), ref_name: "HEAD", old_oid: a1d0e44057382e04c4c3b8c97e45476a531da573, new_oid: f148352480d981f078b609f5a49870baa2e6d107, message: None }` ``` : O f148352 13d (remote origin/main) xxx xx xxxxxxx xxxxxx |\ | o 777ac1f 5m xxxxx | | | o 9c3f752 5m xxxxx | @ 62e3191 5m xxxxx ``` ##### Event ID: 137, transaction ID: 278 1. `ObsoleteEvent { timestamp: 1650762402.4933422, event_tx_id: EventTransactionId(278), commit_oid: NonZeroOid(8d637802135c65b18ae4ee74b6e843ca9f65fcbd) }` 1. `ObsoleteEvent { timestamp: 1650762402.4933422, event_tx_id: EventTransactionId(278), commit_oid: NonZeroOid(0917e2432c2088cad223936d1f083fcc4b01535a) }` 1. `ObsoleteEvent { timestamp: 1650762402.4933422, event_tx_id: EventTransactionId(278), commit_oid: NonZeroOid(a2a2267cc32a5c6b0e78def64aff016235ae9c27) }` 1. `ObsoleteEvent { timestamp: 1650762402.4933422, event_tx_id: EventTransactionId(278), commit_oid: NonZeroOid(1531d1efb28c142256b81d7ca17ff3e4c5147e78) }` 1. `ObsoleteEvent { timestamp: 1650762402.4933422, event_tx_id: EventTransactionId(278), commit_oid: NonZeroOid(385c75dda30476f5301e083367a8eeecc36cd4ef) }` 1. `ObsoleteEvent { timestamp: 1650762402.4933422, event_tx_id: EventTransactionId(278), commit_oid: NonZeroOid(2a4949f6cbaec426b020f8bb17dab807dcdac7d1) }` 1. `ObsoleteEvent { timestamp: 1650762402.4933422, event_tx_id: EventTransactionId(278), commit_oid: NonZeroOid(618594b6577643ea1cece6d4a4ac0a0f59ea0e36) }` 1. `ObsoleteEvent { timestamp: 1650762402.4933422, event_tx_id: EventTransactionId(278), commit_oid: NonZeroOid(a1d0e44057382e04c4c3b8c97e45476a531da573) }` ``` : O f148352 13d (remote origin/main) xxx xx xxxxxxx xxxxxx |\ | o 777ac1f 5m xxxxx | | | o 9c3f752 5m xxxxx | @ 62e3191 5m xxxxx ```

AaronLieberman commented 2 years ago

I just set up WSL2/Ubuntu and installed git-branchless there and the same commands are working fine.

arxanas commented 2 years ago

Hi @AaronLieberman, thanks for the report. I would definitely recommend using WSL where possible, as it's the version tested in CI.

I don't have a Windows setup at the moment, so I can't test this directly. It looks like that for some reason, when hooks are invoked by Git under Powershell, it can't find the git executable. If you manually edit the hook and change git to git.exe, does it start working?

In any event, it seems kind of strange that git succeeds when you run it manually, but fails in the hook. Do you know what shell the hooks are being executed by?

AaronLieberman commented 1 year ago

Closed #717 as dup, as per suggestion.

FYI- workaround for this is to change the paths in the hooks to point to git branchless directly, e.g. C:/Users/aaron/.cargo/bin/git-branchless.exe

AaronLieberman commented 1 year ago

I did a bunch of investigation on this. Some findings:

Turns out that this is completely unrelated to PowerShell. Issue presents itself the same under cmd too. AFAIK, git branchless doesn't work under Windows, unless you use WSL, cygwin or similar.
From what I now understand, git has some special handling under Windows to notice that the git hook is trying to run under sh or bash by parsing the shebang. It then runs its own local copies of cygwin bash or sh (from C:\Program Files\Git\bin on my machine).
The problem seems to be that when it does this, the environment isn't set up right, at least on my machine. For instance, PATH isn't set. When I add an echo "--$PATH--" to my git hook and run the command, I get --/d/Projects/Git/ExpressionTree/.git/hooks--, which obviously includes less stuff than I need. i.e., no git, thus the git: command not found error.
I added PATH="$PATH:/c/Program Files/Git/cmd" to the top of my git hook and git is now found, but then while it can find git, it can't find branchless: git: 'branchless' is not a git command.
I also tried a bunch of other variations on messing with the shebang, as reported as a workaround in a lot of stack overflow posts but it didn't seem to work for me.
How does git normally find branchless? Is it a directory search into the .git directory? I would have thought it would have found it. I even added a pwd to the hook and current directory is correct in the hook too.

AaronLieberman commented 1 year ago

One of the biggest surprises for me is that when I run that shell manually & "C:\Program Files\Git\bin\sh.exe", my path is set up correctly, git is find-able, and branchless commands work without any modifications. Maybe git is incorrectly invoking the shell for the hook without inheriting an appropriate amount of the environment?

arxanas commented 1 year ago

How does git normally find branchless? Is it a directory search into the .git directory? I would have thought it would have found it. I even added a pwd to the hook and current directory is correct in the hook too.

The logic is fairly simple: when invoking git foo, it searches your PATH for an executable named git-foo or gitfoo. I can't find the logic in the Git source code right now, unfortunately.

This is all pretty strange, so maybe this is a Git on Windows bug in the end?

AaronLieberman commented 1 year ago

Definitely seems like a git on Windows bug. Just sort of sucks because it makes git branchless sort of broken on Windows. I'm moved to exclusively using git in WSL/Ubuntu so it's no longer a problem for me on my home computer. I'm starting a new job next week which is going to be on Windows and I don't really know how I'll be interacting with source control and I don't know much about how the environment is set up. Depending on how that turns out, I may dig more into what it takes to make this work properly.

c00t commented 7 months ago

Definitely seems like a git on Windows bug. Just sort of sucks because it makes git branchless sort of broken on Windows. I'm moved to exclusively using git in WSL/Ubuntu so it's no longer a problem for me on my home computer. I'm starting a new job next week which is going to be on Windows and I don't really know how I'll be interacting with source control and I don't know much about how the environment is set up. Depending on how that turns out, I may dig more into what it takes to make this work properly.

No, This is not a Git for Windows problem, after I debugged it, I realized that it is due to the naming of the Path environment variable, in the following code, git-branchless reads the environment variable named PATH, but in windows, the variable name should be Path.

https://github.com/arxanas/git-branchless/blob/e7c3653c5e42f67a967d30ea57ded5a34cc6f9aa/git-branchless-lib/src/git/run.rs#L407-L422

I believe this is due to the fact that the windows shell is insensitive to the case of environment variable names (both pwsh and cmd), which many developers don't pay attention to on windows, you can read the environment variables correctly in the shell with Path PATH or even PaTh.

quick fix:

        let GitRunInfo {
            // We're calling a Git hook, but not Git itself.
            path_to_git: _,
            // We always want to call the hook in the Git working copy,
            // regardless of where the Git executable was invoked.
            working_directory: _,
            env,
        } = self;
        let path = {
            let mut path_components: Vec<PathBuf> =
                vec![std::fs::canonicalize(&hook_dir).wrap_err("Canonicalizing hook dir")?];
            if let Some(path) = env.get(OsStr::new("PATH")) {
                path_components.extend(std::env::split_paths(path));
            }
            #[cfg(target_os = "windows")]
            if let Some(path) = env.get(OsStr::new("Path")) {
                path_components.extend(std::env::split_paths(path));
            }
            std::env::join_paths(path_components).wrap_err("Joining path components")?
        };

yep, windows std also:

fn main() {
    println!("{:?}", std::env::var_os("Path")); // valid
    println!("{:?}", std::env::var_os("PATH")); // valid
    println!("{:?}", std::env::var_os("PaTh")); // valid
    // but 
    let x = std::env::vars_os().collect();
    // put "Path" inside x
}

glencbz commented 6 months ago

This seems to be happening to me sporadically, where git branchless doesn't abandon commits during a restack.

The offending bit seems to be in .git/hooks/reference-transaction:

# This complains with git: command not found
git branchless hook reference-transaction "$@" || (
echo 'branchless: Failed to process reference transaction!'
echo 'branchless: Some events (e.g. branch updates) may have been lost.'
echo 'branchless: This is a bug. Please report it.'
)

As others have noted, this is because of some oddities in $PATH. Adding echo $PATH to the hook shows that PATH is set to /c/path/to/project/.git/hooks, which looks unusual. What's extra weird is that this problem doesn't seem to appear when running git commit --amend --no-edit or git amend, though it does happen with git move.

c00t commented 6 months ago

@glencbz This should be fixed in the master branch, you can test it with cargo install --git https://github.com/arxanas/git-branchless

arxanas / git-branchless