Closed amoldeshpande closed 3 years ago
so, unfortunately, not seeing much improvement with this change either
S J:\github\UnrealEngine> posh-windows-amd64.exe --config C:\Users\amol\.poshthemes\jandedobbeleer.omp.json --shell pwsh --debug
Here are the timings of segments in your prompt:
session(true) - 0 ms - █amol@amolhomevista█
spotify(false) - 11 ms -
path(true) - 0 ms - UnrealEngine█
git(true) - 1359 ms - █ 4.25 ↓15█
battery(true) - 3 ms - █100
node(false) - 0 ms -
shell(true) - 0 ms - ﲵ pwsh█
root(false) - 0 ms -
exit(true) - 0 ms - █
PS J:\github\UnrealEngine> j:..\oh-my-posh3\src\oh-my-posh3.exe --config C:\Users\amol\.poshthemes\amol.omp.json --shell pwsh --debug
Here are the timings of segments in your prompt:
text(true) - 0 ms - █%c03█
git(true) - 1253 ms - █ 4.25 ↓15█
text(true) - 0 ms - █%T█
exit(true) - 0 ms - █
PS J:\github\UnrealEngine>
PS H:\github\tcsh>
Edit: added output from the right branch, I think. some improvement.
Edit2: git status measurement for context
PS J:\github\UnrealEngine> Measure-Command {git status}
Days : 0
Hours : 0
Minutes : 0
Seconds : 1
Milliseconds : 212
Ticks : 12129090
TotalDays : 1.40382986111111E-05
TotalHours : 0.000336919166666667
TotalMinutes : 0.02021515
TotalSeconds : 1.212909
TotalMilliseconds : 1212.909
While reducing the number calls, we can add parameters to add/remove informations from the git prompt(like it is done in starship). So people who only wants a subset of the info will have better performance and those who are willing to pay the price will get everything.
while digging in starship I found they use this library https://github.com/libgit2/libgit2(https://libgit2.org/) which has a go version(https://github.com/libgit2/git2go).
This will introduce more complexity but external call will be avoided and all git actions done using c.
https://github.com/lnu/oh-my-posh3/tree/feature/git2go
I will say that the test with UnrealEngine is unfair. "git status" by itself takes more than a second in that directory. But it does bring up questions about how far you can really take the performance.
I think we should, for the first part, go with @shedric1's branch which reduces the amount of unneeded git calls.
For performance, we should always include a git status alongside it, we can't beat that performance without some magic. (and git status is slower when there are submodules in the git repo).
Libgit2 might be an option, do we know how the git status command compares to git2s status? Also, this extra library won't prevent single exe deployment right?
Sounds reasonable to me. I added "git status" measurements to my comment about @shedric1's change.
My ideal goal would take into account the fact that a simple external command takes ~30-40ms on Windows. We should not take much more than that for most commands.
Someone up there made a suggestion of picking specific git states to print in the prompt. That would be perfect because the main thing I care about usually is the branch. Filters by path for each git part would be even better, but I realize that's just being greedy :-)
@lnu after checking the starship repo, please note they are removing libgit with calls to the native git executable for 1.0. So I don't think we should consider libgit at the current moment.
Well, this thread just keeps growing :-)
@shedric1 I was looking at the same thing this morning, also found out the stash count is being done AT ALL TIMES which is a huge mistake on my part. I agree with swapping out the calls as much as we can, I do believe your changes are a candidate for this. How would you like to proceed? Do you want to give this a go, or do you want me to assist in merging this in properly? It will require a rewrite of tests, splitting functionality to environment, etc, so there's more work to be done than doing the changes alone.
About git2go, that still requires the libgit binary so we can't have a single exe in that case. I'd say avoiding doing git calls is the most straightforward approach here.
@JanDeDobbeleer I won't have a chance to look into a proper version with updated tests and organization for a while, so anyone is welcome to get the ball rolling on it using that branch! I just whipped those up as a proof of concept to see if it worked and offered meaningful performance improvements, but I'm new to Go so a well formatted version might take me a lot longer than someone more familiar with the language. I'll take a crack at it next week if no one else takes up the mantle by then!
I'll pick up the task. I can do this during lunchtime, I'll try to provide intermediate builds here for validation.
Another interesting datapoint. I use SourceTree and its embedded git version. Turns out it defaults to a really old one and you have to manually make it update. I went from git 2.11 to 2.20 when I forced the upgrade.
anyway, to get to the point, git status in my UnrealEngine repo is ~900 ms in 2.20 (so a good 300ms faster), and as a result posh comes in around 1.2 seconds instead of 1.5+
Just another variable to add to the matrix.
I'm excited to see the progress in making the git segment faster (it's 81ms on my home computer, I'll update my work machine tomorrow with 3.86.3) but there is one thing I want to caution: finding git entries manually can be problematic if you're only looking for a .git
folder.
I use git's worktrees pretty extensively and they only have a .git
file, currently with a single entry of gitdir: c:/absolute/repo/.git/worktrees/name
that link the two together. Currently 3.86.3 works with this setup and displays all output properly (branches, staging, push/pull distance, etc) and would hate to see it only work on full clone repositories.
@gibwar Excellent point about worktrees, I'm not sure how the current proof of concept will work with those. As is, it's using the Stat
function from the os
package which should return true if a matching directory or file is found, if I'm reading it correctly. If I'm understanding how worktrees work, this may cause the repo root it finds to be totally wrong.
Should be an easy enough fix though. An additional check can be used once any git path match is found to see if it was a folder or file and proceed as necessary from there. Of course this raises the question of what other git features we're forgetting to account for when manually implementing...
I put some timing around spawned command execution and came up with this:
J:\github\UnrealEngine>h:oh-my-posh3.exe --config C:\Users\amol\.poshthemes\amol.omp.json --shell pwsh --debug
c:\users\amol\appdata\local\Atlassian\SourceTree\git_local\bin\git.exe rev-parse --is-inside-work-tree
43.0001ms
c:\users\amol\appdata\local\Atlassian\SourceTree\git_local\bin\git.exe -c core.quotepath=false -c color.status=false rev-parse --show-toplevel
44.0043ms
c:\users\amol\appdata\local\Atlassian\SourceTree\git_local\bin\git.exe -c core.quotepath=false -c color.status=false status -unormal --short --branch
850.9924ms
c:\users\amol\appdata\local\Atlassian\SourceTree\git_local\bin\git.exe -c core.quotepath=false -c color.status=false rev-list --walk-reflogs --count refs/stash
c:\users\amol\appdata\local\Atlassian\SourceTree\git_local\bin\git.exe -c core.quotepath=false -c color.status=false remote get-url upstream
37.9849ms
I think it shows what we already knew, that status
is the slowest command. Figured it would helpful to have concrete numbers anyway.
Editing to add more timing investigation results:
If I change the status command to -uno
instead of -unormal
, that cuts the time down to 280-300-ish ms.
J:\github\UnrealEngine>h:oh-my-posh3.exe --config C:\Users\amol\.poshthemes\amol.omp.json --shell pwsh --debug
c:\users\amol\appdata\local\Atlassian\SourceTree\git_local\bin\git.exe rev-parse --is-inside-work-tree
39.9979ms
c:\users\amol\appdata\local\Atlassian\SourceTree\git_local\bin\git.exe -c core.quotepath=false -c color.status=false rev-parse --show-toplevel
45.001ms
c:\users\amol\appdata\local\Atlassian\SourceTree\git_local\bin\git.exe -c core.quotepath=false -c color.status=false status -uno --short --branch
290.9995ms
c:\users\amol\appdata\local\Atlassian\SourceTree\git_local\bin\git.exe -c core.quotepath=false -c color.status=false rev-list --walk-reflogs --count refs/stash
c:\users\amol\appdata\local\Atlassian\SourceTree\git_local\bin\git.exe -c core.quotepath=false -c color.status=false remote get-url upstream
44.996ms
This command skips untracked files. However, to get the untracked files alone, one can do
git ls-files --others --exclude-standard --directory
this seems to give almost the same information as -unormal
I am not enough of a git expert to understand the differences. It also takes 1000-ish ms to run.
All of this suggests a way to break up git status into smaller sub-commands that can presumably be parallelized.
A config option to skip untracked file counting might be a good short-term fix.
If all goes well I'll have a working version tomorrow that uses paths to validate if we're in a git repo, including support for worktrees.
As promised a build based on the changes made with @shedric1 's proposal. There are two executables in the zip, fork is build with the go fork that removes the wait in process. In my preliminary tests, this already has some benefits (don't mind the font not being set correctly).
Not inside GIT repo Inside git Repo Git status takes ~256MS
Looks like an nice improvement! Seems to be an overhead of 20MS + the time it longest time it takes for 1 segment. The Fork would also be nice to keep in sync with GO, 40MS improvement is nice!
Keeping the fork synced isn't going to be the challenge so I'm definitely in favor of keeping that. Cool that we're now seeing actual differences, that's clear progress!
Here' the relevant parts from my timings. Nice work!
❯ oh-my-posh --config "c:/users/travis.collins/.poshthemes/travis.omp.json" --debug
git(true) - 138 ms - master
❯ posh-windows-amd64-experimental --config "c:/users/travis.collins/.poshthemes/travis.omp.json" --debug
git(true) - 61 ms - master
❯ posh-windows-amd64-experimental-fork --config "c:/users/travis.collins/.poshthemes/travis.omp.json" --debug
git(true) - 53 ms - master
not seeing much difference between fork and non-fork, but there's a 40ms improvement between the released version. I had to run each version a few times to get somewhat stable measurements to account for the OS fs cache.
x> H:\Archive\posh-windows-amd64-experimental.exe --config C:\Users\amol\.poshthemes\jandedobbeleer.omp.json --shell pwsh --debug
Here are the timings of segments in your prompt:
session(true) - 0 ms - █amol@amolhomevista█
spotify(false) - 14 ms -
path(true) - 0 ms - ForgottenArmies_cpp█
git(true) - 163 ms - █ htn ≡█
battery(true) - 1 ms - █100
node(false) - 0 ms -
shell(true) - 0 ms - ﲵ pwsh█
root(false) - 0 ms -
exit(true) - 0 ms - █
x> H:\Archive\posh-windows-amd64-experimental-fork.exe --config C:\Users\amol\.poshthemes\jandedobbeleer.omp.json --shell pwsh --debug
Here are the timings of segments in your prompt:
session(true) - 0 ms - █amol@amolhomevista█
spotify(false) - 13 ms -
path(true) - 0 ms - ForgottenArmies_cpp█
git(true) - 164 ms - █ htn ≡█
battery(true) - 2 ms - █100
node(false) - 0 ms -
shell(true) - 0 ms - ﲵ pwsh█
root(false) - 0 ms -
exit(true) - 0 ms - █
PS x> C:\bin\posh-windows-amd64.exe --config C:\Users\amol\.poshthemes\jandedobbeleer.omp.json --shell pwsh --debug
Here are the timings of segments in your prompt:
session(true) - 0 ms - █amol@amolhomevista█
spotify(false) - 9 ms -
path(true) - 0 ms - ForgottenArmies_cpp█
git(true) - 213 ms - █ htn ≡█
battery(true) - 2 ms - █100
node(false) - 0 ms -
shell(true) - 0 ms - ﲵ pwsh█
root(false) - 0 ms -
exit(true) - 0 ms - █
x>
I'm testing all the different cases, so far so good. I'll already publish this part.
@TravisTX @lnu @amoldeshpande @royvou @shedric1 @gibwar looking at the latest release, how do we feel about the speed? Improved enough to consider this solved, or do we need follow-up actions to go for the extra smile?
It's good for me (but it's always been pretty good on my computer). I'm at about 63ms in a git repo, and 7ms outside of one. Which is a notable improvement!
Latest release is perceptibly faster than the old one, so I'm fine with calling it good. It's about 100-120ms in a repo like posh3 for me.
thanks !
Decent performance increase from 1s to 500ms, but I'm not getting speeds increases everyone else is having :/.
@Kudostoy0u can you validate if there's an update of git for your system? That might also have an impact.
Here are some versions of things I have: I updated git to 2.30 (should have done that earlier) but still no difference :(
up in the thread I posted a comment with a few typical git commands that posh runs. measure those by commands themselves using the Measure-Command
cmdlet. posh no longer runs all of them, but it will give you an idea of how slow git by itself is in your repo.
Add another ~40-80ms for go
spawning subprocesses on Windows and you'll find your lower bound.
For example, my Unreal Engine repo went from 1200ms to 850-900ms. It's a big improvement, but still completely useless so I have a filter to turn posh off in that repo. This is not the fault of posh but just that git status
takes a long time to run in that particular repo.
BTW I have this function in my pwsh profile:
Hm... This is very erratic behavior. This is the result for an empty git repo, just a folder which I ran git init
:
As soon as I create a file and add it, the performance increases
And that wasn't a one-off thing, I ran the command again and it reported 301 ms
@Kudostoy0u is that the latest release or still one of the artifacts above?
It is the experimental fork. I switched to using it since it does in fact induce quicker prompt response times when in a git repository. Here is my full PowerShell profile:
@Kudostoy0u I would actually use the latest release as a lot of git calls were removed since then. It contains more optimizations than the fork right now.
Hello. The program runs very smothly(almost instantaneously) when outside a git repository, but as soon as I enter a git repository(ir doesn't matter how big or small it is) it gets slow taking more than 300ms to execute on average wich is noticible. An example can be seen in the picture below:
I have followed the steps described in the offitial page for the installation. I've been browsing the internet trying to look for a solution or verifying if this was normal or not but couldn't find an answer. Also, on the youtube videos I visited seemed didn't seem to be any kind of slowdowns when getting in a git repository.
I want to know if this is normal and if it is not I'd like to know how I could try to fix it. Thanks in advance!
If it is the same in all repos regardless of size, it might be that your version of git is too old. They incorporated some caching in Git For Windows at some point to make the performance better.
as I said above, find out how long git status
takes to run in your repo, add 40-50ms for go
overhead on Windows and that will be your lower bound. posh might spawn more than 1 git command, so presumably you have to add 80-100 ms overhead.
I'm in the last version of git-for-windows and this is how long it takes git status:
The sum doesn't add to 300 ms
Apologies, turns out my version of posh is very old. When I tried out the current version, it has performance regression you are describing.
> i:/temp/posh-windows-amd64.exe --shell zsh --config ~/.poshthemes/amol.omp.json --debug
Here are the timings of segments in your prompt:
ConsoleTitle(false) - 0 ms -
os(true) - 0 ms - %{%} %{%}%{%}%{%}%{%} %{%}
text(true) - 0 ms - %{%} %{%}%{%}%c03%{%}%{%} %{%}
git(true) - 270 ms - %{%}%{%}%{%} %{%}%{%}main%{%}%{%} %{%}%{%}%{%}
text(true) - 0 ms - %{%}%{%}%{%} %{%}%{%}%T%{%}%{%} %{%}%{%}%{%}
exit(true) - 0 ms - %{%}%{%}%{%} %{%}%{%} %{%}%{%}%{%}
> c:/bin/posh-windows-amd64.exe --shell zsh --config ~/.poshthemes/amol.omp.json --debug
Here are the timings of segments in your prompt:
os(true) - 0 ms - %{%}█%{%}%{%} %{%}
text(true) - 0 ms - %{%}█%{%}%{%}%c03%{%}%{%}█%{%}
git(true) - 91 ms - %{%}%{%}%{%}█%{%}%{%} main ≡%{%}%{%}█%{%}%{%}%{%}
text(true) - 0 ms - %{%}%{%}%{%}█%{%}%{%}%T%{%}%{%}█%{%}%{%}%{%}
exit(true) - 0 ms - %{%}%{%}%{%} %{%}%{%}█%{%}%{%}%{%}
>
The one I:
is the current version, the one in c:
is 3.70.
Anyway, I don't know the repo protocol about reopening issues, so you may be better off filing a new one.
Ok
@JanDeDobbeleer Have we considered @amoldeshpande suggestion to use -uno
instead of -unormal
in git status
?
@jcwilkes I did, but then we'd lose all untracked info. We could make that configurable though.
I have been using oh my posh 3 for a while now on other systems but I got a new work laptop today and went through the install procedures on the web site using the Admin Powershell method to install the modules. When I then try and set any posh prompt theme it takes a very long time e.g. 20303ms and sometimes more. All commands in the terminal and then very slow
When I add the import and set-poshprompt settings in my PS profile it takes for ever to load
I been able to track it down to setting the prompt and once the prompt theme is loaded. If I remark out all those entries in my PS profile it loads very quick (as normal) and functionality is fine.
@apapiccio can you validate if Windows Defender (or any other scanning tool) isn't blocking the execution of oh-my-posh?
I'll check that out and let you know
Get Outlook for Androidhttps://aka.ms/AAb9ysg
From: Jan De Dobbeleer @.> Sent: Tuesday, March 29, 2022 6:29:11 PM To: JanDeDobbeleer/oh-my-posh @.> Cc: Angelo Papiccio @.>; Mention @.> Subject: Re: [JanDeDobbeleer/oh-my-posh] Performance: Command execution slow on Windows (#305)
@apapicciohttps://github.com/apapiccio can you validate if Windows Defender (or any other scanning tool) isn't blocking the execution of oh-my-posh?
— Reply to this email directly, view it on GitHubhttps://github.com/JanDeDobbeleer/oh-my-posh/issues/305#issuecomment-1081702847, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AGMEMHHUCPHEFHKCBVQW6H3VCLLPPANCNFSM4VQS3D6Q. You are receiving this because you were mentioned.Message ID: @.***>
Hi @JanDeDobbeleer
Today I upgraded oh-my-posh (don't remember from which version unfortunately) from the module-based version to the exe based version. Now it is unusably slow.
Like @apapiccio when I add the command
#oh-my-posh init pwsh --config "$env:POSH_THEMES_PATH\powerlevel10k_rainbow.omp.json" | Invoke-Expression
to the profile it takes ~ 20 seconds to load the profile.
Showing the prompt takes 2-3 seconds.
This was never the case with the module-based version, where I only included
Set-PoshPrompt -theme powerlevel10k_rainbow
in the profile.
That worked really well.
I can also see after the upgrade the execution of oh-my-posh.exe
is very slow:
PS C:\Temp > measure-command { oh-my-posh version }
Days : 0
Hours : 0
Minutes : 0
Seconds : 2
Milliseconds : 527
Ticks : 25274152
TotalDays : 2,92524907407407E-05
TotalHours : 0,000702059777777778
TotalMinutes : 0,0421235866666667
TotalSeconds : 2,5274152
TotalMilliseconds : 2527,4152
So only displaying the version takes 2.5 seconds?! oh-my-posh version is 7.74.3
I suspect some snake oil but as this is a corporate machine I don't have a choice here... We are using the enterprise version of Windows Defender.
For now, for me this is unusable and unfortunately I needed to comment out oh-my-posh from my profile which kind of sucks.
@thmudersfc this is 9/10 Defender that blocks the execution of the executable. So having a conversation with the IT team could fix that. Or, and this is an assumption, the Documents folder is excluded from Defender which is why the module (that uses the same executable) wasn't impacted. Not sure if you can have an ask?
Thanks @JanDeDobbeleer so it is not blocked per se, it works, it is just incredibly slow. With you hint I moved the file to several different locations where I usually install programs (not Program Files) but that did not make any difference at all. I only don't understand what could probably be the difference. Debug mode also does not show any slowness inside oh-my-posh
It takes already 2-3 seconds before debug output starts. The slowest part in debug is the scanning of the node version which takes 700ms, the rest is super fast.
@thmudersfc if debug doesn't show slowness, it's definitely something on the system blocking the execution.
@JanDeDobbeleer Thank you, if needed I will open a ticket with our service desk. or use some sysinternals tools to dig deeper if I find some time The only thing that I did not understand why this happened only after the upgrade.
@thmudersfc the only reason I can come up with is install location.
You can try windbg and load the exe in the debugger. If you see dozens of DLLs loaded dynamically at startup, that could be a reason too. I haven’t looked at it in a while but my feeling is that by loading dynamically go bypasses the windows multithreaded loader and causes slow launch times. You can investigate with WPA/WPT as well
@amoldeshpande we have a static build normally. But that's something I could look into (even if I'm pretty sure it's not that).
Prerequisites
CONTRIBUTING
guideDescription
Slowness in rendering prompt. It's noticeably slow in Windows compared to WSL2, even in the same Terminal.
Environment
Steps to Reproduce
I was trying to figure out why composing the prompt is so slow on windows. I unfortunately don't know much go, but I muddled around with it and found a couple of things.
Firstly, running and parsing git commands seems quite slow.
As you can see here, running the raw git command takes less than half the time it takes to run a posh command where my only config is a git segment (the default one from your theme file)
There is no git repo in that directory (I was trying to get the fastest execution for the git segment, but interestingly the overall execution time of posh does not change even within a git repo)
I tried to do some profiling in go, and came up with this for execution:
I'm always running
where the theme only contains a only git prompt segment exactly like the jandedobbeleer profile
In a repo with git, this is the profile
So, it seems that running multiple git commands slows things down even more , as you'd expect.
With my less-than-zero knowledge of go, I can only theorize that
Love the tool, thanks for the great work !