[bug] Prompt takes long time on very large git repos

[epsilon-0]

When working with very large repositories, specifically https://github.com/openbsd/ports, the prompt takes really long to show up.

This is not inherently an issue with polyglot as any git operation on that repository takes >3 seconds (even with a 24core, 32GB server).

Can polyglot make it so that it can kill the process by limiting the time spent to something like 0.1 seconds?

This is really making it impossible to use it right now :crying_cat_face:

[epsilon-0]

An idea, was to use git -uno as prompted by git, because it was taking too long.

[agkozak]

An idea, was to use git -uno as prompted by git, because it was taking too long.

I think that would probably be the right solution. I'll be happy to add an option that will do that in the next day or two.

Thanks so much for helping me to improve Polyglot!

[agkozak]

Why don't you pull what I've got on the show-tracked branch and give it a try? If you set POLYGLOT_SHOW_TRACKED=0, Polyglot will use git status -uno instead of git status. Tell me if that's fast enough to be usable.

[epsilon-0]

Indeed, it is considerably faster! Unfortunately, I can still notice a lag.

$ time git status -uno
...
0m00.96s real     0m00.12s user     0m09.82s system

I am really happy to go from 3-4 to ~1s and I'm not even sure if there is a better/faster method ?

Would love to know your thoughts.

[agkozak]

Out of curiosity, which shell are you using?

[epsilon-0]

I am using ksh, the standard one that's used with OpenBSD.

[agkozak]

It's probably not going to get a lot better than what you proposed on ksh. The reason that I use zsh (along with my agkozak prompt) is that I can make the Git part of the prompt asynchronous. A prompt can be printed to the screen as quickly as possible -- without waiting for git status -- and then the Git status displays whenever it's ready. I've never seen that done with ksh -- it might be very hard to accomplish, if not impossible.

[epsilon-0]

Ah, I see.

For what its worth, openbsd ksh (and hence loksh, which is the portable version of ksh) actually has really nice support for asynchronous processes, using co-processes - https://man.openbsd.org/ksh#Co-processes.

How does it update the prompt, if the async process finishes after the prompt has already been printed? Does it store what the current status was and uses it for the next time ?

[agkozak]

You're right -- those co-processes could be very useful. I'll play around with them a bit one of these days.

More than anything, I'm going to have to figure out how to have a pdksh process redraw the prompt when it's time; there would be a $GIT_STATUS variable in the prompt that would initially be empty and would then be set to the Git status -- but I'll need to redraw the prompt for it to display. Do you have any idea how that's done?

I'd love to take on the task of making an asynchronous ksh prompt, but it's unlikely to end up being part of the Polyglot Prompt. I've had to code this prompt very carefully so that the various shells don't reject anything too specific to any given shell. dash is the worst; it won't even parse code with some of ksh93's builtin variables in it.

[epsilon-0]

I believe you :smiley_cat:
I will let you decide on the technical aspects and see how it can be done.

Thanks

[epsilon-0]

I don't know much about prompts and redrawing a prompt.
I will have to ask the other obsd shell peeps. Will tell when I get info

[agkozak]

Give me a little while to think about a possible spinoff ksh prompt.

In the meantime, will the change that you suggested and I added to Polyglot suffice for your purposes? If so, I'll test it out a little more and work on merging it into the master branch.

[epsilon-0]

Oh, yes, of course, it is very nice. Thanks a lot.

Now I am basically looking at this as a functionality to conquer :smiley_cat: Would be really cool to have it but also not the biggest hindrance.

[agkozak]

I merged the changes to master, with one change: I've renamed the setting to POLYGLOT_SHOW_UNTRACKED. I think that makes more sense, doesn't it?

[epsilon-0]

sounds nice! I am still unsure about the co-processes, as I am nowhere near shell expert. I suggest keeping this open until we find a resolution. Some brave soul may venture forth with a solution.

[agkozak]

Sure. Another way to do it would be to have a background process that writes the Git status to a temporary file and then triggers a trap that reads from that file into a variable. The problem remains how to redraw the prompt. I'll continue to think about this problem.

[tkapias]

I often need to visit a project with more than 200k files because of submodules. The untracked solution is not enough.

I added a timeout for 5 seconds in the script and a "T" symbol in case of timeout.

262c262
<       POLYGLOT_GIT_STATUS=$(LC_ALL=C GIT_OPTIONAL_LOCKS=0 env git status -uno 2>&1)
---
>       POLYGLOT_GIT_STATUS=$(LC_ALL=C GIT_OPTIONAL_LOCKS=0 timeout 5 env git status -uno 2>&1 || echo statusTimeout)
264c264
<       POLYGLOT_GIT_STATUS=$(LC_ALL=C GIT_OPTIONAL_LOCKS=0 env git status 2>&1)
---
>       POLYGLOT_GIT_STATUS=$(LC_ALL=C GIT_OPTIONAL_LOCKS=0 timeout 5 env git status 2>&1 || echo statusTimeout)
268a269,271
>     case $POLYGLOT_GIT_STATUS in
>       *'statusTimeout'*) POLYGLOT_SYMBOLS="${POLYGLOT_SYMBOLS}T" ;;
>     esa