Closed jojojames closed 3 years ago
When I don't have to do anything with the submodules, then I keep the Modules section collapsed and everything is snappy. When I do have to deal with submodules, then I just accept the slowdown.
Hmnn, I looked deeper into this since you've said things should still be snappy.
before borg migration
magit-insert-error-header 3e-06
magit-insert-diff-filter-header 0.009832
magit-insert-head-branch-header 0.011721 !
magit-insert-upstream-branch-header 7e-06
magit-insert-push-branch-header 4e-06
magit-insert-tags-header 0.012647 !
magit-insert-status-headers 0.04159 !!
magit-insert-merge-log 0.004533
magit-insert-rebase-sequence 0.000233
magit-insert-am-sequence 0.000106
magit-insert-sequencer-sequence 0.000203
magit-insert-bisect-output 0.000106
magit-insert-bisect-rest 2.6e-05
magit-insert-bisect-log 2.5e-05
magit-insert-untracked-files 0.126452 !!
magit-insert-unstaged-changes 0.124834 !!
magit-insert-staged-changes 0.012754 !
magit-insert-stashes 0.004464
magit-insert-unpushed-to-pushremote 1.1e-05
magit-insert-unpushed-to-upstream-or-recent 0.028529 !
magit-insert-unpulled-from-pushremote 3e-06
magit-insert-unpulled-from-upstream 1e-06
after borg migration
magit-insert-error-header 2e-06
magit-insert-diff-filter-header 0.0091
magit-insert-head-branch-header 0.01059 !
magit-insert-upstream-branch-header 0.010068 !
magit-insert-push-branch-header 0.004419
magit-insert-tags-header 0.012116 !
magit-insert-status-headers 0.051897 !!
magit-insert-merge-log 0.004361
magit-insert-rebase-sequence 0.000186
magit-insert-am-sequence 8.1e-05
magit-insert-sequencer-sequence 0.000176
magit-insert-bisect-output 8.1e-05
magit-insert-bisect-rest 1.8e-05
magit-insert-bisect-log 1.9e-05
magit-insert-untracked-files 1.05173 !!
magit-insert-unstaged-changes 1.063564 !!
magit-insert-staged-changes 0.009534
magit-insert-stashes 0.004324
magit-insert-modules 0.006176
magit-insert-unpushed-to-pushremote 6.2e-05
magit-insert-unpushed-to-upstream-or-recent 0.032899 !!
magit-insert-unpulled-from-pushremote 5e-06
magit-insert-unpulled-from-upstream 0.008047
It looks like you're right, that inserting modules is pretty fast but for some reason, the untracked/unstaged sections have grown up to 1 second each from .1 seconds.
Any tips or advice here appreciated, I took a look at those two functions but it wasn't immediately obvious where to begin looking or how to profile those two.
I don't know what "borg migration" entails. For example did you use submodules before too?
Anyway, I don't think borg is responsible for that. Maybe the use of submodules contributes to it. I generally recommend against inserting untracked files as that tends to be slow unless you make sure there usually aren't any untracked files.
I don't have time right now to help users with things like this and kindly ask you to figure it out ourself. (If you come up with some actionable advice, then please let me know though.)
I don't have time right now to help users with things like this and kindly ask you to figure it out ourself. (If you come up with some actionable advice, then please let me know though.)
That's reasonable, I tracked it down to this:
https://github.com/magit/magit/commit/a7699f868f6fd32aae8aa8e64a72ea989a5c25cf
Looks like, magit-insert-untracked-files passes a flag returned by 'magit-ignore-submodules-p to the eventual git shell command it makes, but by default, there are no git settings set by the user to set up the submodule-ignore which results in a slow experience after bootstrap.
e.g. It is running this command for me:
$ git status -z --porcelain #slow
vs
git status -z --porcelain --ignore-submodules=all #fast
I am guessing you already have some kind of submodule-ignore setting set in your own repos, whether global or local to those repos, that allows you to avoid seeing this problem.
I don't think I have. I just make sure that there usually aren't many untracked files and unstaged changes, laying around. If there is an untracked file, then I either track it or I gitignore it. Performance isn't great but quite acceptable:
magit-insert-untracked-files 0.013418741 !
magit-insert-unstaged-changes 0.25856443 !!
But yeah, setting the variable that is mentioned in that commit should also help (but of course your would lose information).
Yeah, setting up the submodule config cleared everything up for me.
My repository had 0 untracked/unstaged files so the 1 second to resolve the files was strange to me.
Now it looks like this:
magit-insert-error-header 3e-06
magit-insert-diff-filter-header 0.009667
magit-insert-head-branch-header 0.010138 !
magit-insert-upstream-branch-header 0.010126 !
magit-insert-push-branch-header 0.004609
magit-insert-tags-header 0.012483 !
magit-insert-status-headers 0.052818 !!
magit-insert-merge-log 0.004689
magit-insert-rebase-sequence 0.000218
magit-insert-am-sequence 0.0001
magit-insert-sequencer-sequence 0.00019
magit-insert-bisect-output 0.0001
magit-insert-bisect-rest 2.4e-05
magit-insert-bisect-log 2.3e-05
magit-insert-untracked-files 0.041522 !!
magit-insert-unstaged-changes 0.037503 !!
magit-insert-staged-changes 0.010198 !
magit-insert-stashes 0.004354
magit-insert-modules-overview 0.005184
magit-insert-modules-unpulled-from-upstream 3.3e-05
magit-insert-modules-unpulled-from-pushremote 1.3e-05
magit-insert-modules-unpushed-to-upstream 1.1e-05
magit-insert-modules-unpushed-to-pushremote 1e-05
magit-insert-modules 0.010912 !
magit-insert-unpushed-to-pushremote 4.2e-05
magit-insert-unpushed-to-upstream-or-recent 0.034282 !!
magit-insert-unpulled-from-pushremote 3e-06
magit-insert-unpulled-from-upstream 0.008295
For completeness sake and for future readers:
I found something like this:
$ git config --add diff.ignoreSubmodules dirty
fixed the issue for me but doesn't persist across repos when cloning so tweaked the Makefile's bootstrap.
.DEFAULT_GOAL := default
default: bootstrap-borg
@ echo Calling git submodule update --init --recursive...
@ git submodule update --init --recursive
@ echo Calling git config --add diff.ignoreSubmodules dirty...
@ git config --add diff.ignoreSubmodules dirty
@ echo Call make bootstrap to finish bootstrap.
DRONES_DIR = $(shell git config "borg.drones-directory" || echo "lib")
-include $(DRONES_DIR)/borg/borg.mk
bootstrap-borg:
@git submodule--helper clone --name borg --path $(DRONES_DIR)/borg \
--url git@github.com:emacscollective/borg.git
@cd $(DRONES_DIR)/borg; git symbolic-ref HEAD refs/heads/master
@cd $(DRONES_DIR)/borg; git reset --hard HEAD
.PHONY: default
Originally posted by @tarsius in https://github.com/emacscollective/borg/issues/12#issuecomment-318936162
I was looking for more info on this but this seems to be sparse. @tarsius, do you just deal with this while using borg or is there some tricks you're using to get performance up. With around 100 submodules, refreshing the buffer is noticeably slower but still manageable.