search

hbons / SparkleShare

Share and collaborate by syncing with any Git repository instantly. Linux, macOS, and Windows.
https://sparkleshare.org
Other
4.88k stars 577 forks source link

Is it normal to take 30 minutes to add an empty repository? #1938

Open MountainX opened 4 years ago

MountainX commented 4 years ago

I'm running sparkleshare 3.28-4 on Arch Linux (from https://www.archlinux.org/packages/community/any/sparkleshare/)

What happened:

I created a new empty repository in GitLab.

What I expected to happen:

I expected adding the project to SparkleShare would be fast.

What actually happened:

SparkleShare took 30 minutes to complete this small task. The CPU load and memory load on my client were normal. There are no errors in the logs. I do not see any problems. It just take a really long time. It does finally finish properly.

In fact, all operations seem to take a lot longer than expected. I just started using SparkleShare and I love the idea of it. I'm testing it out with a few projects on GitLab. Most have 1 small plain text file for testing. Syncing either direction takes 10 to 20 minutes even if I just change 1 line of text in 1 file. Is this normal?

Is there a way to speed things up?

Thank you. This is a nice project. I hope it will work for me. If it does, I'll send a donation.

MountainX commented 4 years ago

I did the same operation on a friend's computer. Her computer is running sparkleshare 3.28-4 on Arch Linux (same as me). The repository was added within a few seconds (under a minute).

Do you have any ideas for the differences in time? There is not a single error in my SparkleShare log. The operation proceeds normally and finishes correctly. It just takes a very long time. What could be the reason?

Also some syncing operations still take up to 10 minutes on her computer, however. I saw on the Wiki that it is possible to do manual git pulls and pushes. What are the steps for this? Thanks.

hbons commented 4 years ago

can you clone the same repo from a terminal? is it still slow then?

MountainX commented 4 years ago

can you clone the same repo from a terminal? is it still slow then?

Yes, I cloned the same repo in a terminal and it took about 1 second. It was almost instant because it is an empty repo.

I even used the same command for the clone that SparkleShare uses:

git clone --progress --no-checkout --depth=1 "ssh://git@gitlab.com/group/project"

I see that command in the SparkleShare log and it does not have any errors associated with it. But the entire* process takes 30 minutes in SparkleShare. Memory and CPU load are not high.

I repeated the process with a new empty repo. This time it took from 17:54 to 18:11 which is about 17 minutes. Cloning that same repo in the terminal is also less than 1 second. I will attach the log file...

MountainX commented 4 years ago

BTW - the only major difference I can find between my friend's computer and mine is that I have the Nextcloud client installed. (We both run Arch Linux and KDE with almost identical settings.) Have you heard of any conflicts between Nextcloud and SparkleShare? (I'm not saying there are. I'm just asking if you are aware of any. If you think I should remove Nextcloud client, I will, but I was hoping to use both.)

MountainX commented 4 years ago

sparkleshare_log_redacted.txt

hbons commented 4 years ago

thanks for the detailed description. that's weird, in the log the clone process finishes very quickly. but it looks like the UI isn't updated for a while...

MountainX commented 4 years ago

Do you have any suggestions for any troubleshooting I can try?

hbons commented 4 years ago

No idea at this point...

kevincox commented 4 years ago

I have noticed that initial clones were slow for a long time but recently I was watching the log while waiting and I saw something interesting. The clone happens at a fairly reasonable speed but the config setting happens veeeeeery slowly. I tried running the printed commands in the repo directly and they finished in 10s of milliseconds. So there must be something wrong with the command spawn/wait cycle. I peeked at the code and nothing seemed obvious.

17:46:24 Fetcher | /home/kevincox/.config/org.sparkleshare.SparkleShare/tmp/passwords | Fetching folder: ssh://git@bitbucket.org/kevincox/passwords
17:46:24 Auth | Fetching host key for bitbucket.org
17:46:24 Cmd | ssh-keyscan -t rsa -p 22 bitbucket.org
17:46:33 Auth | Fingerprint matches
17:46:33 Cmd | tmp | git ls-remote --heads "ssh://git@bitbucket.org/kevincox/passwords"
17:46:45 Cmd | tmp | git clone --progress --no-checkout --depth=1 "ssh://git@bitbucket.org/kevincox/passwords" "/home/kevincox/.config/org.sparkleshare.SparkleShare/tmp/passwords"
17:46:45 Git | Cloning into '/home/kevincox/.config/org.sparkleshare.SparkleShare/tmp/passwords'...
17:46:46 Git | remote: Counting objects: 799, done.        
17:46:46 Git | remote: Total 799 (delta 0), reused 740 (delta 0)        
17:46:53 Fetcher | Finished
17:46:53 Cmd | passwords | git rev-parse HEAD
17:46:58 Cmd | passwords | git config core.autocrlf input
17:47:06 Cmd | passwords | git config core.quotepath false
17:47:24 Cmd | passwords | git config core.precomposeunicode true
17:47:40 Cmd | passwords | git config core.ignorecase false
17:47:46 Cmd | passwords | git config core.filemode false
17:48:01 Cmd | passwords | git config core.safecrlf false
17:48:13 Cmd | passwords | git config core.excludesfile ""
17:48:36 Cmd | passwords | git config core.packedGitLimit 128m
17:48:47 Cmd | passwords | git config core.packedGitWindowSize 128m
17:48:50 Cmd | passwords | git config pack.deltaCacheSize 128m
17:49:17 Cmd | passwords | git config pack.packSizeLimit 128m
17:49:48 Cmd | passwords | git config pack.windowMemory 128m
17:50:03 Cmd | passwords | git config push.default matching
17:50:24 Cmd | passwords | git config filter.lfs.required true
17:56:25 Git | remote: Counting objects: 1243           
17:56:25 Git | remote: Counting objects: 1507, done.        
17:56:36 Git | remote: Total 1507 (delta 267), reused 1453 (delta 258)        
17:56:51 Git | passwords | Checking for remote changes...
17:56:51 Cmd | passwords | git rev-parse HEAD
17:57:03 Cmd | passwords | git ls-remote --heads --exit-code origin master
17:57:04 Fetcher | Finished
17:57:04 Cmd | documents | git rev-parse HEAD
17:57:10 Git | passwords | No remote changes, local+remote: 5ea873fcd006380d8d9848222eb97d88631897cc
17:57:10 Cmd | documents | git config core.autocrlf input
17:57:21 Cmd | documents | git config core.quotepath false
17:57:27 Cmd | documents | git config core.precomposeunicode true
17:58:02 Cmd | documents | git config core.ignorecase false
17:58:04 Cmd | documents | git config core.filemode false
17:58:27 Cmd | documents | git config core.safecrlf false
17:59:05 Cmd | documents | git config core.excludesfile ""
17:59:34 Cmd | documents | git config core.packedGitLimit 128m
18:00:06 Cmd | documents | git config core.packedGitWindowSize 128m
18:00:34 Cmd | documents | git config pack.deltaCacheSize 128m
18:01:07 Cmd | documents | git config pack.packSizeLimit 128m
18:01:31 Cmd | documents | git config pack.windowMemory 128m
18:01:51 Git | passwords | Checking for remote changes...
18:01:51 Cmd | passwords | git rev-parse HEAD
18:02:08 Cmd | passwords | git ls-remote --heads --exit-code origin master
18:02:08 Cmd | documents | git config push.default matching
18:02:27 Git | passwords | No remote changes, local+remote: 5ea873fcd006380d8d9848222eb97d88631897cc
18:02:27 Cmd | documents | git config filter.lfs.required true
18:02:36 Cmd | documents | git config filter.lfs.clean 'git-lfs clean %f'
18:02:42 Cmd | documents | git config filter.lfs.smudge "env GIT_SSH_COMMAND='ssh -i /home/kevincox/.config/org.sparkleshare.SparkleShare/ssh/2018-10-27_16h45.key -o UserKnownHostsFile=/home/kevincox/.config/org.sparkleshare.SparkleShare/ssh/known_hosts -o IdentitiesOnly=yes -o PasswordAuthentication=no -F /dev/null' git-lfs smudge %f"
18:02:52 Cmd | documents | git rev-parse HEAD
18:03:02 Cmd | documents | git show-ref --verify --quiet refs/heads/SparkleShare
18:03:09 Cmd | documents | git checkout --quiet --force HEAD
18:03:33 Config | Wrote to '/home/kevincox/.config/org.sparkleshare.SparkleShare/projects.xml'
18:03:33 /home/kevincox/SparkleShare/bitbucket.org/documents | Initializing...
18:03:33 Cmd | documents | git config remote.origin.url "ssh://git@bitbucket.org/kevincox/documents"
18:03:42 Cmd | documents | git config core.sshCommand ssh -i /home/kevincox/.config/org.sparkleshare.SparkleShare/ssh/2018-10-27_16h45.key -o UserKnownHostsFile=/home/kevincox/.config/org.sparkleshare.SparkleShare/ssh/known_hosts -o IdentitiesOnly=yes -o PasswordAuthentication=no -F /dev/null
18:03:47 Cmd | chmod 700 /home/kevincox/SparkleShare/bitbucket.org/documents/.git/hooks/pre-push
18:04:01 Cmd | documents | git --no-pager log --since=1.month --name-status --date=iso --find-renames --no-merges --no-color
18:04:10 Git | documents | Checking for remote changes...
18:04:10 Cmd | documents | git rev-parse HEAD
18:04:10 Cmd | documents | git rev-parse HEAD
18:04:24 Cmd | documents | git config core.ignorecase true
18:04:30 Cmd | documents | git config core.ignorecase false
18:04:41 Cmd | documents | git rev-parse --abbrev-ref HEAD
18:04:45 Cmd | documents | git ls-remote --heads --exit-code origin master
18:05:02 Git | documents | No remote changes, local+remote: f038f1db9f8241a8180dc8a74cf407ccfdd6c7bf
18:05:02 Cmd | documents | git status --porcelain
tsbarnes commented 3 years ago

I'm running into the same issue, cloning a repo takes an incredibly long time, and there are no errors in the logs. I'm running KDE on ArchLinux, if that helps.