When searching for packages, results are slow to appear

vertigo220 commented 6 years ago

Loading search results take a very long time. In fact, it's unusably slow, and significantly slower than Synaptic and Mint package managers and apt. The only reason I installed as many programs with it as I did was so I could thoroughly test it out, but it took hours (in fact, over a day IIRC) to install the ~120 programs I did. Here's my findings and thoughts on the matter:

I ran some tests, timing how long it took. The first search I performed, after not using Chocolately for a couple weeks or so, took 8:53 (8 minutes, 53 seconds). Looking at the console output, it was "fetching packages" and going through the list of my installed programs. I'm not sure if all ~120 were listed, but it went through quite a few. AFAICT, it found 6 updates. Nine minutes is an awfully long time for that. I'm not sure if it actually downloaded the updates, which would at least help explain the long time, but if it did, then that's part of the problem, since it should do that in the background, not make me sit there waiting.
Speaking of having to sit and wait, it shouldn't be going through my installed programs when I'm trying to perform a search. It should do what I'm asking it to do at the time: look for programs, not work on the ones I have. And it certainly shouldn't recheck them every, single time I do a search or go to the next or previous page of results.
After that, I did another search. This time, the console output showed more or less the same thing, but it "only" took 1:18. Another test took 1:54. It then said there was an update for Chocolately, so I updated it, restarted it, and did another "search," (see next point for why I quoted this) this time taking 1:07 (I'm not sure if it was faster this time due to the update or because it had already got a lot of the stuff that required more time out of the way, but that's still far too long).
Another issue, which is mainly an issue because of how long searches take, is that when clicking "chocolatey" in the left pane (below "This PC"), it automatically performs a search. So some of the "searches" I was doing and timing, and far too many "searches" I did as I was installing software and testing things out, were simply due to this. It shouldn't automatically start searching when you click "chocolatey," especially if it's going to be so slow. It should just load a blank screen and let you search for what you actually want right away. If anything, have it do the default search the first time and that's it.
It should also cache results and have back/forward buttons to allow for quickly going back and forth, which would help at least somewhat with the agonizingly slow searching. I realize the cache would have to be time-limited, to avoid installing outdated packages, but even 15-30 minutes would be very helpful, though it could easily cache them for a few days or more and simply fetch updated information for packages as you select them.
Going to the next/previous page also takes a long time (not quite as long, but still probably half a minute or so). It should pre-load the next page and keep previous pages cached.

As you can see, with an average of ~1:30 per search, considering one search per installed application, plus the forced blank searches, plus having to search for some multiple times, partly due to no back/forward or cache feature, partly due to figuring stuff out as I went, and partly due to other issues, you can figure I wasted probably close to 3.5 hours just waiting in order to install the programs I did. That's prohibitively slow and a very serious problem. I love the idea of Chocolatey, so I really hope this can be resolved.

This is on Win10x64, btw. Also, the search is much faster using the CLI, so it's a GUI-related issue.

gep13 commented 5 years ago

@vertigo220 I have been making some changes in this area, and I would be curious if you can see any improvement. If you are interested in taking a look, the related issue is this one https://github.com/chocolatey/ChocolateyGUI/issues/525 and instructions for how to install the pre-release version are here: https://github.com/chocolatey/ChocolateyGUI#installation

gep13 commented 5 years ago

There will also likely be improvements once we update to this: https://github.com/chocolatey/ChocolateyGUI/issues/670

opusforlife2 commented 5 years ago

What's the status on this? Searching is still slow, and #670 is done with. @gep13

taotien commented 5 years ago

This is still a problem. Opening up https://chocolatey.org/packages is way faster

rwperrott commented 4 years ago

Definitely annoying, including, astonishingly, no, so-far identifiable, cancel button for the very slow search, for: no search term, the wrong search term, or a stale search term, is blocking a new search...

BananaAcid commented 4 years ago

what triggers me, is having to wait for the main tab to do its stuff and wait, just to be able to select the "chocolatey" tab and wait again to be able to search and then wait again. I do not care for prefilling the search list - I want to go and search for packages. (Too many steps to wait for - why is there no cancel button?)

Is the wait maybe related to the .NET-No-Proxy-Configured-Delay-Bug? (webClient.Proxy = null fix)

Btw, the loading could be done async (Task.Run() !) to not block the UI.

gep13 commented 4 years ago

@BananaAcid @rwperrott @opusforlife2 @taotien can any of you please confirm what version of Chocolatey GUI you are using?

Can you please try the latest pre-release version of Chocolatey GUI, installation instructions on the readme of this project, and report back if there is any improvement.

BananaAcid commented 4 years ago

got any command line snippet I can throw at powershell to install that specific version?

Have v0.16.0 running

gep13 commented 4 years ago

@BananaAcid yip, in the readme that I mentioned...

https://github.com/chocolatey/ChocolateyGUI/blob/develop/README.md#installation

BananaAcid commented 4 years ago

Ok thanks, installed it from there. I see, initially opening it is quick - while taking a while in the background to check for new versions. Like it. But clicking on the online tab ('chocolatey' tab) blocks and requires a wait, before being able to search.

opusforlife2 commented 4 years ago

@gep13 Does the pre-release version install over the stable version or does it install separately? If it does install over the existing installation, does it get updated back to the stable branch on the next release?

gep13 commented 4 years ago

@BananaAcid said... But clicking on the online tab blocks and requires a wait, before being able to search.

Can you confirm what you mean by the online tab? By default, there is no tab called online in Chocolatey GUI. I want to make sure that we are talking about the same thing.

gep13 commented 4 years ago

@opusforlife2 said... Does the pre-release version install over the stable version or does it install separately? If it does install over the existing installation, does it get updated back to the stable branch on the next release?

Yes, the pre-release version replaces the currently installed stable version of the application. When 0.17.0 is released, it will install over the top of the pre-release version, so it should be a seamless transition when it is released.

opusforlife2 commented 4 years ago

@gep13 Thanks. Installed the pre-release version. As @BananaAcid said, it is immediately noticeable that the startup time has been cut down considerably.

I'll phrase the same issue differently for clarity: When you click on the 'chocolatey' tab (the only one apart from 'This PC'), the application immediately blocks the user from doing anything else by showing a big box, with the optional console output saying it is fetching packages. This process can go on for several minutes, and is completely pointless 95% of the time, because there is a very small chance of the user finding the software they want on the very first page of results that is loaded.

Can this process not happen in the background? The priority should be given to the user being able to search for the package they want.

A further issue as mentioned by @rwperrott is that once a search is executed... it will go on and on, regardless of what the user does, for several minutes until it either succeeds or fails. This means that if you entered the wrong spelling by mistake, you're in for a long and annoying wait.

So there should be a way to cancel a search as well.

gep13 commented 4 years ago

@opusforlife2 so, it sounds like there are several moving parts here, and I think it would help to have these broken up into separate issues for tracking purposes. Based on what has been said here, I see three issues/concerns...

(This issue) Queries to chocolatey.org take a long time to return
Feature Request: Ability to cancel an ongoing search
Feature Request: Provide the ability to not populate the package results when navigating to a source tab

@opusforlife2 would you, or someone else in this thread mind taking the time to create those two feature request issues?

Now, regarding it taking minutes to populate that first screen when clicking on the chocolatey tab, I want to dig into that a little more. I have just done the same, using Fiddler to capture all the outgoing requests, and it took a total of 2.348 seconds to complete.

Would someone in this thread be able to provide a similar screen shot with timings of each request? I would like to know what is taking a long time to complete.

opusforlife2 commented 4 years ago

@gep13 Done! :)

gep13 commented 4 years ago

@opusforlife2 thank you for doing that, I appreciate it!

vertigo220 commented 4 years ago

Started playing around with chocolatey again in the hopes I'd be able to use it on some new builds, but this is still a problem. Switching to the chocolatey "tab" still takes a while to load, as do searches. When viewing the command line output, both actions are doing the same thing: listing many/all of the installed apps, like this:

Fetching packages... Package name/installed ver/new ver/False . . .

And it does this every time, even if I perform the same exact search immediately after.

I am, however, able to switch to the chocolatey tab right away after opening the GUI.

gep13 commented 4 years ago

@vertigo220 Can you confirm exactly what version you are using? Based on your other comments, I suspect you are still using 0.16.0.

vertigo220 commented 4 years ago

You're right. I was wondering why it hadn't been updated in so long, didn't realize you had a separate development version. Just tried upgrading but it failed. Typed the following into an admin PowerShell console:

choco upgrade chocolateygui --source https://www.myget.org/F/chocolateygui/ --pre

and got this:

Chocolatey v0.10.15
Upgrading the following packages:
chocolateygui
By upgrading you accept licenses for the packages.

You have chocolateygui v0.16.0 installed. Version 0.17.0-unstable0318 is available based on your source(s).
chocolateygui not upgraded. An error occurred during installation:
 Unable to resolve dependency 'dotnet4.5.2 (≥ 4.5.2.20140902)'.
chocolateygui package files upgrade completed. Performing other installation steps.
The upgrade of chocolateygui was NOT successful.
chocolateygui not upgraded. An error occurred during installation:
 Unable to resolve dependency 'dotnet4.5.2 (≥ 4.5.2.20140902)'.
 Unsuccessful operation for chocolateygui.
  Rollback to previous version (package files only)?([Y]es/[N]o): y

Chocolatey upgraded 0/1 packages. 1 packages failed.
 See the log for details (C:\ProgramData\chocolatey\logs\chocolatey.log).

Failures
 - chocolateygui (exited 1) - chocolateygui not upgraded. An error occurred during installation:
 Unable to resolve dependency 'dotnet4.5.2 (≥ 4.5.2.20140902)'.

I have ver 4.8.03752 of dotNET installed.

mkevenaar commented 4 years ago

@vertigo220 add ;chocolatey to the end of your source parameter. Assuming you have the default sources available

vertigo220 commented 4 years ago

So just go to Settings > Sources and put that in the ID box and save?

gep13 commented 4 years ago

@vertigo220 no, @mkevenaar is suggesting that you try the following:


choco upgrade chocolateygui --source https://www.myget.org/F/chocolateygui/;chocolatey --pre
``

LordMike commented 4 years ago

@gep13 which means

choco upgrade chocolateygui --source "https://www.myget.org/F/chocolateygui/;chocolatey" --pre

(quotes around the source)

gep13 commented 4 years ago

@LordMike Ah, that is a great catch! Thank you!

LordMike commented 4 years ago

Also .. Updating to 0.17.0-unstable0318 reduces initial load time for the feed (chocolatey) immensely for me. I'm down to ~2 seconds where I was at 8-10s+.

vertigo220 commented 4 years ago

Ok, now updated, and I realize I should have done some timed tests beforehand for comparison, but so far what I've noticed is that initial startup actually seems a tad slower (first time took ~10-20 seconds, don't remember exactly, but that's to be expected; subsequent startups take ~8 seconds, whereas before it was only probably 2-3, but still not a big deal) and, more importantly, switching to the chocolatey tab as well as performing searches each take ~8-10 seconds, which is definitely faster, though still seems like it could/should be faster still. The same goes for switching pages, which definitely should be faster, since it could easily preload the next 2-3 pages so they're ready right away, i.e. initially load page one then as soon as it's displayed load pages 2-3 or 2-4, then when page 2 is accessed load page 4/5, and so on. The console readout isn't going through all the installed packages anymore, instead just saying "Fetching packages..."

Arcitec commented 4 years ago

Dear God @gep13 please release a new version of Chocolatey GUI. The last release was two years ago in february 2018 and is unusably slow. :-O I have tested the 0.17 version above and it is very usable and fast enough to not be annoying. Pretty much the same speed as running "choco outdated" and "choco search" on the command line. The old 0.16 version just hangs for minutes and I was going to uninstall the GUI to never use it again. Luckily I found this ticket with the 0.17 pre-release...

opusforlife2 commented 4 years ago

@gep13 I have to agree with this comment. The difference between 0.16 and 0.17 is like night and day. The entire process of opening it, waiting for it to refresh, then updating software never took less than 15 minutes earlier. Now it is done under 5 minutes.

opusforlife2 commented 4 years ago

A heads up for people in this thread: I found that the 0.17 pre-release has a toggle to prevent a search from executing until you press Enter. Enjoy!

vertigo220 commented 4 years ago

@opusforlife2 I assume you're talking about the delayed search option, which isn't new; it just has a description now to explain what it does, so that may be why you didn't notice it before.

opusforlife2 commented 4 years ago

Blast! Foiled again.

vertigo220 commented 4 years ago

So I just used chocolatey to set up a new computer--something I'm hoping I don't regret due to the problems mentioned in the pinning issue--and this slowness made it a major PITA. After spending an entire day installing programs and being very frustrated at how long everything was taking, I finally thought to switch to the command line and that was much, much faster. When installing programs through the GUI, searching would take several seconds at least, which seemed to grow longer when I had a fair number of apps installed, now taking 15-20 seconds. This is pretty brutal, and a massive waste of time. Doing the math, that means I spent 1.5 (one and a half!) hours just waiting on searches.

And that's not even the worst part. When actually installing software, it took forever every single time, with a couple (Activeperl and, I think though I could be wrong, Android SDK Platform Tools) taking so long I thought chocolatey had stopped working. Just guessing, I'd say those two took at least 15-20 minutes and the others mostly took between 3 and 10 minutes (and typically more like 5-10).

It also isn't an issue of download, because they would download fairly quickly but then still take forever to finish installing. And when I switched to using the command line, searches take more like 3-5 seconds (so significantly faster there) and installs are also much, much faster. Of course, I didn't compare the speed of the same program done each way, but I did enough each way to be able to say that there's a definite, huge difference. What took me an entire day with the GUI would have taken 1/3 of that time, or less, had I just used the command line the entire time. So while the changes seem to have helped some, there's clearly much more needed to bring the two in line with each other.

Another major annoyance with the slowness of the GUI is that it takes a few minutes once launching chocolatey to even be able to view the installed programs. It should display the list immediately, since it should always know what's installed. I wonder if it's checking for updates first, but that doesn't seem quite right since when the list finally shows it's still checking. So I'm not sure what the deal is with that, but something that should be very quick--launching chocolatey and scrolling or typing to search to see if a program is installed in chocolatey--takes far too long.

And switching to the repo (from 'this PC' "tab" to 'chocolatey' "tab") is still frustrating, since it still automatically loads some generic results (and no idea what's so special about those ones), making the user wait to be able to search for what they actually want.

gep13 commented 4 years ago

@vertigo220 I am sorry to hear that you have had a back experience with Chocolatey GUI. This:

@vertigo220 said... Another major annoyance with the slowness of the GUI is that it takes a few minutes once launching chocolatey to even be able to view the installed programs.

Is certainly not my experience. And it certainly shouldn't be taking this long. Are you in a position to provide more information about your machine and the configuration of Chocolatey? For example, what are the specs of your machine? Do you have any virus scanning software active on the machine? How many sources do you have configured in Chocolatey? How many packages do you have installed? What is the network connection speed that you have?

@vertigo220 said... And switching to the repo (from 'this PC' "tab" to 'chocolatey' "tab") is still frustrating, since it still automatically loads some generic results (and no idea what's so special about those ones), making the user wait to be able to search for what they actually want.

You may be interested in the new Prevent Preload feature which has just been added into the develop branch. https://github.com/chocolatey/ChocolateyGUI/issues/706 This stops the fetching of the first set of results when navigating to a source, and instead allows you to search directly for a package you are interested in.

opusforlife2 commented 4 years ago

With the latest release, I have no more issue regarding slowness of fetching results or checking for updates. All that's left for me is the GUI blocking windows that I have talked about in #720.

gep13 commented 4 years ago

@opusforlife2 said... With the latest release, I have no more issue regarding slowness of fetching results or checking for updates. All that's left for me is the GUI blocking windows that I have talked about in #720.

Glad to hear. That issue is on my list to get completed, so hopefully that will be soon.

BananaAcid commented 4 years ago

with 0.17.0 it still gets stuck a long time on the 'chocolatey' tab with 'Fetching packages... ' ... no chance to use the search ...

still SUPER frustrating. Why do I need a list of "Popularity" ordered Items? Would be nice to configure it to NONE if that would help to boot it up.

Whatever button I did press, it does now hang on 'Loading page 1...' forever ... on 400Mbit/s connection ...

gep13 commented 4 years ago

@BananaAcid see the comment above...

You may be interested in the new Prevent Preload feature which has just been added into the develop branch. #706 This stops the fetching of the first set of results when navigating to a source, and instead allows you to search directly for a package you are interested in.

I believe this is what you are looking for.

vertigo220 commented 4 years ago

@gep13

I just launched it on my computer, and it was pretty quick. My computer is well-spec'd, with an i7-7820, 64GB of RAM, and a 1TB 960 EVO. The computer it's slow on, one that I was setting up for my parents, is much slower, with an AMD 5350, 8GB of RAM, and a slower SSD. As for anti-malware, it does have a few different things running, but the only one that's "active" is Win Defender. The computer is noticeably slower on most things than mine and even another computer with similarly old but faster hardware that I'm also working on, so that certainly could be a contributing factor. Nonetheless, chocolatey should, ideally, be able to perform well on even slower systems. Perhaps it could run a background service to monitor things so when it's run, everything is there, ready and waiting. At the very least, it should display the list of installed software, which it should be able to do within a couple seconds even on the slowest of computers, before doing any startup checks it does on the installed software. Very often I launch it simply to check if a program is installed through chocolatey (vs downloading and running the installer manually), and so all I need is to see the list of installed apps, yet it takes a significant amount of time for that to become available.

chocolatey / ChocolateyGUI

When searching for packages, results are slow to appear #595