Currently the module installer downloads all of the uncached mods in a changeset before it starts installing anything. It would be more time-efficient to start installing some mods while others are still downloading, when feasible.
Currently a mod's SHA256 hash is calculated after we finish downloading it, which means we receive all of those bytes over the network, save them to disk, and then load them back from disk into memory to hash them. This can take a while for large files. It would be better to calculate the hash while the download is in progress, since we have to wait for the bytes to come across the network anyway.
Currently ckan available always lists all the available mods. If you have some idea of what you're looking for (e.g., I wanted to list the "OPX" mods for testing), it would be nice to be able to apply a filter.
Problems
The GUI progress screen's per-mod progress bars can sometimes get stuck at nearly-complete instead of disappearing when their task finishes
If you Shift-click the All filter in GUI, an exception is thrown
Mac OS has a default keybind for F10, which conflicts with ConsoleUI's use of that key
If you try to install a mod that recommends a DLC that you don't have installed, you get an error that says "CKAN can't install expansion 'Making History' for you.", even though you're not trying to install it.
Causes
The progress bars are driven by Progress<> objects, which run code on a thread pool and can be de-allocated before that actually happens if the task that created them finishes
Shift-clicking All doesn't make sense as an operation because the intersection of a set with All is the same set you started with (n.b., in contrast to the other filters, there is no tooltip advertising this key option), but there is code partially implementing it anyway. This partial implementation passes an empty list to EditModSearches.MergeSearches, which expects at least one element.
The relationship resolver throws an exception if it finds a DLC in its traversal of relationships, but as of #3917, that's how we find recommendations.
Changes
Now downloads and installs are performed in parallel. A downloaded mod will only be installed once all of its dependencies have been installed, to preserve the exFAT-friendly timestamp ordering from #3667. For upgrades, the downloads begin in parallel with removals and continue into the installs if they take that long. This should reduce the overall time required for installs and upgrades. Upon a download error or user cancellation, the installs are reverted but any completed downloads remain cached.
Fixes #3793.
The progress screen is fairly drastically refactored to be closer to how I imagined it ought to have worked all along:
Where previously only the main progress bar showed the rate and percentage in a separate label located above it, now every progress bar shows those details in an internal label.
Known issue: The internal progress bar label is not compatible with "marquee" style progress bars (where instead of reflecting a percentage completion, it just plays an animation). The label is shown for one frame of animation and then disappears. I think this is OK for now since we don't use the labels with marquee style to display actual useful information. This might be something we can address in the future with more messing around with the WinForms APIs.
The overall progress bar now combines total progress on all downloads and installs since they are no longer two separate stages
Per-mod progress bars don't disappear anymore but instead are scrolled out of view, with the active progress bars kept at the bottom of the list and the pane sized to fit them
The user can scroll through the progress bars if they wish and move the splitter that separates them from the log messages box
Previously only downloading and validating had per-mod progress bars, but now installing and removing steps also have them
Now the Progress<> objects are replaced with ProgressImmediate<> objects that run their callbacks right away instead of in a thread pool, so we no longer lose updates
Now the progress bar captions show the approximate remaining time, which we compute by dividing the remaining bytes by the average of two rates: the total bytes per second of the download since it started, and the bytes per second over the last three seconds.
Fixes #4160.
Now the ResumingWebClient's constructor accepts an optional HashAlgorithm parameter. If supplied, it will be used to calculate the hash of the incoming byte stream as received from the server. Upon completion, the resulting hash is propagated outwards to the NetAsyncModulesDownloader where it is checked against the mod's hash metadata. This eliminates hashing from the download validation step (but we still need that step for a few things like checking that the file is a valid ZIP, etc.). This should speed up installs a bit.
Now when a mod recommends a non-installed DLC, the install won't error out, and the DLC will show up in the recommendations list, as intended.
Tests are created to exercise this and catch future regressions.
Fixes #4242.
Now Shift-clicking the All filter works exactly like clicking it normally.
Fixes #4244.
Now ConsoleUI supports Ctrl+T in addition to F10 to open the menu, and the screen tip on Mac is changed from F10 to Ctrl+T.
Fixes #4248.
Now ckan available supports a globbing pattern (meaning you can use ? and * as wildcards) to filter the list of available mods
Motivations
ckan available
always lists all the available mods. If you have some idea of what you're looking for (e.g., I wanted to list the "OPX" mods for testing), it would be nice to be able to apply a filter.Problems
Causes
Progress<>
objects, which run code on a thread pool and can be de-allocated before that actually happens if the task that created them finishesEditModSearches.MergeSearches
, which expects at least one element.Changes
Progress<>
objects are replaced withProgressImmediate<>
objects that run their callbacks right away instead of in a thread pool, so we no longer lose updatesResumingWebClient
's constructor accepts an optionalHashAlgorithm
parameter. If supplied, it will be used to calculate the hash of the incoming byte stream as received from the server. Upon completion, the resulting hash is propagated outwards to theNetAsyncModulesDownloader
where it is checked against the mod's hash metadata. This eliminates hashing from the download validation step (but we still need that step for a few things like checking that the file is a valid ZIP, etc.). This should speed up installs a bit.F10
toCtrl+T
. Fixes #4248.ckan available
supports a globbing pattern (meaning you can use?
and*
as wildcards) to filter the list of available mods