Prune server downloads and lua cache

[aStonedPenguin]

It's always been an ongoing joke how large Garry's Mod installs can get over time. This leads people to uninstall in search of freeing space. That is undoubtedly is not positive for the game. While you can run the factory reset batch file in the install folder or prune these yourself that is not a user friendly solution any significant portion of people will do.

I suggest touching all downloads when joining a server and then pruning them when they go unused for a period of time. This would not only reduce space usage but reduce the chance of conflicting content.

[thegrb93]

Just a note on the last comment, unused gmas don't conflict. The bug where gmas that are mounted persist across sessions can though but that's a different open issue.

[thegrb93]

Unless you're talking about the builtin source downloads, in which case, yeah those are dumb as balls.

[aStonedPenguin]

The conflict comment only refers to fastdl.

[Kefta]

Automatically deleting content is definitely not gonna happen, but I assume aside from the memory issues, your main concern is the file conflicts due to FastDL between different servers. Perhaps FastDL could be updated to a system where each server has its own subfolder inside downloads/ with its own unique content that doesn't clash. The general approach I see would be:

• Server sends resource file list w/ checksums
• Client iterates over that file list to see what it doesn't currently have w/ that matching checksum. During this process, it could iterate over all servers' individual downloads/ folders to check for content that has already been downloaded before. How deduplication would work (not copying the same file to every server folder) in this manner and how efficient this process would be are the biggest concerns here
• Client either requests each file it doesn't have (what FastDL currently does) or sends the modified file list back to the server so that it can send it in some sort of batch

This sort of approach also tackles https://github.com/Facepunch/garrysmod-requests/issues/963, but really, the core of this issue is handling file naming conflicts.

As for the memory issues, perhaps an option could be added to the menu to delete specific or all downloads/ content like it exists for GMAs now.

[JShorthouse]

Automatically deleting content is definitely not gonna happen

What exactly would be wrong with this? IMO storing server content locally is just a caching system and should be treated as such. You wouldn't expect your browser's cache to balloon to 200GB worth of content from every website you've ever visited so I don't see why gmod should behave like this either.

[viral32111]

Or perhaps at the very least, give user's an easier way to clear the server downloads/cache like what was originally said. Just like the web browser analogy, when you clear its cache all you usually have to do is go into the browser settings. You don't have to navigate to the browser's local data folder and delete lots of random files/folders. If the seperated server content is going to be a thing, then give the user the choice of easily deleting content for a specific server that they know they don't play on anymore. A small button in the options dialog possibly under the Multiplayer tab would do just fine for this, it doesn't have to be anything intrusive like right on the main menu.

I can't think of anything wrong with automatically deleting downloaded server content from a server you haven't played on in over a year.

[Kefta]

I know many such cases where users dive into their downloads/ folder to find content they downloaded from a server years ago, or people that collect models/maps/etc. from all the servers they visited. Hell, I've done this. As I already said,

As for the memory issues, perhaps an option could be added to the menu to delete specific or all downloads/ content like it exists for GMAs now.

which just sounds like what you're suggesting @viral32111.

[viral32111]

I know many such cases where users dive into their downloads/ folder to find content they downloaded from a server years ago, or people that collect models/maps/etc. from all the servers they visited.

Then simply have a togglable option for if the content should be automatically pruned, heck even an entry for the time would be great too. Users that play on alot of different servers throughout the month could have it set quite low, while users who rarely play on more than 2-3 servers can have it quite high or disabled outright. Again, this would probably go best in the multiplayer tab on the options dialog.

That toggleable option should be enabled by default, as I doubt the majority of the GMod userbase is diving into their downloads folder scavanging for server content.

[thegrb93]

What's the big deal? shift+del the download folder when your HD gets full; no reason to make things complicated.

[viral32111]

There is no big deal, it's easy to do that, but the majority of players won't ever go anywhere near the game install folder as it isn't something most games expect you to do.

While you can run the factory reset batch file in the install folder or prune these yourself that is not a user friendly solution any significant portion of people will do.

Or perhaps at the very least, give user's an easier way to clear the server downloads/cache like what was originally said.

[JShorthouse]

I know many such cases where users dive into their downloads/ folder to find content they downloaded from a server years ago, or people that collect models/maps/etc. from all the servers they visited

Just because people currently do this it doesn't mean it's the way it should work. If browsers were made so that the cache never cleared itself I'm sure many people would also enjoy looking through their browser cache folder to find memes they looked at 5 years ago.

But that isn't how browser cache works because browser developers realise that filling up peoples harddrives with 200GB of cache would be an awful idea.

Do you hear people complaining that their browser clears it's cache and "loses" all of the images they've looked at? No, because people understand that they need to manually save things if they want to keep them.

[JShorthouse]

Then simply have a togglable option for if the content should be automatically pruned, heck even an entry for the time would be great too

I think this is the best option. While I'm sure there is a legitimate use case for users who purposefully want to keep a copy of every single file they have already downloaded that would make this feature an archive, not a cache.

Is the intention to allow clients to cache content, or to force everyone into keeping an archive? I would guess that 95%+ of players do not care about their downloads folder or even know of its existence. All they see is gmod ballooning to 200GB on their harddrive.

[Kefta]

Then simply have a togglable option for if the content should be automatically pruned, heck even an entry for the time would be great too

I can get behind that. Whether it should be on by default or not is the bigger question here, and what the default time/space should be.

Is the intention to allow clients to cache content, or to force everyone into keeping an archive?

Both. As the downloads/ folder has never been cleared by default in any Source game, it seems to function more as an archive by default.

[JShorthouse]

If you want a softer approach then maybe a good compromise would be to keep the current archival behaviour by default but to add a "Manage Content" menu item / icon that allows users to enable pruning but also provides a clear visual notification when their downloaded content exceeds a certain threshold (say 25GB). For example, a red "!" could appear next to the menu item, which when clicked on would then explain "Your downloaded content is taking up over 25GB of harddrive space, would you like to enable pruning?".

This way the default behaviour isn't changing but users are given clear warning about the size of their downloads folder along with the tools to reduce its size if they so wish.

[robotboy655]

but also provides a clear visual notification when their downloaded content exceeds a certain threshold (say 25GB).

That's fancy on paper until you realize you want the game to scan potentially thousands of files/folders in download/ folder which would be extremely slow to do on every start up or even occasionally.

[Kefta]

There could be a screen that popped up to verify deleting files so that users could prevent it before the auto-clearing occurred. After then, a progress bar could be shown to update a clearing process counter. It would be slow but have to be approved

[viral32111]

I believe what was originally said with a time based system would be better than a size threshold, as there are probably some server's that have upwards of 25GB of content ( :/ ). Users reguarly just playing on this server would then always be receiving some sort of visual indication that their game downloads is taking up too much space. Obviously they wouldn't clear their downloads, as they play on the server reguarly. The only way I'd go with this method is if it was toggleable or the threshold was user adjustable.

Just give the users a notification (or something similar) that informs them they've got old/unused server content in their downloads folder that can be safely deleted, then delete it if they explicitly click some sort of "delete content" button.

[JShorthouse]

That's fancy on paper until you realize you want the game to scan potentially thousands of files/folders in download/ folder

Fair point

A solution could be to keep a csv file of file_location, last_access_time, file_size in the downloads folder, which would make pruning a lot more efficient too. Obviously this would get out of sync if users went messing around in their downloads folder but since this feature is specifically aimed at users who aren't doing that I think this would work.

For advanced users there could be an option manually start a rescan of the whole downloads folder. Although if the pruning feature was implemented well I can't see much reason for anyone to ever manually delete things from their downloads folder anyway.