secondlife / viewer

🖥️ Second Life's official client
GNU Lesser General Public License v2.1
209 stars 53 forks source link

Make the SL Viewer respect HTTP_PROXY environment variable(s) #1587

Open canny[bot] opened 3 months ago

canny[bot] commented 3 months ago

Currently, all viewers I've tested (official and TPV) support a Web proxy configuration, both for the embedded browser as well as for HTTP traffic to retrieve assets.

The latter is possibly one of those underused options of which residents are not aware. As of the time of writing, local disk-based cache allows setting apart a few dozens of Gigabytes (at best) where assets can be stored after being downloaded, to be reused as needed. But the number of textures in SL increases all the time. As such, in practice, the cache hit ratio is quite low, but that is something that nobody can be blamed for (we want* more content, not less!).

Many households, these days, employ Network-attached storage (NAS) to make backups or to hold a family's complete media library. Over the years, these have been become more and more sophisticated, run some variant of (embedded) Linux, and offer several other functions beyond merely providing shared, networked disk storage.

Most notably, they also offer HTTP(S) proxy/cache services as well. The idea is that all that cheap disk space can be effectively used for the whole family to store anything they've downloaded to their multiple devices, thus potentially cutting on bandwidth costs. Wireless devices will also benefit considerably, since the potentially wired "always on" NAS can download and store data much faster and less packet errors, and connections to retrieve Web data will just be maintained inside the home wireless network.

Some of these NAS are even embedded in other tools, such as Smart TVs, set-top boxes, or even combined routers; many might not even be aware that their computers are silently connecting to a proxy server, either locally or at least within the domain of the broadband Internet service provider — fast to access as opposed to directly connecting to the wide Internet. Others might be aware of this functionality but never bothered to activate it.

Regarding Second Life, proxying content pulled via HTTP (from the SL Grid's asset servers, for example) is extremely efficient, because all content has its own UUID, and there is no need to worry about "stale" data. UUIDs never "expire", so to speak; once downloaded, they're "eternal". Proxy servers are especially good dealing with that kind of content — as the saying goes, it's a match made in heaven!

And that difference is clearly perceptible. At my own household, we routinely use up to three different computers to access SL. All will tap the proxy server first, which has around a Terabyte of disk space set apart just to store SL content. The number of devices connecting to SL may increase once the mobile version of SL gets released, and I expect the same will be true for most residents.

If several members of a household even share communities, frequently attending events on those, then the benefits are even higher. Whoever is the "first" to download a specific access will download it to everybody else — no need to bother with having everybody downloading it over and over again. And, once downloaded, it stays downloaded — at the proxy/cache server in the local network, accessible within ~300 nanoseconds (the speed of a 1 GBps Ethernet).

Anyway, the point is that effectively using a local proxy/cache server for all household computers (and, in the very near future, mobile devices as well) is a Good Thing which is probably not deployed so widely merely because it's not really promoted anywhere.

One way of getting it to become more widely used is to have it configured automatically — because most people quickly forget what the exact URLs of their home NAS is, and so forth.

I'm not quite sure how it works under Windows — but I know that it does work! — but under Unix-inspired operating systems it's terribly easy. You just have to configure two (or potentially four) environment variables:

and set them to the proxy/cache server's URL, something looking like http://my-nas-ip.local:3128/.

That's it. Almost everything will instantly start using the configured proxy/cache server — browsers, obviously, but also other more esoteric things, such as apps "calling home" to verify if they have the latest versions or not. Command-line tools will also use the proxy as well.

Now, most of this magic comes from cURL — the popular C-based library that is ubiquitous among Unix-like environments, and which has sort-of-standardised a de facto protocol for signaling the existence of a proxy server. The Second Life viewer also uses the cURL library (at least you list them under Licenses), albeit a rather old and outdated version, but "new enough" to recognise those environment variables automatically. In other words: everything that uses cURL is automatically configured to use the proxy settings, unless you explicitly turn it off, which seems to be the case of the SL Viewer (and most TPVs I've tested).

It's true that some tools and libraries use the lower-case version of those two variables (as said, this is not an Internet standard!), which causes a problem, because environment variables are case-sensitive. The reasonable approach, therefore, is to look for all variants.

There is also a special environment variable, NO_PROXY, where people can optionally list URLs which are not supposed to go through the proxy — for instance, because their content is highly dynamic and generated on demand. As explained, SL's assets are exactly the opposite: they are "highly static" in the sense that an asset is bound to an UUID until the Cold Death of the Universe, and will not change a single bit until then.

Granted, different operating systems handle these variables differently. For example, on macOS, which is primarily GUI-oriented, the user does not need to enter the console to manually configure the environment variables to use a proxy server. Instead, that is done on the Network control panel. Things like auto-configuration (i.e. searching for a nearby proxy server using a well-known URL) are handled there as well, but the end result is that the environment variables get set for everything, instantly.

This is mostly important for those who connect to several different networks, and will be even more important with SL on mobile: as people roam around the world with their SL-compatible devices, they might use different networks, some of which automatically setting up a proxy server. The operating system knows when there was such a change of networks and self-configures itself according to where it currently is. You can override the settings, obviously. But the idea is that you configure them once and then everything will use the appropriate settings, no matter which connection you're currently using).

Linux has way too many distros for me to identify a common trend, but my guess is that, when using one the more popular GUI managers, they will have something in place similar to what macOS provides. Others rely on the user to save the variables on their own and relog for them to take effect (similar to how SL has to be restarted when there is a change).

While almost all apps making Web requests will respect those variables, the way they do it is not uniform. Chromium-based browsers, for instance, will be more than happy to do everything automatically, no tweaks necessary. Mozilla Firefox, however, is a bit different — you need to explicitly tell it to use the "system settings". If not, it assumes "No proxy".

Personally, I would suggest emulating Mozilla Firefox, simply because it's more aligned with the "opt-in to new features" policy of most LL tools/settings. Whatever the choice might be, the important bit is to understand that "system settings" essentially will read the environment variables set there (again, you will need to see how it's done under Windows — it's probably something that is stored on the system-wide Registry).

Here is the good news: since you already use the cURL library, all the above should be pretty much pre-implemented, you will only need to flip a (virtual) switch somewhere in the code, and change a bit the UI for these preferences panels. There is no need to worry about anything else; cURL ought to be ready to fill in the details, by merely calling the right function with the right parameters.

Anyway — please consider this "improvement" worthy of consideration. It's almost trivial to implement with merely a handful of lines of code...

Some additional references

https://secondlife.canny.io/admin/board/feature-requests/p/make-the-sl-viewer-respect-http-proxy-environment-variables

canny[bot] commented 3 months ago

This issue has been linked to a Canny post: Make the SL Viewer respect HTTP_PROXY environment variable(s) :tada: