ZeroK-RTS / Zero-K-Infrastructure

Website, lobby launcher and server, steam deployment, .NET based tools and other vital parts of Zero-K infrastructure
GNU General Public License v3.0
53 stars 52 forks source link

Replays through site aren't accessible (only affects Linux systems?) #2945

Open completesomething opened 1 year ago

completesomething commented 1 year ago

On clicking "Watch Replay Now":

Screenshot_20230530_065015

I'm running the Steam Linux version natively, NOT through Proton. Supposedly works for Windows users from what I've read. Distro is Debian sid. Browser is Firefox.

Anarchid commented 1 year ago

Here's the text of the actual error message in console:

Zero-K.exe Information: 0 : Chobbyla << DownloadFile {"Name":"20230529_194021_ZeroWars v2.1_105.1.1-1696-g311f4e2 BAR105.sdfz","FileType":"DEMO"}
Zero-K.exe Error: 0 : Error downloading file 20230529_194021_ZeroWars v2.1_105.1.1-1696-g311f4e2 BAR105.sdfz : System.UriFormatException: Invalid URI: The format of the URI could not be determined.
  at System.Uri.CreateThis (System.String uri, System.Boolean dontEscape, System.UriKind uriKind) [0x0007b] in <33b19a7ad5234d94abf4fd9b47566616>:0 
  at System.Uri..ctor (System.String uriString) [0x00014] in <33b19a7ad5234d94abf4fd9b47566616>:0 
  at PlasmaDownloader.PlasmaDownloader.GetResource (PlasmaDownloader.DownloadType type, System.String name) [0x001b3] in <b07a70ad4ea143f08572dfed5925588a>:0 
  at ChobbyLauncher.ChobbylaLocalListener.Process (ChobbyLauncher.DownloadFile args) [0x00046] in <b07a70ad4ea143f08572dfed5925588a>:0 
Anarchid commented 1 year ago

The line that throws this message is here: https://github.com/ZeroK-RTS/Zero-K-Infrastructure/blob/0d19ce8ea5a5a7d96af5f8af0c9858195cd632fd/Shared/PlasmaDownloader/WebFileDownload.cs#LL33C42-L33C45

M5PR commented 1 year ago

Do you have logs of the lines before? Either that is an invalid URL and does not completely match the Windows version, or the validation is done differently by the installed runtime framework dependencies.

Anarchid commented 1 year ago

Either that is an invalid URL and does not completely match the Windows version, or the validation is done differently by the installed runtime framework dependencies.

Likely latter, since Linux has to use Mono.

Anarchid commented 1 year ago

Do you have logs of the lines before?

Zero-K.exe Information: 0 : Chobbyla << DownloadFile {"Name":"20230604_111825_Icy Crater v4_105.1.1-1774-g5e966db BAR105.sdfz","FileType":"DEMO"}
Zero-K.exe Error: 0 : Error downloading file 20230604_111825_Icy Crater v4_105.1.1-1774-g5e966db BAR105.sdfz : System.UriFormatException: Invalid URI: The format of the URI could not be determined.
  at System.Uri.CreateThis (System.String uri, System.Boolean dontEscape, System.UriKind uriKind) [0x0007b] in <33b19a7ad5234d94abf4fd9b47566616>:0 
  at System.Uri..ctor (System.String uriString) [0x00014] in <33b19a7ad5234d94abf4fd9b47566616>:0 
  at PlasmaDownloader.PlasmaDownloader.GetResource (PlasmaDownloader.DownloadType type, System.String name) [0x001b3] in <b07a70ad4ea143f08572dfed5925588a>:0 
  at ChobbyLauncher.ChobbylaLocalListener.Process (ChobbyLauncher.DownloadFile args) [0x00046] in <b07a70ad4ea143f08572dfed5925588a>:0 

Maybe it doesn't compose the URI / doesn't add the scheme/host, hm. Realistically though the launcher should do it.

M5PR commented 1 year ago

I mean the URL that is logged in the line just before that. It's a complete URL.

The caller of Chobbyla composes a filename by adding and removing directory naming for linux and windows systems. That can contain side-effects in the filename-generation, particularly for hidden leading and trailing characters. Amplified by the deserialization of the JSON.

If we assume this is exactly the same URL, likely Linux has a different version of System.Runtime, if the Mono-library has a higher version fi., a different set of validation rules are given through a newer Internet standard. From >4.5 there is additional encoding and parsing rules given:

In .NET Framework 4.5 and later versions, IRI is always enabled and can't be changed using a configuration option. To enable support for IRI in .NET Framework versions prior to .NET Framework 4.5, set a configuration option in the machine.config or in the app.config file. Specify whether you want Internationalized Domain Name (IDN) parsing applied to the domain name and whether IRI parsing rules should be applied

After the directory comes the following:

_20230427_005608_LimeQuat Republic v1105.1.1-1696-g311f4e2 BAR105.sdfz

The substitution and escaping of reserved characters is annoying by itself. There may now be different versions for which one applies the default configuration of applying IDN parsing and IRI parsing rules.

The scheme is resolved through the given string. And it obviously does start with "https://", so likely the cause is in the 'filename' part.

I think Uri#EscapeDataString should only be applied for that filename-part, not the entire URL, actually.

In general the entire system is leaky due to it running over a filesystem-like service. The interface of finding and downloading files should be abstracted and contain appropriate args: identifier, (maybe) name, map-type, data, version, etc., available as meta-information (can be queried). So a rest-service with get-replay(args) and get-replays(args).

The actual retrieval is then done server-side by a singular identifier from mem/database or a fileserver. But most importantly, it does not matter to the user. If you want the information to be easily human readable on the client side, via the filesystem, unpackage the replays and split the meta-data from the contents.

M5PR commented 1 year ago

Having the filename be crucial yet complex can also lead to issues in validating that the file exists on the client side.

While the meta-information about version etc. is already included in the file itself.

M5PR commented 1 year ago

Anyway, I suggest building in the escapeBlabla over the filename. Add additional error-handling and logging, so even if it doesn't work, it leads to a solution.

Maybe check if there are differences in builds regarding the System.URI. Overall, try to ensure builds are similar in the future by aligning all configured dependencies to a similarly resolved versioning.

Also, do set the domain from which to retrieve replays to a constant.

The entire system is up for refactor if you ask me, but I'm n00b at c#,.net.

Anarchid commented 1 year ago

Your fix attempt would be likely better than mine at this point; deferring

M5PR commented 1 year ago

I have setup wsl2 with an Ubuntu system and can confirm there is a bug. An aggressive Mono-upgrade lead to a heavy-rain of errors. The .net framework and it's weird copy Mono are good reminders why important things don't run on that.

Licho1 commented 1 year ago

Do you know what characters cause problem?