search

steadygaze / frame-randomizer

Web app to generate and guess random frames from a TV show.
GNU Affero General Public License v3.0
5 stars 3 forks source link

Feature request: Always preload the next frame in the background to eliminate visible loading times #2

Open Hexstream opened 1 year ago

Hexstream commented 1 year ago

Problem

After clicking on New Frame, there is always some loading time before the next frame shows up.

This is purely wasted time for the user and any speedrun watchers.

Solution

Always preload the next frame in the background after showing the current frame.

Additional context

Tridashie's speedrun makes this problem pretty obvious, for instance.

Obviously this would reduce "real time" but not affect "guessing time".

There should be no significant impact on the server, since the next frame will most likely end up being loaded anyway.

steadygaze commented 1 year ago

This is a good idea, and actually one person DM'd me a similar idea. However, sending the image before it's time to guess it does allow a malicious client to view the image before they're supposed to. I can think of some cheat workflows just using the devtools.

For context, the typical conversation between client and server goes like this:

  1. Client asks to be given a frame. Server picks one and responds with the path they can find it at on the server.
  2. Client requests and loads the frame (as an image).
  3. Client presents a guess. Server responds with the correct answer.
  4. Repeat, ad infinitum.

This illustrates that there are two competing goals: responsiveness versus security/preventing cheating. Naturally, the fastest way to implement things would be to preload frames, plus send the answer along with the image and do answer validation client-side. Now that there is a concept of a "practice mode" versus a verified run, we could actually do that in practice mode without worrying too much.

I also think we can actually give the client the path of the next frame with the answer even in verified mode, but just not load it until it needs to be shown, and we can verify when the image is loaded in the server-side run data. On the other hand the latency win wouldn't be as big, so I could see the argument for always preloading, and making that anti-cheating compromise.

I also thought about encrypting the image somehow and sending the encrypted version, while only sending the decryption key when it's requested. However, this would use extra compute resources (both client-side and server-side), and decryption time would eat into the hypothetical gain in perceived responsiveness. (Not to mention being harder to implement and understand.)

Hexstream commented 1 year ago

However, sending the image before it's time to guess it does allow a malicious client to view the image before they're supposed to.

Actually, I'm not sure that being able to see the next image along with the current one would be a significant advantage. It seems to me that the brain capacity to process each image would be split in half, or worse. It's probably faster to concentrate on one image at a time.

And you couldn't even submit the answer for the next image before the current one, although the malicious client could fix this by making sure to send the answer for the next image only after the answer for the current one, but then the server could flag too-fast answers as suspicious, which would then imply that the malicious client would have to add a randomized delay to the next answer when it's entered before the current answer...

(In more high-profile events, there is usually a live streaming requirement anyway...)

All in all, regarding this feature, I'm much more concerned about the guaranteed significant improvement for the vast majority of normal users than the potential insignificant gains for a few cheaters.

I can think of some cheat workflows just using the devtools.

Could you elaborate?

Client presents a guess. Server responds with the correct answer.

This seems just plain bizarre and incorrect if you are concerned about cheaters.

So a malicious client just has to submit a random almost-certainly-incorrect answer to be presented with the right answer? Why not just validate the answer server-side and return true or false?

edit:

I also thought about encrypting the image somehow and sending the encrypted version, while only sending the decryption key when it's requested. However, this would use extra compute resources (both client-side and server-side), and decryption time would eat into the hypothetical gain in perceived responsiveness. (Not to mention being harder to implement and understand.)

Great idea, actually, and modern cryptography is probably WAY more efficient than you realize.

I think the impact on the server would be pretty minimal, the main cost would be implementing it.

(As for the client, we're probably talking less than a millisecond of overhead per image decryption, saving several seconds of user time in aggregate. No-brainer!)

Just implementing it normally without cryptography would already be a great boon, though.

steadygaze commented 1 year ago

This seems just plain bizarre and incorrect if you are concerned about cheaters.

So a malicious client just has to submit a random almost-certainly-incorrect answer to be presented with the right answer? Why not just validate the answer server-side and return true or false?

So, in the current form of the game, you only get one guess. There is no concept of multiple guesses, or indicating when you've given up. The only way to get the answer from the server is to submit your final answer first. If you try the game or review Tridashie's footage, this should become clear. Additionally, there are two modes: "practice" mode and verified run mode, where all actions like correct/incorrect answers are logged server-side. So in verified run mode, we remember on the server-side if the client submitted an incorrect guess.

I can think of some cheat workflows just using the devtools.

Could you elaborate?

Alright, so consider if we send the image early. The preloaded image will show up in Chrome/Firefox devtools if the Network tab is open, and the cheater can view a preview of the image.

image

Even if livestreaming, the cheater could place the devtools on a second monitor (or not share their whole screen) so that they are out of view. They can also not include their cursor in the video, which is simply toggleable with a checkbox in OBS, or use Alt-Tab/arrow keys in a desktop environment that doesn't show any previews of windows when Alt-Tabbing. Furthermore, the cheater can do this while the "guess" timer is not running. Once they figure out the answer, they can simply show the frame normally, and then submit the answer as fast as they can type it. (The "guess" timer instead of the "real time" timer is the important one for speedrunning.)

I suspect there are even better workflows with other tools (not to mention if a cheater were dedicated enough to make their own or mod the client, though I suspect the game is not popular enough for that to be likely).

All in all, regarding this feature, I'm much more concerned about the guaranteed significant improvement for the vast majority of normal users than the potential insignificant gains for a few cheaters.

Agreed. So, referring to the previous idea of "practice" mode (the one most casual users will use), we can simply send everything to the client in this mode, and only care about cheating in the verified run mode. Additionally, the one optimization I suggested previously will at least save one network round-trip for each frame.

Actually, I'm not sure that being able to see the next image along with the current one would be a significant advantage. It seems to me that the brain capacity to process each image would be split in half, or worse. It's probably faster to concentrate on one image at a time.

Agreed, but my main concern isn't the ability to think about two frames at once. See what I said previously about the guessing timer.

Hexstream commented 1 year ago

So, in the current form of the game, you only get one guess. [...]

Haha, sorry, I had totally missed that...

Furthermore, the cheater can do this while the "guess" timer is not running. Once they figure out the answer, they can simply show the frame normally, and then submit the answer as fast as they can type it. [...] Agreed, but my main concern isn't the ability to think about two frames at once. See what I said previously about the guessing timer.

Right, sorry, I had missed that too...

Although, if you can make frame loading effectively instantaneous thanks to preloading, then you no longer need to differentiate between real time and guessing time in the verified speedrun case, right? (Assuming you don't really care about letting users take small pauses during a speedrun.)