Flash loop doesn't show "video" (which consists of DefineBitsJPEG3 tags) and lags the player pretty heavily

[Diatonator]

Nightly 2021-01-11 Both Chrome and Windows desktop tested.

https://z0r.de/2021

No video content here (h263 or the like). No AS3.

1) No video shown while music plays. 2) Either a Chrome tab, or a Desktop app - both hang heavily when they try to play this flash. Chrome tab can be closed only after a significant delay if this flash is being played, Windows desktop app - I had to kill it with the task manager.

DefineBitsJPEG3 tags:

Those JPEGs are pretty small as you could see from the screen. Could it contribute to the issue?

[Diatonator]

Still repro on Nightly 2021-12-30

[Diatonator]

Still freaking out on Nightly 2022-07-31

[n0samu]

This actually seems to be the same issue as https://github.com/ruffle-rs/ruffle/issues/6144#issuecomment-1051311291. I've added some trace statements to the SWF and attached it below: 2021_trace.zip Frame 1:

trace("starting movie. Frames loaded:");
trace(_framesloaded);
ifFrameLoaded(1)
{
   trace("going to frame 2");
   gotoAndPlay(2);
}
trace(this._currentFrame);
trace("going to frame 1");
gotoAndPlay(1);
trace(this._currentFrame);
trace("done with frame 1 script");

Frame 2:

trace("Now on frame 2 script. _currentFrame value is:");
trace(this._currentFrame);
stop();

Here are the traces from Flash Player:

starting movie. Frames loaded:
0
1
going to frame 1
1
done with frame 1 script
Now on frame 2 script. _currentFrame value is:
2

When the frame 1 script runs, Flash Player thinks it has zero frames loaded. So gotoAndPlay(2); doesn't execute. It executes gotoAndPlay(1);, which goes to the same frame it's already on (basically does nothing). After the frame 1 script is done executing Flash Player moves onto frame 2 and stays there, and the animation plays.

On the other hand, Ruffle repeats these same traces over and over again:

starting movie. Frames loaded:
2
going to frame 2
2
going to frame 1
1
done with frame 1 script
Now on frame 2 script. _currentFrame value is:
1

So Ruffle is jumping back and forth between frame 1 and frame 2 because when it starts frame 1, it correctly reports that it has all the frames loaded. I guess Ruffle needs to pretend that it doesn't have any frames loaded at first. But when and why exactly should it do this? I don't know.

[torokati44]

Perhaps #7858 will help with this as well, then?

[Herschel]

I think _framesloaded might actually be num_frames_loaded - 1, with a special case of setting it to num_frames when all frames have been loaded.

If I make a 4-frame SWF with a bunch of scribbles, run it with the 14.4k download simulator, and trace out _framesloaded each frame: https://user-images.githubusercontent.com/36278/188994904-0c0f5b99-82fc-4f16-a84c-3120f2643e42.mp4 It goes from 0, 1, 2, then straight to 4.

[n0samu]

Adding even more to the confusion, here is an SWF identical to the one I attached earlier, except I removed the WaitForFrame instruction: 2021_trace_unconditional.zip Now gotoAndPlay(2); always executes. Yet Flash Player still plays it fine while Ruffle gets stuck in a loop. The traces from Flash Player seem to indicate that it never actually goes to frame 2 even though it executes the gotoAndPlay? I don't understand it.

starting movie. Frames loaded:
0
going to frame 2
1
going to frame 1
1
done with frame 1 script
Now on frame 2 script. _currentFrame value is:
1

[n0samu]

Perhaps #7858 will help with this as well, then?

I just tried it and well... sort of? The first time I load a file it works because the frame isn't loaded quickly enough to satisfy ifFrameLoaded. But Ruffle gets stuck in a loop as usual when loading the file subsequent times.

[Herschel]

Adding even more to the confusion, here is an SWF identical to the one I attached earlier, except I removed the WaitForFrame instruction: 2021_trace_unconditional.zip Now gotoAndPlay(2); always executes. Yet Flash Player still plays it fine while Ruffle gets stuck in a loop. The traces from Flash Player seem to indicate that it never actually goes to frame 2 even though it executes the gotoAndPlay? I don't understand it.

This makes sense given the values of _framesloaded. You can only ever go to a frame that has been loaded -- trying to go farther will stop at the last loaded frame. Since _framesloaded is initially 0 for this SWF, the gotoAndPlay(2) can't get to frame 2 and will stop at frame 1.

So the next question is why is _framesloaded 0 immediately for this SWF, but not for other SWFs. For example, if I make a blank SWF with two frames, trace _framesloaded is 2 immediately.

This is definitely along the lines of #7858, where it's affected by how Flash Player decides to chunk the loading. Specifically, often having embedded image (DefineBits image tag) adds a chunk point where the loading will "yield" for a frame as the image is decoded, and _framesloaded will be the earlier frame. So in my earlier example of a 2 frame SWF, if I put an image on frame 2, _framesloaded is then 0 immediately. If I were to put that image on frame 3 instead, _framesloaded would be 1 immediately.

frames-loaded-2-frame.zip

The scary thing is that it's semi-random, depending on the size/complexity of the assets in some way. If I include a 1x1 image instead of a larger one, _framesloaded is 2 immediately. Basically, these kind of SWFs work by accident of the specific loading behavior of the FP, which will be difficult to emulate. I suppose we could always create a chunk point at any asset for now.

[Herschel]

Actually I think this was fixed by some of the SWF loading/chunking changes, at least running on desktop, unrelated to #8820.

[n0samu]

I just tried it in latest Ruffle, both the extension and desktop app, and the results were exactly the same as before. So it's not fixed, at least for me.

[torokati44]

Do you think this can be closed now, @Diatonator?

[Diatonator]

@torokati44 seems to work on nightly 2023-07-16.

[torokati44]

Alright then, closing. Thank you for all the testing and reports!

[n0samu]

No, this is not fixed! Please understand that this issue will not be fixed until Ruffle's chunked preloading behavior matches Flash's. It will need to be fixed by some future PR done after a lot of research and experimentation. This is not something that will randomly start working one day.