Add a Preloader to the webPlayer

[jetrotal]

Hello, i've been testing the webplayer page with different Pull Requests.

One thing I noticed is that some times the player gets stuck on Preparing... (0/1) Sometimes it proceeds to load the game after a long time waiting, some time it doesn't.

I never know if it's still loading, or if something went wrong until I check the browser's console.

---

Would be possible to apply a div over everything that gives us a progress on the loading states of a game?

I made this css + js loader: https://codepen.io/jetrotau/pen/mdXpgVV

it has 150x150px, The way you update it is by calling those js functions:

updateLoader(50);
updateLoader(101); // hides  div if preloader reaches 101%
updateLoader("ERROR - RELOAD THE PAGE","red");

[carstene1ns]

If it stops loading somewhere, this should be a bug. Emscripten has an own way to make the progress display, it could be changed to anything. However, you need to actually detect the error for everything to work.

[Ghabry]

The Preparing... is the download of the .wasm file. The download is invoked by the generated js-file. No idea if their is a way to get the download progress.

[fdelapena]

Related: https://github.com/EasyRPG/Player/issues/562

[jetrotal]

hm... I saw the error yesterday, i should have reported it as soon as I saw it...

here's how the browser loads https://easyrpg.org/play/pr2730/:

This long blue line, is the time it takes to load index-pr.wasm, where i'm at the preparing page.

Maybe you can place the complete file size as percentage, and compare it to current file size to tell the user how close it is to finish loading it...?

[Ghabry]

Emscripten bug: https://github.com/emscripten-core/emscripten/issues/16278

[jetrotal]

OK, I'm slowly finding a way to make the pre loader work, bear with me

---

From what I understood, this is how index-pr.js reads the wasm file:

function getBinaryPromise() {
  // If we don't have the binary yet, try to to load it asynchronously.
  // Fetch has some additional restrictions over XHR, like it can't be used on a file:// url.
  // See https://github.com/github/fetch/pull/92#issuecomment-140665932
  // Cordova or Electron apps are typically loaded from a file:// url.
  // So use fetch if it is available and the url is not a file, otherwise fall back to XHR.
  if (!wasmBinary && (ENVIRONMENT_IS_WEB || ENVIRONMENT_IS_WORKER)) {
    if (typeof fetch == 'function'
      && !isFileURI(wasmBinaryFile)
    ) {
      return fetch(wasmBinaryFile, { credentials: 'same-origin' }).then(function(response) {
        if (!response['ok']) {
          throw "failed to load wasm binary file at '" + wasmBinaryFile + "'";
        }
        return response['arrayBuffer']();
      }).catch(function () {
          return getBinary(wasmBinaryFile);
      });
    }
    else {
      if (readAsync) {
        // fetch is not available or url is file => try XHR (readAsync uses XHR internally)
        return new Promise(function(resolve, reject) {
          readAsync(wasmBinaryFile, function(response) { resolve(new Uint8Array(/** @type{!ArrayBuffer} */(response))) }, reject)
        });
      }
    }
  }

  // Otherwise, getBinary should be able to get it synchronously
  return Promise.resolve().then(function() { return getBinary(wasmBinaryFile); });
}

based on that I ducktaped a function that can get the wasm's Uint8Array length while it's still loading:

var resp;
var reader;
var receivedLength = 0;
var contentLength = 0;

var test = fetch(wasmBinaryFile, {    credentials: 'same-origin'})
 .then((response) => {
    resp = response;
    reader = resp.body.getReader();
    contentLength = resp.headers.get('Content-Length');
    Watch_Uint8Array_length(reader)
 });

async function Watch_Uint8Array_length(obj) {
    while (true) {
        const { done, value } = await obj.read();

        if (done) break;

        receivedLength += value.length;
        console.log(receivedLength)
    }
}

console.log(receivedLength) returns to me the current length of the wasm Uint8Array as it is being sent:

---

the resp.headers.get('Content-Length') should give me the total file length information, that is inside its header, but that property does not appear on my console, all I have is:

then I read:

Just because your browser receives the header and you can see the header in devtools doesn’t mean your frontend JavaScript code can see it. If the response from a cross-origin request doesn’t have an Access-Control-Expose-Headers: Content-Length response — as indicated in this answer — then it’s your browser itself that will block your code from being able to access the header. For a cross-origin request, your browser will only expose a particular response header to your frontend JavaScript code if the Access-Control-Expose-Headers value contains the name of that particular header.

Maybe enabling the CORS property Access-Control-Expose-Headers: Content-Length would help with it...? Idk

This is where I'm stuck right now.

[jetrotal]

Update:

We found this post and concluded that the gzip compression from nginx also hides the content-length from browsers.

So, from what I got, the compression task will be moved to Jenkins...

---

Now the code is working as intended:

var resp;
var reader;
var receivedLength = 0;
var contentLength = 0;
var percentage = {old:"",current:""};

fetch(wasmBinaryFile, {credentials: 'same-origin'})
 .then((response) => {
    resp = response;
    reader = resp.body.getReader();
    contentLength = resp.headers.get('Content-Length');
    Watch_Uint8Array_length(reader)
 });

async function Watch_Uint8Array_length(obj) {
    while (true) {
        const { done, value } = await obj.read();

        if (done) {
            // updateLoader(101);  // uncomment this if you are using my preloader;
            break;
        }

/*
if (ERROR_CASE) { // How do I detect a download error?
            updateLoader("ERROR - RELOAD THE PAGE","red");
            break;
        } 
// */

        receivedLength += value.length;
        percentage.current = Math.round((receivedLength / contentLength) * 100);

        if (percentage.old === percentage.current) continue;
        else percentage.old = percentage.current;

        console.log(percentage.current +"%");
        // updateLoader(percentage.current); // uncomment this if you are using my preloader;
    }
}

You can test it on a webplayer by pasting the code above on your console.

---

To implement this on the player, you must delete my fetch function and combine this then:

 .then((response) => {
    resp = response;
    reader = resp.body.getReader();
    contentLength = resp.headers.get('Content-Length');
    Watch_Uint8Array_length(reader)
 });

with the then from the wasm loader.

You guys can also uncomment the functions related to the preload from https://codepen.io/jetrotau/pen/mdXpgVV
if you guys wish to use it.

[Ghabry]

We cannot use anything that relies on a certain server configuration. Remember that our web player is the same as the one others will put on their website when hosting games.

This required gzip step before the deployment must be also explained to everyone that builds the Player manually.

[jetrotal]

well, one solution is to have an extra file or line in the js file that already has the wasm lengtth, like a gencache file, or Module.filesize

But that implies on adding an extra step to the build, as carstene1ns commented on discord.

[jetrotal]

I took @Ghabry concerns in consideration, and made the following changes:

force the loader to look for Module.filesize or resp.headers.get('Content-Length')

  contentLength = Module.filesize ? Module.filesize : resp.headers.get("Content-Length");

If those two fail to return a number, then displays unprecise progress updates.

Here's an complete code, that already creates a DIV inside the player's page, you can test it by pasting it into your console:

var resp, reader, receivedLength = 0, contentLength = 0; 
var percentage = {old:"", current:""};

const logoURL = "https://raw.githubusercontent.com/jetrotal/OpenRTP-CheckList/gh-page/img/logo.svg"; 

const preloaderHTML = `
<div id=preloader>
  <div id=loader><img src=${logoURL} id=loaderLogo onload="updateLoader('preparing');"></div>
  <style id=preloaderStyle>
    :root {
      --loaderColor: #72b740;
      --loaderBGcolor: #262f23;
      --loaderVal: 0;
      --loaderStatus: "";
      --loaderDisplay: visible;
      --loaderBar: 1;
    }

    #loaderLogo {
      opacity: .3;
      width: 100px
    }

    #preloader {
      font-family: Ubuntu, Droid Sans, Trebuchet MS, sans-serif;
      display: var(--loaderDisplay);
      height: 100%;
      left: 0;
      position: fixed;
      top: 0;
      width: 100%;
      z-index:99999;
    }

    #loader {
      align-items: center;
      display: flex;
      height: 150px;
      justify-content: center;
      left: 50%;
      margin: -75px 0 0-75px;
      position: relative;
      top: 50%;
      width: 150px;
    }

    #loader:before {
      color: var(--loaderColor);
      content: var(--loaderStatus);
      font-weight: 900;
        font-size: 16px;
      position: absolute;
      text-align: center;
      white-space: pre-wrap;
    }

    #loader:after {
      border: 3px solid;

      border-image: linear-gradient(to right, var(--loaderColor) 0,
      var(--loaderColor) calc((var(--loaderVal) - 1) * 1%),
      var(--loaderBGcolor) calc(var(--loaderVal) * 1%)) 0 0 100%;

      opacity: var(--loaderBar);
      bottom: 10px;
      content: "";
      left: 15px;
      position: absolute;
      right: 15px;
    }
  </style>
</div>
`;

function initPreloader(response) {

  document.getElementById("status").innerHTML += preloaderHTML;

  resp = response;
  reader = resp.body.getReader();
  contentLength = Module.filesize ? Module.filesize : resp.headers.get("Content-Length");

  Watch_Uint8Array_length(reader);

}

function formatBytes(bytes, decimals = 2) {
  if (0 === bytes) return "0 Bytes";

  const i = Math.floor(Math.log(bytes) / Math.log(1024));
  return parseFloat((bytes / Math.pow(1024, i)).toFixed(0 > decimals ? 0 : decimals)) + " " + "Bytes KB MB GB TB PB EB ZB YB".split(" ")[i];
}

async function Watch_Uint8Array_length(obj) {
  for (var message;;) {
    const {done, value} = await obj.read();

      if (done){ 
       updateLoader("done");
       break;
               }
    receivedLength += value.length;

    if (contentLength) {
      percentage.current = Math.round(receivedLength / contentLength * 100);

      if (percentage.old === percentage.current) continue;
      else percentage.old = percentage.current;

      message = percentage.current;
    } else message = "Downloading\n " + formatBytes(receivedLength);

    updateLoader(message);    
  }
}

function updateLoader(val, color = "#72b740") {
  !( "string" === typeof val || val instanceof String ) ? changeProp("Bar", 1) :
      (changeProp("Bar", 0), val = val.split("\n").join("\\a")) ;

  if ("done" == val || 101 == val) return changeProp("Display", "none");

  var status = isNaN(val) ? '"' + val + '"' : '"' + val + '%"';
  changeProp("Display", "visible");
  changeProp("Val", val);
  changeProp("Color", color);
  changeProp("Status", status);
}

function changeProp(el, val, type = "--loader") {
  document.documentElement.style.setProperty(type + el, val);
}

fetch(wasmBinaryFile, {credentials:"same-origin"}).then(response => {
  initPreloader(response);
});

[jetrotal]

some more updates:

I put the preloader on a separeted JS file, to make it easy to modify and edit stuff. As seen here: https://github.com/jetrotal/testing-stuff and can be tested here: https://jetrotal.github.io/testing-stuff/?game=TestGame-2000

to implement the new preloader.js file, you'll have to add a script tag over the index.js one

<script async type="text/javascript" src="preloader.js"></script>
<script async type="text/javascript" src="index.js"></script>

and add try {initPreloader(response)} catch (e) {}; right after the line var result = WebAssembly.instantiateStreaming(response, info);

---

One issue I already commented on discord is that index.js unzips index.wasm gzip as it is being loaded. So, the file's percentage will always be bigger than 100% since it's comparing the gzip file size with the unzipped file size.

The code has a workaround that gives an imprecise loading status whenever the percentage is bigger than 100%. The ideal solution is to have Module.filezile as @carstene1ns commented on discord.

---

There's another issue, that i don't know yet if is it really an issue, The browser's network console is telling me that i'm trying to load index.wasm twice:

But it also tells me that its reading it from disk's cache, which means not being downloaded twice. Would that be a bad deal memory wise?

I'm trying to solve that by destroying all vars that may contain data that the preloader reads:

 if (done){ 
        resp = reader = receivedLength = contentLength = "";
        percentage = {old:"", current:""};
        updateLoader("done");
        break;
               }

[Ghabry]

A week ago I saw a command line argument to emcc that allows transforming the javascript file (index.js) so it would be possible to inject our code directly instead of using an external solution.

Only problem is that I cannot remember which argument it was, havn't written it down :facepalm: