Session Keepalive?

[Koopzington]

Since i occasionally use AL in a way that involves a bunch of tabs which don't get attention for some time i regularly run into the wellknown "Session expired, please refresh" issue. So i've been thinking: Would adding "automatic session token refreshing" be a possible feature for the script? I haven't been digging too deep into this but i found out it was stored in window.al_token. So in theory you could create a hidden iframe, load AL in there and then update the token through window.al_token = iframe.contentWindow.al_token;. I haven't tested this though. Do you figure this workaround would be worth looking into or should we just wait until AL takes care of this issue themselves?

[hohMiyazawa]

This sounds interesting, as it's an annoying part of the UX.

Definitely worth looking into, but a lot of stuff has to be tested first.. Loading an iframe sounds like it may have excessive performance implications. Perhaps a cheaper way of getting those tokens exists? Maybe sending postmessage messages with updated tokens to old tabs will work?

It's not even certain that just updating the token will fix the issue, as the native javascript may be keeping track of it in some other ways.

Experimental results will be highly appreciated!

[hohMiyazawa]

Posting a couple of example tokens may have a small probability of making this trivial. They are most likely random, but in the off chance they encode a timestamp or something, the solution will be so much easier.

I don't think there are any security implications, since the auth system is separate.

[Koopzington]

Ultimately the new token will be present in the result HTML in the form of a <script>window.al_token = "token";</script>. So there should be plenty of alternatives to iframes, like making the request to AL through the Fetch API and then extracting the token through string operations, however i don't know what the site will do if you do that without a valid token as it also likes to autorefresh, whatever the trigger for this may be.

[hohMiyazawa]

Questions that need answers:

1. When are new tokens created?
2. How long does it take for a token to be expired?
3. What happens when one tampers with window.al_token?
4. Where can tokens be harvested from?

[hohMiyazawa]

Experiments:

a. Setting window.al_token to a fresh token in an expired tab. b. Setting window.al_token to an invalid value.

[hohMiyazawa]

Ultimately the new token will be present in the result HTML in the form of a <script>window.al_token = "token";</script>. So there should be plenty of alternatives to iframes, like making the request to AL through the Fetch API and then extracting the token through string operations, however i don't know what the site will do if you do that without a valid token as it also likes to autorefresh, whatever the trigger for this may be.

This seems possible. Should perform much better than rendering the page and executing all the native javascript.

[hohMiyazawa]

b) Makes native use of the API fail until something is refreshed again.

[hohMiyazawa]

But at least the site does not autorefresh instantly when the token is changed to something invalid.

[hohMiyazawa]

That confirms that anilist is checking window.al_token after creation though! Very nice.

[hohMiyazawa]

I imagine two types of script functionality are possible:

1) Broadcast session tokens from newly refreshed tabs to old stale tabs. Will solve some of the issue, but will not prevent the session expiring if you go away from the computer for a while.

2) Regular fetching of new session tokens in the background.

[Koopzington]

another thing i noticed is that you don't get a new token on every refresh so in theory you should be able to do the same thing in every tab and always get the current, valid token instead of developing any cross-tab-communication shenanigans.

[hohMiyazawa]

I already have cross-tabl-communication shenanigans :3 The fewer requests sent the better.

But yeah, as it's just requesting a couple of hundred bytes of HTML it should be fine. Currently trying to time how long a token lasts. Do you have any numbers?

[hohMiyazawa]

As expected, different browsers open have different access tokens. Tokens appear to be random in content. I still have the same token as 40min ago.

[hohMiyazawa]

And now I got a new one.

[hohMiyazawa]

Updating a stale tab with a new access token, and then crossing out the "session expired" message appears to work fine.

[hohMiyazawa]

Do you prefer to be named or not in a shout-out post?

[Koopzington]

feel free to, i don't mind <3 and thanks for the hard work

[hohMiyazawa]

If tokens can't be issued until the previous one expires, it's inevitable to have those "session expired" messages from time to time during regular browsing. (Unless one frequently polls, and I prefer to not do that).

But the old stale tabs problem, or leaving the computer for a while issue could be solved this way.

[Koopzington]

can you maybe listen for the event that triggers the display of the message? and uh... just hide it with CSS? :P

[hohMiyazawa]

That may actually work very well. Detect it, hide it, and resolve the problem silently.

[hohMiyazawa]

Now I just have to wait until another token expires, see where the message is occurring, and then make a mutationObserver for that.

Then do some testing until I'm confident it works, and then make a css rule to just hide it :)

[hohMiyazawa]

Experimental implementation: https://github.com/hohMiyazawa/Automail/commit/015d9534b33a2d177937c64f7f229f7ca6fe2620

[hohMiyazawa]

Issue: Anilist seems really keen on autorefreshing. That has to be stopped, somehow.

[Koopzington]

To get back on this... the autorefresh get's triggered by the 403. In the main.js you can find the following (formatted through chrome dev tools) This may look confusing at first sight but it's essentially an if that checks wether the last time the refresh of the al_token happened and if it's been over a minute ago, the page refreshes. Which means... we'd need to localStorage.setItem("session-reload", Date.now()); before the 403 happens...uh...

Okay this is silly but i guess we have no other choice than doing the "You have unsaved changes" route by doing something like

addEventListener('beforeunload', function(e){
    e.preventDefault();
    // Get the new al_token
    // Revert the damage caused by the 403
    // Return something because browser demand it despite no longer showing the string.
    return e.returnValue = "we don't do that here";
});

What i mean by "damage" is things like reverting eternal loading animations back into "Load More" buttons in case of infinity scrolls. Not sure if any other things on the page need to get fixed in case of a 403.

If you want to analyze that you can just invalidate the al_token and localStorage.setItem("session-reload", aTimestampWayIntotheFuture);, then cause the 403.

[hohMiyazawa]

In what other cases would a page reload be triggered? Do we want to intercept all those?

[hohMiyazawa]

Do you think something along these lines is appropriate?

// Revert the damage caused by the 403
new MutationObserver(function(){
    let messages = Array.from(document.querySelectorAll(".el-message--error.is-closable"));
    if(messages.some(message => message.textContent === "Session expired, please refresh")){
        message.querySelector(".el-message__closeBtn").click()
    }
}).observe(
    document.body,
    {attributes: false, childList: true, subtree: false}
)

addEventListener("beforeunload", function(e){
    e.preventDefault();
    let oldSessionReload = localStorage.getItem("session-reload");
    // set timestamp immediately, so Anilist doesn't reload the page
    localStorage.setItem("session-reload", Date.now());
    // Get the new al_token
    fetch("index.html").then(function(response){
        return response.text()
    }).then(function(html){
        let token = html.match(/window\.al_token\ =\ "([a-zA-Z0-9]+)";/);
        console.log("token",token);
        if(!token){
            //idk, stuff changed, better clean up after the failed attempt
            localStorage.setItem("session-reload", oldSessionReload);
            return
        }
        window.al_token = token;
        //alert the other tabs so they don't have to do the same
        aniCast.postMessage({type:"sessionToken",value:token});
    }).catch(function(){
        //fail silently, but clean up, trust Anilist to do the right thing by default
        localStorage.setItem("session-reload", oldSessionReload)
    })
    // Return something because browser demand it despite no longer showing the string.
    return e.returnValue = "we don't do that here";
});

[hohMiyazawa]

Reload interception issues:

https://github.com/hohMiyazawa/Automail/blob/master/src/modules/settingsPage.js#L583

https://github.com/hohMiyazawa/Automail/blob/master/src/modules/ALbuttonReload.js#L6

[hohMiyazawa]

Or maybe:

addEventListener("beforeunload", function(e){
    let messages = Array.from(document.querySelectorAll(".el-message--error.is-closable"));
    if(messages.some(message => message.textContent === "Session expired, please refresh")){
        // Revert the damage caused by the 403
        message.querySelector(".el-message__closeBtn").click()
    }
    else{
        // not the reload we are looking for
        return
    }
    e.preventDefault();
    let oldSessionReload = localStorage.getItem("session-reload");
    // set timestamp immediately, so Anilist doesn't reload the page
    localStorage.setItem("session-reload", Date.now());
    // Get the new al_token
    fetch("index.html").then(function(response){
        return response.text()
    }).then(function(html){
        let token = html.match(/window\.al_token\ =\ "([a-zA-Z0-9]+)";/);
        console.log("token",token);
        if(!token){
            //idk, stuff changed, better clean up after the failed attempt
            localStorage.setItem("session-reload", oldSessionReload);
            return
        }
        window.al_token = token;
        //alert the other tabs so they don't have to do the same
        aniCast.postMessage({type:"sessionToken",value:token});
    }).catch(function(){
        //fail silently, but clean up, trust Anilist to do the right thing by default
        localStorage.setItem("session-reload", oldSessionReload)
    })
    // Return something because browser demand it despite no longer showing the string.
    return e.returnValue = "we don't do that here";
});

[Koopzington]

Hmm... i'm not sure if we can actually do much in the beforeunload listener, especially the asynchronous nature of the fetch will cause issues. I think a better approach would be introducing a global variable with which we can either deny or allow the refresh.

new MutationObserver(function(){
    let messages = Array.from(document.querySelectorAll(".el-message--error.is-closable"));
    if(messages.some(message => message.textContent === "Session expired, please refresh")){
        message.querySelector(".el-message__closeBtn").click()
        // update timestamp and al_token
    // Revert the damage caused by the 403
    }
}).observe(
    document.body,
    {attributes: false, childList: true, subtree: false}
)

let allowRefresh = false;
addEventListener("beforeunload", function(e){
    e.preventDefault();
    if (allowRefresh) {
        return;
    }
    // Return something because browser demand it despite no longer showing the string.
    return e.returnValue = "we don't do that here";
});
// Add eventlistener for Keypresses of F5 or CTRL + R which set allowRefresh to true
// Set allowRefresh to true in other parts of Automail that do window.location.reload()

[hohMiyazawa]

I was counting on using the "session expired" message to detect if this is a refresh to be blocked.

From your disassembly, it looks like the message is displayed before the refresh is initiated.

For the asynchronous complication, doesn't it make sense to just keep blocking the refresh until the fetch resolves?

[Koopzington]

upon further inspection i made the following observations:

• The throw you see in the AL code is responsible for ultimately displaying the message you are seeking.
• When a throw happens, the execution stack ends right there and jumps to the nearest catch block which will be responsible for error handling (and message display)
• Therefore there are only two cases:
  • The message is getting rendered because the query that caused the 403 was a mutation or queried the currently authenticated user through Viewer.
  • The page get's refreshed
• In both cases the token and timestamp will need to get updated

I'm leaning myself out of the window and say that in the case where the message get's displayed there's no DOM changes aside from the message itself that need to get taken care of (since Save buttons don't turn into loading animations or the like).

Furthermore i've realized that the current experimental implementation of the token refresh in not only doesn't work due to technical limitations imposed through Grease/TamperMonkey but there was also a little error.

At least when running Automail in Tampermonkey in Chrome and going through the procedure with the stepdebugger i noticed that https://github.com/hohMiyazawa/Automail/blob/015d9534b33a2d177937c64f7f229f7ca6fe2620/src/modules/keepAlive.js#L16 token is an array containing the full string and the desired token so we'll need to use token[1].

Due to security reasons userscripts run in a sandbox where there may be copies of the window object available to you but modifications to it won't be available to the page scripts. There is a workaround available though. https://github.com/hohMiyazawa/Automail/blob/015d9534b33a2d177937c64f7f229f7ca6fe2620/src/modules/keepAlive.js#L20 would turn into window.eval('window.al_token = "' + token[1] + '";');

[hohMiyazawa]

@Koopzington SInce That means the error message is not a sufficient condition to detect this, right? That means the code I posted will not work.

[hohMiyazawa]

// Add eventlistener for Keypresses of F5 or CTRL + R which set allowRefresh to true
// Set allowRefresh to true in other parts of Automail that do window.location.reload()

This is the part of this approach I really don't like though. How can one be sure to catch all legitimate cases were the page should be reloaded? Could lead to confusing cases for users were the page doesn't work as expected.

Instead, why don't just check if the session token is expired and make a decision from that?

• If token is expired, block refresh and get new token.
• If token is not expired, allow the refresh.

[hohMiyazawa]

//by default, allow all refreshes
let allowRefresh = true;

let getSessionToken = function(){
    //block all refreshes while we get the token
    allowRefresh = false;

    // keep this for later, in case we fail to get a new token
    let oldSessionReload = localStorage.getItem("session-reload");

    // set timestamp immediately, so Anilist doesn't reload the page
    localStorage.setItem("session-reload", Date.now());

    // Get the new al_token
    fetch("index.html").then(function(response){
        return response.text()
    }).then(function(html){
        html.match(/window\.al_token\ =\ "([a-zA-Z0-9]+)";/);
        console.log("token",token);
        if(!token){
            //idk, stuff changed, better clean up after the failed attempt
            throw "no token found"
        }
        window.eval('window.al_token = "' + token[1] + '";');
        //alert the other tabs so they don't have to do the same
        aniCast.postMessage({type:"sessionToken",value:token[1]});

        //allow refreshes again, since there's a valid token in place
        allowRefresh = true;
    }).catch(function(){
        //fail silently, but clean up, trust Anilist to do the right thing by default
        localStorage.setItem("session-reload", oldSessionReload);
        allowRefresh = true;
    })
}

new MutationObserver(function(){
    let messages = Array.from(document.querySelectorAll(".el-message--error.is-closable"));
    if(messages.some(message => message.textContent === "Session expired, please refresh")){
        message.querySelector(".el-message__closeBtn").click()
        getSessionToken();
    }
}).observe(
    document.body,
    {attributes: false, childList: true, subtree: false}
)

addEventListener("beforeunload", function(e){
    if(allowRefresh){
        let currentTime = Date.now();
        let tokenTime   = parseInt(localStorage.getItem("session-reload"));

        //check if the token is invalid
        if(!(tokenTime && tokenTime > currentTime - 6e4)){
            e.preventDefault();
            getSessionToken();
            // Return something because browser demand it despite no longer showing the string.
            return e.returnValue = "we don't do that here";
        }
    }
    else{
        return e.returnValue = "we don't do that here";
    }
});

[hohMiyazawa]

More problems: can the refresh be prevented without showing the message box? The confirm box kinda defeats the purpose. Only think I can think of is to hijack window.location.reload.

[hohMiyazawa]

window.location is not configurable in modern browsers :(

[hohMiyazawa]

Therefore, we must prevent Anilist from calling window.location.reload in the first place.

Which means we have to modify the session-reload time so the check never passes.

[hohMiyazawa]

But that means we are responsible for keeping the session token up to date. Which brings back the question of how long it is valid.

[Koopzington]

you mean you don't like the browser asking the user wether they want to leave the page or not? The only way to avoid this would really be to set the session-reload like a day into the future or so. But if you do this, how are you going to detect that a 403 happened? prevent that a 403 will ever happen by regularly checking the validity in the background?

[hohMiyazawa]

I don't see any other way atm.

[hohMiyazawa]

Any detection must happen before the y() function is called.

[Koopzington]

I saw the 5 minute-loop in the PR but that might still cause issues if a page get's loaded <5 before the token expires.

I'll write a little script to find out how long the the session lifetime is as soon as i get home. Especially now that we know about the session-reload we should be able to find out more concrete numbers.

[hohMiyazawa]

Do the science!

[hohMiyazawa]

403 prints an error to the console.

Crazy idea: hijack console.error for detection.

[Koopzington]

So on 18:03 yesterday i started using the following script:

(function() {
    'use strict';

    let loadedAt = Date.now();
    console.log('Page loaded on: ' + new Date(loadedAt), 'Current token: ' + window.eval('window.al_token'));

    window.setInterval(function(){
        // Get the new al_token
        fetch("index.html").then(function(response){
            return response.text()
        }).then(function(html){
            let token = html.match(/window\.al_token\ =\ "([a-zA-Z0-9]+)";/);
            console.log(new Date() + ' - ' + ((Date.now() - loadedAt) / 60000) + ' minutes since page load. Matched token: ' + token[1]);
        });
    }, 60000);
})();

You might be wondering why there's no usage of the session-reload in there: I realized that variable actually has no other use than to effectively prevent the page from calling multiple refreshes in case multiple queries run into 403s simultanously. It's only being used in y().

At 23:04 my laravel_session changed (response headers) together with the token despite the set-cookie header saying it would've been valid until 09:04 the next day (Max Age 43200 = 12 hours). Times when the token changed:

• 23:04
• 01:39
• 01:40 (yes, just a minute and it's not due to my ISP's 24h disconnect)
• 02:14
• 03:55
• 04:14
• 07:14

Parallely i had an icognito window open running the same script at a 25 minutes interval starting from 20:49 because i started suspecting some shenanigans like PHP's garbage collector deleting sessions due to inactivity (with the default lifetime being 24 minutes) and the minutely requests from my first test possibly resulting in an endless session.

In this case the tokens changed between:

• 23:48 and 00:13
• 00:13 and 00:38
• as of 7:18 now still using the same token

I can't make heads or tails of this...