[ZZZ/GI] Cache values for stamina problem

[YozenPL]

Hello,

I'm using the cache values for my external display and I saw something strange. I'll use ZZZ as example. I set the threshold for 230/240. When I reach the 230 I'm receiving a notification via webhook (+SMS via REST from my script which is reading the cache). I'm login to the game and use my stamina. Let's say I decreased it to 100. Cache value is still showing the 230 (even after 3 hours - I know because I'm getting the 1 SMS per hour for it) until I restart the hoyolab-auto. Could you please look at it?

[torikushiii]

Hi,

I've pushed a new commit that add more sanity checks for stamina, I'm still not sure how to replicate this issue because I never encountered them with different tests, let me know if the issue still persist.

[YozenPL]

Hello. I Just tested it. 230 threshold was reached at 15:30 CET. I used 100 stamina and now 1 hour later the cache file is still showing 230.

[torikushiii]

Interesting, this behavior should have never happened because the cache file will invalidate itself after 1 hour, did you tried to delete the cache file and let it create a new one?

[YozenPL]

No. When this issue is happening I'm just stopping and starting the HoyoLab Auto.

[torikushiii]

forgot to mention it above that can you try to delete it if you have not tried it and let the cache file create itself? my bad 😅

[YozenPL]

Restart refreshed the data in the file, but I delete it anyway as you suggested. I'll be able to check it behavior tomorrow.

[YozenPL]

For GI stamina is updating after the webhook message was sent - which is good. But for ZZZ it stays at 230 in cache file.

[torikushiii]

Interesting, I've never seen this behavior before, thanks for the report and I'll try to look into it further

[torikushiii]

Hello, I've updated the code again and haven't tested it yet extensively (but currently still monitoring the local calculations) but it should cover all the bases, let me know if somehow it still not working.

[YozenPL]

The issue remains. I used some of my stamina 3,5h ago and in cache it still says it is 240. Also I notice 2 additional json files (one per game). They contains "currentStamina" but those values are also wrong.

[torikushiii]

at what platform are you running this project on? are you using Docker or straight from NodeJS

[YozenPL]

Directly on RapberryPi 3.

[torikushiii]

hello, can you try this one, replace all the code at the cache.js file which is located at hoyolab-modules/cache.js, and don't forget to delete your cache.json file.

I've added few more checks for the expiry update.

module.exports = class DataCache {
    static data = new Map();
    static expirationInterval;
    static lastForceRefresh = new Map();

    constructor (expiration, rate, forceRefreshInterval = 3_600_000) {
        this.expiration = expiration;
        this.rate = rate;
        this.forceRefreshInterval = forceRefreshInterval;

        if (!DataCache.expirationInterval) {
            DataCache.expirationInterval = setInterval(() => {
                DataCache.data.clear();
                DataCache.lastForceRefresh.clear();
            }, this.expiration);
        }
    }

    async set (key, value, lastUpdate = Date.now()) {
        const data = { ...value, lastUpdate };
        DataCache.data.set(key, data);
        DataCache.lastForceRefresh.set(key, lastUpdate);

        if (app.Cache) {
            await app.Cache.set({
                key,
                value: data,
                expiry: this.forceRefreshInterval
            });
        }
    }

    async get (key) {
        let cachedData = DataCache.data.get(key) || (app.Cache && await app.Cache.get(key));

        if (cachedData) {
            const now = Date.now();
            const timeSinceLastUpdate = now - cachedData.lastUpdate;
            const timeSinceLastForceRefresh = now - (DataCache.lastForceRefresh.get(key) || 0);

            if (timeSinceLastUpdate > this.forceRefreshInterval || timeSinceLastForceRefresh > this.forceRefreshInterval) {
                await DataCache.invalidateCache(cachedData.uid);
                return null;
            }

            cachedData = await this.#updateCachedData(cachedData, key);
            if (cachedData) {
                DataCache.data.set(key, cachedData);
            }
        }

        return cachedData;
    }

    async #updateCachedData (cachedData, key) {
        const now = Date.now();
        const secondsSinceLastUpdate = (now - cachedData.lastUpdate) / 1000;

        if (secondsSinceLastUpdate > this.expiration / 1000) {
            await DataCache.invalidateCache(cachedData.uid);
            return null;
        }

        const account = app.HoyoLab.getAccountById(cachedData.uid);

        if (cachedData.stamina) {
            const recoveredStamina = Math.floor(secondsSinceLastUpdate / this.rate);
            const oldStamina = cachedData.stamina.currentStamina;
            cachedData.stamina.currentStamina = Math.min(
                cachedData.stamina.maxStamina,
                cachedData.stamina.currentStamina + recoveredStamina
            );
            cachedData.stamina.recoveryTime = Math.max(0, cachedData.stamina.recoveryTime - Math.round(secondsSinceLastUpdate));

            const staminaThresholdReached = cachedData.stamina.currentStamina > account.stamina.threshold;
            const staminaAlmostFull = (cachedData.stamina.maxStamina - cachedData.stamina.currentStamina) <= 10;

            if (cachedData.stamina.currentStamina >= cachedData.stamina.maxStamina
                || staminaThresholdReached
                || (staminaAlmostFull && staminaThresholdReached)
                || cachedData.stamina.recoveryTime <= 0
                || oldStamina > cachedData.stamina.currentStamina) {
                await DataCache.invalidateCache(cachedData.uid);
                return null;
            }
        }

        if (cachedData.expedition && cachedData.expedition.list.length > 0) {
            let shouldInvalidate = false;
            for (const expedition of cachedData.expedition.list) {
                expedition.remaining_time = Math.max(0, Number(expedition.remaining_time) - Math.round(secondsSinceLastUpdate));
                if (expedition.remaining_time <= 0) {
                    shouldInvalidate = true;
                }
            }
            if (shouldInvalidate) {
                await DataCache.invalidateCache(cachedData.uid);
                return null;
            }
        }

        if (cachedData.shop && cachedData.shop.state === "Finished") {
            await DataCache.invalidateCache(cachedData.uid);
            return null;
        }

        if (cachedData.realm) {
            cachedData.realm.recoveryTime = Math.max(0, cachedData.realm.recoveryTime - Math.round(secondsSinceLastUpdate));
            if (cachedData.realm.currentCoin >= cachedData.realm.maxCoin || cachedData.realm.recoveryTime <= 0) {
                await DataCache.invalidateCache(cachedData.uid);
                return null;
            }
        }

        cachedData.lastUpdate = now;
        DataCache.lastForceRefresh.set(key, now);
        return cachedData;
    }

    static async invalidateCache (key) {
        DataCache.data.delete(key);
        DataCache.lastForceRefresh.delete(key);
        if (app.Cache) {
            await app.Cache.delete(key);
        }
    }

    static destroy () {
        clearInterval(DataCache.expirationInterval);
        DataCache.data.clear();
        DataCache.lastForceRefresh.clear();
    }
};

[YozenPL]

Ok, I replaced it and removed the json as suggested. We will see the outcome in max 48h :) Thank you.

[YozenPL]

And still the same. cache file is updating since my stamina from GI is updating correctly, but for ZZZ count stayed at 231 for 2,5h. And my current stamina for ZZZ is 64. It looks like it is stopping the checking for current stamina after the threshold is reached - I'm just saying how it looks from my perspective. I'm disabling the thresholds now to see if issue will remain.

[YozenPL]

Another update: I deleted the cache file and started the application at 14:37 CET. It's been an hour already and cache file have entry related to GI only, not ZZZ. Last cache file update was done at 15:00 CET

If it's working like cron and cache file should be invalidated at every XX:00 hour then somehow script was not able to fetch my data from API maybe?

[torikushiii]

fetching the data from API should work fine since you still get the data from hoyolab.

You were mentioning that "disabling" the threshold im assuming that you set it to 0? If you are, then the missing ZZZ at cache file is intended since it will invalidate the ZZZ cache after 1-2 request were made and will write itself to cache again after below the threshold.

[YozenPL]

I disabled the check as comments says it is for notification "Enable this if you want to get notified when your stamina is above the threshold". Does it also means that stamina values will be not updated?

[torikushiii]

yes unfortunately, it will disable entirely the stamina check for ZZZ.

from the way you described the issue, it sounds like the ZZZ cache is stuck somewhere. I’m trying to replicate this with all possible outcomes, but so far I haven’t had any luck encountering what you described.

[torikushiii]

can you try this one, I'm trying to invalidate it by checking if the difference between the game's maximum stamina and your current cached stamina is less than 10. (maybe not the best approach, but im trying to see what causes this)

updated at 20:30 CET

module.exports = class DataCache {
    static data = new Map();
    static expirationInterval;
    static lastForceRefresh = new Map();

    constructor (expiration, rate, forceRefreshInterval = 3_600_000) {
        this.expiration = expiration;
        this.rate = rate;
        this.forceRefreshInterval = forceRefreshInterval;

        if (!DataCache.expirationInterval) {
            DataCache.expirationInterval = setInterval(() => {
                DataCache.data.clear();
                DataCache.lastForceRefresh.clear();
            }, this.expiration);
        }
    }

    async set (key, value, lastUpdate = Date.now()) {
        const data = { ...value, lastUpdate };
        DataCache.data.set(key, data);
        DataCache.lastForceRefresh.set(key, lastUpdate);

        if (app.Cache) {
            await app.Cache.set({
                key,
                value: data,
                expiry: this.forceRefreshInterval
            });
        }
    }

    async get (key) {
        let cachedData = DataCache.data.get(key) || (app.Cache && await app.Cache.get(key));

        if (cachedData) {
            const now = Date.now();
            const timeSinceLastUpdate = now - cachedData.lastUpdate;
            const timeSinceLastForceRefresh = now - (DataCache.lastForceRefresh.get(key) || 0);

            if (timeSinceLastUpdate > this.forceRefreshInterval || timeSinceLastForceRefresh > this.forceRefreshInterval) {
                await DataCache.invalidateCache(cachedData.uid);
                return null;
            }

            cachedData = await this.#updateCachedData(cachedData, key);
            if (cachedData) {
                DataCache.data.set(key, cachedData);
            }
        }

        return cachedData;
    }

    async #updateCachedData (cachedData, key) {
        const now = Date.now();
        const secondsSinceLastUpdate = (now - cachedData.lastUpdate) / 1000;
        const hourInSeconds = 3600;

        if (secondsSinceLastUpdate > hourInSeconds) {
            await DataCache.invalidateCache(cachedData.uid);
            return null;
        }

        const account = app.HoyoLab.getAccountById(cachedData.uid);

        if (cachedData.stamina) {
            const recoveredStamina = Math.floor(secondsSinceLastUpdate / this.rate);
            cachedData.stamina.currentStamina = Math.min(
                cachedData.stamina.maxStamina,
                cachedData.stamina.currentStamina + recoveredStamina
            );
            cachedData.stamina.recoveryTime = Math.max(0, cachedData.stamina.recoveryTime - Math.round(secondsSinceLastUpdate));

            const staminaAlmostFull = cachedData.stamina.maxStamina - cachedData.stamina.currentStamina <= 10;
            const staminaThresholdReached = cachedData.stamina.currentStamina >= account.stamina.threshold;

            if (cachedData.stamina.currentStamina >= cachedData.stamina.maxStamina
                || staminaThresholdReached
                || staminaAlmostFull) {
                await DataCache.invalidateCache(cachedData.uid);
                return null;
            }
        }

        if (cachedData.expedition && cachedData.expedition.list.length > 0) {
            for (const expedition of cachedData.expedition.list) {
                expedition.remaining_time = Math.max(0, Number(expedition.remaining_time) - Math.round(secondsSinceLastUpdate));
                if (expedition.remaining_time <= 0) {
                    await DataCache.invalidateCache(cachedData.uid);
                    return null;
                }
            }
        }

        if (cachedData.shop && cachedData.shop.state === "Finished") {
            await DataCache.invalidateCache(cachedData.uid);
            return null;
        }

        if (cachedData.realm) {
            cachedData.realm.recoveryTime = Math.max(0, cachedData.realm.recoveryTime - Math.round(secondsSinceLastUpdate));
            if (cachedData.realm.currentCoin >= cachedData.realm.maxCoin || cachedData.realm.recoveryTime <= 0) {
                await DataCache.invalidateCache(cachedData.uid);
                return null;
            }
        }

        cachedData.lastUpdate = now;
        DataCache.lastForceRefresh.set(key, now);
        return cachedData;
    }

    static async invalidateCache (key) {
        DataCache.data.delete(key);
        DataCache.lastForceRefresh.delete(key);
        if (app.Cache) {
            await app.Cache.delete(key);
        }
    }

    static destroy () {
        clearInterval(DataCache.expirationInterval);
        DataCache.data.clear();
        DataCache.lastForceRefresh.clear();
    }
};

[YozenPL]

It did not helped. The another thing which is strange to me: When I set stamina threshold to 230 it is staying at this value in cache file after it was reached. I did a small test. I used some of my stamina and I had 160. I removed the cache file, set the threshold to 170 and restart the application. Now the cache file stamina stays at 170 after it was reached.

I believe after the script get the data from API it is updating the stamina counter locally for 1 hour, am I right? After the threshold is reached it is stopping doing it. Plus it is no longer fetching the data from API for ZZZ even is stamina counter is lover than threshold. For GI there is no problem at all.

[torikushiii]

I see :/

I believe after the script get the data from API it is updating the stamina counter locally for 1 hour, am I right?

yes, that is correct.

After the threshold is reached it is stopping doing it. Plus it is no longer fetching the data from API for ZZZ even is stamina counter is lover than threshold.

yes this is intended, but it will fetch from API again until it reached threshold again.

guess I'll try to isolate it to ZZZ only this time and see if its any help.

[torikushiii]

sorry for the late reply, this one have a separate check for zzz

module.exports = class DataCache {
    static data = new Map();
    static expirationInterval;
    static lastForceRefresh = new Map();

    constructor (expiration, rate, forceRefreshInterval = 3_600_000) {
        this.expiration = expiration;
        this.rate = rate;
        this.forceRefreshInterval = forceRefreshInterval;

        if (!DataCache.expirationInterval) {
            DataCache.expirationInterval = setInterval(() => {
                DataCache.data.clear();
                DataCache.lastForceRefresh.clear();
            }, this.expiration);
        }
    }

    async set (key, value, lastUpdate = Date.now()) {
        const data = { ...value, lastUpdate };
        DataCache.data.set(key, data);
        DataCache.lastForceRefresh.set(key, lastUpdate);

        if (app.Cache) {
            await app.Cache.set({
                key,
                value: data,
                expiry: this.forceRefreshInterval
            });
        }
    }

    async get (key) {
        let cachedData = DataCache.data.get(key) || (app.Cache && await app.Cache.get(key));

        if (cachedData) {
            const now = Date.now();
            const timeSinceLastUpdate = now - cachedData.lastUpdate;
            const timeSinceLastForceRefresh = now - (DataCache.lastForceRefresh.get(key) || 0);

            if (timeSinceLastUpdate > this.forceRefreshInterval || timeSinceLastForceRefresh > this.forceRefreshInterval) {
                await DataCache.invalidateCache(cachedData.uid);
                return null;
            }

            cachedData = await this.#updateCachedData(cachedData, key);
            if (cachedData) {
                DataCache.data.set(key, cachedData);
            }
        }

        return cachedData;
    }

    async #updateCachedData (cachedData, key) {
        const now = Date.now();
        const secondsSinceLastUpdate = (now - cachedData.lastUpdate) / 1000;
        const hourInSeconds = 3600;

        if (secondsSinceLastUpdate > hourInSeconds) {
            await DataCache.invalidateCache(cachedData.uid);
            return null;
        }

        const account = app.HoyoLab.getAccountById(cachedData.uid);
        const gameType = account.platform;

        if (cachedData.stamina) {
            const recoveredStamina = Math.floor(secondsSinceLastUpdate / this.rate);
            cachedData.stamina.currentStamina = Math.min(
                cachedData.stamina.maxStamina,
                cachedData.stamina.currentStamina + recoveredStamina
            );
            cachedData.stamina.recoveryTime = Math.max(0, cachedData.stamina.recoveryTime - Math.round(secondsSinceLastUpdate));

            const staminaAlmostFull = cachedData.stamina.maxStamina - cachedData.stamina.currentStamina <= 10;
            const staminaThresholdReached = cachedData.stamina.currentStamina >= account.stamina.threshold;

            if (gameType === "nap") {
                console.log({
                    cachedData: cachedData.stamina,
                    staminaConfig: account.stamina
                });

                const threshold = account.stamina.threshold;
                if (cachedData.stamina.currentStamina >= threshold) {
                    await DataCache.invalidateCache(cachedData.uid);
                    return null;
                }
            }
            else if (cachedData.stamina.currentStamina >= cachedData.stamina.maxStamina
                    || staminaThresholdReached
                    || staminaAlmostFull) {
                await DataCache.invalidateCache(cachedData.uid);
                return null;
            }
        }

        if (cachedData.expedition && cachedData.expedition.list.length > 0) {
            for (const expedition of cachedData.expedition.list) {
                expedition.remaining_time = Math.max(0, Number(expedition.remaining_time) - Math.round(secondsSinceLastUpdate));
                if (expedition.remaining_time <= 0) {
                    await DataCache.invalidateCache(cachedData.uid);
                    return null;
                }
            }
        }

        if (cachedData.shop && cachedData.shop.state === "Finished") {
            await DataCache.invalidateCache(cachedData.uid);
            return null;
        }

        if (cachedData.realm) {
            cachedData.realm.recoveryTime = Math.max(0, cachedData.realm.recoveryTime - Math.round(secondsSinceLastUpdate));
            if (cachedData.realm.currentCoin >= cachedData.realm.maxCoin || cachedData.realm.recoveryTime <= 0) {
                await DataCache.invalidateCache(cachedData.uid);
                return null;
            }
        }

        cachedData.lastUpdate = now;
        DataCache.lastForceRefresh.set(key, now);
        return cachedData;
    }

    static async invalidateCache (key) {
        DataCache.data.delete(key);
        DataCache.lastForceRefresh.delete(key);
        if (app.Cache) {
            await app.Cache.delete(key);
        }
    }

    static destroy () {
        clearInterval(DataCache.expirationInterval);
        DataCache.data.clear();
        DataCache.lastForceRefresh.clear();
    }
};

[YozenPL]

It did not helped. To speed up things I used all my stamina and set the threshold to 10. After reaching 10 it stays on this value in cache file. It is 2,5h already :D

   ],
   [
       "keyv:XXXXXX",
       {
           "value": "{\"value\":{\"uid\":\"XXXXXXX\",\"nickname\":\"XXXXX\",\"lastUpdate\":1724152203643,\"cardSign\":\"Completed\",\"stamina\":{\"currentStamina\":10,\"maxStamina\":240,\"recoveryTime\":82538},\"dailies\":{\"task\":400,\"maxTask\":400},\"weeklies\":{\"bounty\":5,\"bountyTotal\":5,\"surveyPoints\":8000,\"surveyPointsTotal\":8000},\"shop\":{\"state\":\"Open\"}},\"expires\":1724155803644}",
           "expire": 1724155803644
       }
   ],

Maybe I could enable some kind of logging? It will be easier to debug it. Another thought: Maybe the function which is checking the threshold to trigger the webhook to sent notification is breaking something for ZZZ? Problem is always occurring: if (threshold >= currentStamina)

[torikushiii]

I'm assuming there's a console log that appears for cachedData and staminaConfig ? because I put it it here

if (gameType === "nap") {
    console.log({
        cachedData: cachedData.stamina,
        staminaConfig: account.stamina
    });
}

Maybe the function which is checking the threshold to trigger the webhook to sent notification is breaking something for ZZZ?

I'll try checking out this approach

[YozenPL]

Hmmm I have 58 stamina I set threshold to 59 which I will have at 17:58 CET. At 18:00 script should "see" that I have reached the threshold. I'll observe the console output.

[YozenPL]

I got notification on Discord:

But on console - there is nothing:

[torikushiii]

interesting, can you try this one, i've added log point to every invalidation and wait for your stamina to regen at least 2 points

module.exports = class DataCache {
    static data = new Map();
    static expirationInterval;
    static lastForceRefresh = new Map();

    constructor (expiration, rate, forceRefreshInterval = 3_600_000) {
        this.expiration = expiration;
        this.rate = rate;
        this.forceRefreshInterval = forceRefreshInterval;

        if (!DataCache.expirationInterval) {
            DataCache.expirationInterval = setInterval(() => {
                DataCache.data.clear();
                DataCache.lastForceRefresh.clear();
            }, this.expiration);
        }
    }

    async set (key, value, lastUpdate = Date.now()) {
        const data = { ...value, lastUpdate };
        DataCache.data.set(key, data);
        DataCache.lastForceRefresh.set(key, lastUpdate);

        if (app.Cache) {
            await app.Cache.set({
                key,
                value: data,
                expiry: this.forceRefreshInterval
            });
        }
    }

    async get (key) {
        let cachedData = DataCache.data.get(key) || (app.Cache && await app.Cache.get(key));

        if (cachedData) {
            const now = Date.now();
            const timeSinceLastUpdate = now - cachedData.lastUpdate;
            const timeSinceLastForceRefresh = now - (DataCache.lastForceRefresh.get(key) || 0);

            if (timeSinceLastUpdate > this.forceRefreshInterval || timeSinceLastForceRefresh > this.forceRefreshInterval) {
                console.log(`[DEBUG] Force refreshing cache for ${key} due to time since last update: ${timeSinceLastUpdate}ms, time since last force refresh: ${timeSinceLastForceRefresh}ms`);
                await DataCache.invalidateCache(cachedData.uid);
                return null;
            }

            cachedData = await this.#updateCachedData(cachedData, key);
            if (cachedData) {
                DataCache.data.set(key, cachedData);
            }
        }

        return cachedData;
    }

    async #updateCachedData (cachedData, key) {
        const now = Date.now();
        const secondsSinceLastUpdate = (now - cachedData.lastUpdate) / 1000;
        const hourInSeconds = 3600;

        if (secondsSinceLastUpdate > hourInSeconds) {
            console.log(`[DEBUG] Force refreshing cache for ${key} due to time since last update: ${secondsSinceLastUpdate}s #2`);
            await DataCache.invalidateCache(cachedData.uid);
            return null;
        }

        const account = app.HoyoLab.getAccountById(cachedData.uid);

        if (cachedData.stamina) {
            const recoveredStamina = Math.floor(secondsSinceLastUpdate / this.rate);
            cachedData.stamina.currentStamina = Math.min(
                cachedData.stamina.maxStamina,
                cachedData.stamina.currentStamina + recoveredStamina
            );
            cachedData.stamina.recoveryTime = Math.max(0, cachedData.stamina.recoveryTime - Math.round(secondsSinceLastUpdate));

            const staminaAlmostFull = cachedData.stamina.maxStamina - cachedData.stamina.currentStamina <= 10;
            const staminaThresholdReached = cachedData.stamina.currentStamina >= account.stamina.threshold;

            if (cachedData.stamina.currentStamina >= cachedData.stamina.maxStamina
                || staminaThresholdReached
                || staminaAlmostFull) {
                console.log(`[DEBUG] Force refreshing cache for ${key} due to stamina recovery`);
                await DataCache.invalidateCache(cachedData.uid);
                return null;
            }
        }

        if (cachedData.expedition && cachedData.expedition.list.length > 0) {
            for (const expedition of cachedData.expedition.list) {
                expedition.remaining_time = Math.max(0, Number(expedition.remaining_time) - Math.round(secondsSinceLastUpdate));
                if (expedition.remaining_time <= 0) {
                    console.log(`[DEBUG] Force refreshing cache for ${key} due to expedition completion`);
                    await DataCache.invalidateCache(cachedData.uid);
                    return null;
                }
            }
        }

        if (cachedData.shop && cachedData.shop.state === "Finished") {
            console.log(`[DEBUG] Force refreshing cache for ${key} due to shop state`);
            await DataCache.invalidateCache(cachedData.uid);
            return null;
        }

        if (cachedData.realm) {
            cachedData.realm.recoveryTime = Math.max(0, cachedData.realm.recoveryTime - Math.round(secondsSinceLastUpdate));
            if (cachedData.realm.currentCoin >= cachedData.realm.maxCoin || cachedData.realm.recoveryTime <= 0) {
                console.log(`[DEBUG] Force refreshing cache for ${key} due to realm recovery`);
                await DataCache.invalidateCache(cachedData.uid);
                return null;
            }
        }

        cachedData.lastUpdate = now;
        DataCache.lastForceRefresh.set(key, now);
        return cachedData;
    }

    static async invalidateCache (key) {
        console.log(`[DEBUG] Invalidated cache for ${key}`);
        DataCache.data.delete(key);
        DataCache.lastForceRefresh.delete(key);
        if (app.Cache) {
            await app.Cache.delete(key);
        }
    }

    static destroy () {
        clearInterval(DataCache.expirationInterval);
        DataCache.data.clear();
        DataCache.lastForceRefresh.clear();
    }
};

[YozenPL]

It is putting the log:

But I set the threshold to 69 and it stays as 69 in cache file. Maybe after the threshold - it is stopping the "local count" of the stamina? Or maybe it is counting it, but not updating value in the cache?

[torikushiii]

after invalidating, the cache should be removed from the cache file and will request new data to HoyoLab, does it still sends notification? Im starting to think this is maybe related to read/write issue, not sure yet tho becuase you said GI work just fine...

that logs tells me that the invalidation works just perfectly just fine.

[YozenPL]

Can you add to the logs the stamina counter? (the value from variable responsible for local counter). Because even if it somehow does not fetch the data from ZZZ it still should count the stamina right?

[YozenPL]

Ok SOMETHING strange. I got 2024-08-20 20:00:07 <INFO:ZenlessZoneZero:CheckIn> (XXXX) XXX Today's Reward: Crystallized Plating Agents x2

Messages on the console and stamina in cache updated from 69 to 79. Also the DEBUG logs was shown. Now I believe stamina will stay at 79. I'll put an update in ~30mins.

[torikushiii]

i've added a few extra log points

module.exports = class DataCache {
    static data = new Map();
    static expirationInterval;
    static lastForceRefresh = new Map();

    constructor (expiration, rate, forceRefreshInterval = 3_600_000) {
        this.expiration = expiration;
        this.rate = rate;
        this.forceRefreshInterval = forceRefreshInterval;

        if (!DataCache.expirationInterval) {
            DataCache.expirationInterval = setInterval(() => {
                DataCache.data.clear();
                DataCache.lastForceRefresh.clear();
            }, this.expiration);
        }
    }

    async set (key, value, lastUpdate = Date.now()) {
        const data = { ...value, lastUpdate };
        DataCache.data.set(key, data);
        DataCache.lastForceRefresh.set(key, lastUpdate);

        if (app.Cache) {
            await app.Cache.set({
                key,
                value: data,
                expiry: this.forceRefreshInterval
            });
        }
    }

    async get (key) {
        let cachedData = DataCache.data.get(key) || (app.Cache && await app.Cache.get(key));

        console.log("1", cachedData?.stamina);
        if (cachedData) {
            cachedData = await this.#updateCachedData(cachedData, key);
            console.log("2", cachedData);
            if (cachedData) {
                DataCache.data.set(key, cachedData?.stamina);
            }
        }

        console.log("3", cachedData?.stamina);
        return cachedData;
    }

    async #updateCachedData (cachedData, key) {
        const now = Date.now();
        const secondsSinceLastUpdate = (now - cachedData.lastUpdate) / 1000;
        const hourInSeconds = 3600;

        // replace uid with xxxx except the first 2 characters
        const uid = key.replace(/(?<=.{2})./g, "x");

        console.log({
            message: "[DEBUG] Checking cache for data update",
            uid,
            oldCachedData: cachedData.stamina
        });

        if (secondsSinceLastUpdate > hourInSeconds) {
            console.log(`[DEBUG] Force refreshing cache for ${uid} due to time since last update: ${secondsSinceLastUpdate}s`);
            await DataCache.invalidateCache(cachedData.uid);
            return null;
        }

        const account = app.HoyoLab.getAccountById(cachedData.uid);
        console.log({
            message: "[DEBUG] Old data",
            uid,
            threshold: account.stamina.threshold,
            oldData: cachedData.stamina
        });

        if (cachedData.stamina) {
            const recoveredStamina = Math.floor(secondsSinceLastUpdate / this.rate);
            cachedData.stamina.currentStamina = Math.min(
                cachedData.stamina.maxStamina,
                cachedData.stamina.currentStamina + recoveredStamina
            );
            cachedData.stamina.recoveryTime = Math.max(0, cachedData.stamina.recoveryTime - Math.round(secondsSinceLastUpdate));

            const staminaAlmostFull = cachedData.stamina.maxStamina - cachedData.stamina.currentStamina <= 10;
            const staminaThresholdReached = cachedData.stamina.currentStamina >= account.stamina.threshold;

            console.log({
                message: "[DEBUG] Updated data",
                recoveredStamina,
                currentStamina: cachedData.stamina.currentStamina,
                recoveryTime: cachedData.stamina.recoveryTime,
                staminaAlmostFull,
                staminaThresholdReached
            });

            if (cachedData.stamina.currentStamina >= cachedData.stamina.maxStamina
                || staminaThresholdReached
                || staminaAlmostFull) {
                console.log(`[DEBUG] Force refreshing cache for ${uid} due to stamina recovery`);
                await DataCache.invalidateCache(cachedData.uid);
                return null;
            }
        }

        if (cachedData.expedition && cachedData.expedition.list.length > 0) {
            for (const expedition of cachedData.expedition.list) {
                expedition.remaining_time = Math.max(0, Number(expedition.remaining_time) - Math.round(secondsSinceLastUpdate));
                if (expedition.remaining_time <= 0) {
                    console.log(`[DEBUG] Force refreshing cache for ${uid} due to expedition completion`);
                    await DataCache.invalidateCache(cachedData.uid);
                    return null;
                }
            }
        }

        if (cachedData.shop && cachedData.shop.state === "Finished") {
            console.log(`[DEBUG] Force refreshing cache for ${uid} due to shop state`);
            await DataCache.invalidateCache(cachedData.uid);
            return null;
        }

        if (cachedData.realm) {
            cachedData.realm.recoveryTime = Math.max(0, cachedData.realm.recoveryTime - Math.round(secondsSinceLastUpdate));
            if (cachedData.realm.currentCoin >= cachedData.realm.maxCoin || cachedData.realm.recoveryTime <= 0) {
                console.log(`[DEBUG] Force refreshing cache for ${uid} due to realm recovery`);
                await DataCache.invalidateCache(cachedData.uid);
                return null;
            }
        }

        cachedData.lastUpdate = now;
        DataCache.lastForceRefresh.set(key, now);
        return cachedData;
    }

    static async invalidateCache (key) {
        console.log(`[DEBUG] Invalidated cache for ${key}`);
        DataCache.data.delete(key);
        DataCache.lastForceRefresh.delete(key);
        if (app.Cache) {
            await app.Cache.delete(key);
        }
    }

    static destroy () {
        clearInterval(DataCache.expirationInterval);
        DataCache.data.clear();
        DataCache.lastForceRefresh.clear();
    }
};

[YozenPL]

I saw another strange thing from ~19:00:

Debug for ZZZ was executed only once when it fetched the 69 stamina. And after that DEBUG is no longer executed for it.

[torikushiii]

i've added a few extra log points

can you try this one, i just updated it again and added more few extra log points and censoring your UID

[YozenPL]

Yes I'm testing it. I'm waiting for round hour because between 20:22 and "now", there was only 1 log.

[YozenPL]

FYI: With thins new cache.js even GI is not updating the data in cache file.

[torikushiii]

im dumb... i putted wrong ternary at the wrong place, can you try this one, you should see recoveryTime got updated

module.exports = class DataCache {
    static data = new Map();
    static expirationInterval;
    static lastForceRefresh = new Map();

    constructor (expiration, rate, forceRefreshInterval = 3_600_000) {
        this.expiration = expiration;
        this.rate = rate;
        this.forceRefreshInterval = forceRefreshInterval;

        if (!DataCache.expirationInterval) {
            DataCache.expirationInterval = setInterval(() => {
                DataCache.data.clear();
                DataCache.lastForceRefresh.clear();
            }, this.expiration);
        }
    }

    async set (key, value, lastUpdate = Date.now()) {
        const data = { ...value, lastUpdate };

        DataCache.data.set(key, data);
        DataCache.lastForceRefresh.set(key, lastUpdate);

        if (app.Cache) {
            await app.Cache.set({
                key,
                value: data,
                expiry: this.forceRefreshInterval
            });
        }
    }

    async get (key) {
        // 1. Attempt to get data from memory cache
        let cachedData = DataCache.data.get(key);
        if (cachedData) {
            return this.#updateCachedData(cachedData);
        }

        // 2. Attempt to get data from keyv cache
        if (app.Cache) {
            cachedData = await app.Cache.get(key);
            if (cachedData) {
                DataCache.data.set(key, cachedData);
                return this.#updateCachedData(cachedData);
            }
        }

        return null;
    }

    async #updateCachedData (cachedData) {
        const now = Date.now();
        const secondsSinceLastUpdate = (now - cachedData.lastUpdate) / 1000;
        const hourInSeconds = 3600;

        // replace uid with xxxx except the first 2 characters
        const uid = cachedData.uid.replace(/(?<=.{2})./g, "x");

        // console.log({
        //  message: "[DEBUG] Checking cache for data update",
        //  uid,
        //  oldCachedData: cachedData.stamina
        // });

        if (secondsSinceLastUpdate > hourInSeconds) {
            console.log(`[DEBUG] Force refreshing cache for ${uid} due to time since last update: ${secondsSinceLastUpdate}s`);
            await DataCache.invalidateCache(cachedData.uid);
            return null;
        }

        const account = app.HoyoLab.getAccountById(cachedData.uid);
        console.log({
            message: "[DEBUG] Old data",
            uid,
            threshold: account.stamina.threshold,
            oldData: cachedData.stamina
        });

        if (cachedData.stamina) {
            const recoveredStamina = Math.floor(secondsSinceLastUpdate / this.rate);
            cachedData.stamina.currentStamina = Math.min(
                cachedData.stamina.maxStamina,
                cachedData.stamina.currentStamina + recoveredStamina
            );
            cachedData.stamina.recoveryTime = Math.max(0, cachedData.stamina.recoveryTime - Math.round(secondsSinceLastUpdate));

            const staminaAlmostFull = cachedData.stamina.maxStamina - cachedData.stamina.currentStamina <= 10;
            const staminaThresholdReached = cachedData.stamina.currentStamina >= account.stamina.threshold;

            console.log({
                message: "[DEBUG] Updated data",
                recoveredStamina,
                currentStamina: cachedData.stamina.currentStamina,
                recoveryTime: cachedData.stamina.recoveryTime,
                staminaAlmostFull,
                staminaThresholdReached
            });

            if (cachedData.stamina.currentStamina >= cachedData.stamina.maxStamina
                || staminaThresholdReached
                || staminaAlmostFull) {
                console.log(`[DEBUG] Force refreshing cache for ${uid} due to stamina recovery`);
                await DataCache.invalidateCache(cachedData.uid);
                return null;
            }
        }

        if (cachedData.expedition && cachedData.expedition.list.length > 0) {
            for (const expedition of cachedData.expedition.list) {
                expedition.remaining_time = Math.max(0, Number(expedition.remaining_time) - Math.round(secondsSinceLastUpdate));
                if (expedition.remaining_time <= 0) {
                    console.log(`[DEBUG] Force refreshing cache for ${uid} due to expedition completion`);
                    await DataCache.invalidateCache(cachedData.uid);
                    return null;
                }
            }
        }

        if (cachedData.shop && cachedData.shop.state === "Finished") {
            console.log(`[DEBUG] Force refreshing cache for ${uid} due to shop state`);
            await DataCache.invalidateCache(cachedData.uid);
            return null;
        }

        if (cachedData.realm) {
            cachedData.realm.recoveryTime = Math.max(0, cachedData.realm.recoveryTime - Math.round(secondsSinceLastUpdate));
            if (cachedData.realm.currentCoin >= cachedData.realm.maxCoin || cachedData.realm.recoveryTime <= 0) {
                console.log(`[DEBUG] Force refreshing cache for ${uid} due to realm recovery`);
                await DataCache.invalidateCache(cachedData.uid);
                return null;
            }
        }

        cachedData.lastUpdate = now;
        DataCache.lastForceRefresh.set(cachedData.uid, now);
        return cachedData;
    }

    static async invalidateCache (key) {
        const uid = key.replace(/(?<=.{2})./g, "x");
        console.log(`[DEBUG] Invalidated cache for ${uid}`);
        DataCache.data.delete(key);
        DataCache.lastForceRefresh.delete(key);
        if (app.Cache) {
            await app.Cache.delete(key);
        }
    }

    static destroy () {
        clearInterval(DataCache.expirationInterval);
        DataCache.data.clear();
        DataCache.lastForceRefresh.clear();
    }
};

[YozenPL]

It was running fine until the thresholds was reached. ZZZ reached the threshold - it no longer receiving the DEBUG logs. Only GI logs are received. ZZZ current stamina remains static - 203. I typed my "logs" in the terminal - you will see it on the screenshot.

Is it possible to add timestamp to those DEBUG logs?

[YozenPL]

I'm not sure why but now the GI stamina is updated every hour only after the invalidation of the cache (no local counter).

//UPDATE GI nor ZZZ*

[torikushiii]

so the cache file updated just fine, yes? but not the local calculations?

[YozenPL]

Issue remains. Only when threshold < currentStamina it is working fine.

[YozenPL]

Analyzing the screenshots (in order):

1. DEBUG for GI and ZZZ is received normally, data related to stamina is normally updated in cache file
2. Thresholds for both games are reached
3. DEBUG for GI is only appearing on the console, and GI data is only updated but local update is not working. Only the update from the API call are updating the cache file.
4. There are still no DEBUG and cache file update for ZZZ.

[torikushiii]

sorry for the late reply, can you try this one, this should (i hope) fix this issue

module.exports = class DataCache {
    static data = new Map();
    static expirationInterval;
    static lastForceRefresh = new Map();

    constructor (expiration, rate, forceRefreshInterval = 3_600_000) {
        this.expiration = expiration;
        this.rate = rate;
        this.forceRefreshInterval = forceRefreshInterval;

        if (!DataCache.expirationInterval) {
            DataCache.expirationInterval = setInterval(() => {
                DataCache.data.clear();
                DataCache.lastForceRefresh.clear();
            }, this.expiration);
        }
    }

    async set (key, value, lastUpdate = Date.now()) {
        const data = { ...value, lastUpdate };

        DataCache.data.set(key, data);
        DataCache.lastForceRefresh.set(key, lastUpdate);

        if (app.Cache) {
            await app.Cache.set({
                key,
                value: data,
                expiry: this.forceRefreshInterval
            });
        }
    }

    async get (key) {
        // 1. Attempt to get data from memory cache
        let cachedData = DataCache.data.get(key);
        if (cachedData) {
            return this.#updateCachedData(cachedData);
        }

        // 2. Attempt to get data from keyv cache
        if (app.Cache) {
            cachedData = await app.Cache.get(key);
            if (cachedData) {
                DataCache.data.set(key, cachedData);
                return this.#updateCachedData(cachedData);
            }
        }

        return null;
    }

    async #updateCachedData (cachedData) {
        const now = Date.now();
        const secondsSinceLastUpdate = (now - cachedData.lastUpdate) / 1000;
        const hourInSeconds = 3600;

        // replace uid with xxxx except the first 2 characters
        const uid = cachedData.uid.replace(/(?<=.{2})./g, "x");

        // console.log({
        //  message: "[DEBUG] Checking cache for data update",
        //  uid,
        //  oldCachedData: cachedData.stamina
        // });

        if (secondsSinceLastUpdate > hourInSeconds) {
            console.log(`[DEBUG] Force refreshing cache for ${uid} due to time since last update: ${secondsSinceLastUpdate}s`);
            await DataCache.invalidateCache(cachedData.uid);
            return null;
        }

        const account = app.HoyoLab.getAccountById(cachedData.uid);
        console.log({
            message: "[DEBUG] Old data",
            uid,
            threshold: account.stamina.threshold,
            oldData: cachedData.stamina
        });

        if (cachedData.stamina) {
            const recoveredStamina = Math.floor(secondsSinceLastUpdate / this.rate);
            cachedData.stamina.currentStamina = Math.min(
                cachedData.stamina.maxStamina,
                cachedData.stamina.currentStamina + recoveredStamina
            );
            cachedData.stamina.recoveryTime = Math.max(0, cachedData.stamina.recoveryTime - Math.round(secondsSinceLastUpdate));

            const staminaAlmostFull = cachedData.stamina.maxStamina - cachedData.stamina.currentStamina <= 10;
            const staminaThresholdReached = cachedData.stamina.currentStamina >= account.stamina.threshold;

            console.log({
                message: "[DEBUG] Updated data",
                recoveredStamina,
                currentStamina: cachedData.stamina.currentStamina,
                recoveryTime: cachedData.stamina.recoveryTime,
                staminaAlmostFull,
                staminaThresholdReached
            });

            if (cachedData.stamina.currentStamina >= cachedData.stamina.maxStamina
                || staminaThresholdReached
                || staminaAlmostFull) {
                console.log(`[DEBUG] Force refreshing cache for ${uid} due to stamina recovery`);
                await DataCache.invalidateCache(cachedData.uid);
                return null;
            }
        }

        if (cachedData.expedition && cachedData.expedition.list.length > 0) {
            for (const expedition of cachedData.expedition.list) {
                expedition.remaining_time = Math.max(0, Number(expedition.remaining_time) - Math.round(secondsSinceLastUpdate));
                if (expedition.remaining_time <= 0) {
                    console.log(`[DEBUG] Force refreshing cache for ${uid} due to expedition completion`);
                    await DataCache.invalidateCache(cachedData.uid);
                    return null;
                }
            }
        }

        if (cachedData.shop && cachedData.shop.state === "Finished") {
            console.log(`[DEBUG] Force refreshing cache for ${uid} due to shop state`);
            await DataCache.invalidateCache(cachedData.uid);
            return null;
        }

        if (cachedData.realm) {
            cachedData.realm.recoveryTime = Math.max(0, cachedData.realm.recoveryTime - Math.round(secondsSinceLastUpdate));
            if (cachedData.realm.currentCoin >= cachedData.realm.maxCoin || cachedData.realm.recoveryTime <= 0) {
                console.log(`[DEBUG] Force refreshing cache for ${uid} due to realm recovery`);
                await DataCache.invalidateCache(cachedData.uid);
                return null;
            }
        }

        cachedData.lastUpdate = now;
        DataCache.lastForceRefresh.set(cachedData.uid, now);

        DataCache.data.set(cachedData.uid, cachedData);
        await app.Cache.set({
            key: cachedData.uid,
            value: cachedData,
            expiry: this.forceRefreshInterval
        });

        return cachedData;
    }

    static async invalidateCache (key) {
        const uid = key.replace(/(?<=.{2})./g, "x");
        console.log(`[DEBUG] Invalidated cache for ${uid}`);
        DataCache.data.delete(key);
        DataCache.lastForceRefresh.delete(key);
        if (app.Cache) {
            await app.Cache.delete(key);
        }
    }

    static destroy () {
        clearInterval(DataCache.expirationInterval);
        DataCache.data.clear();
        DataCache.lastForceRefresh.clear();
    }
};

[YozenPL]

I set the stamina cron to 5mins to get the updates faster. Stamina for GI is not refreshing/not counted locally?

[YozenPL]

To be honest let's leave it. I reverted back to original cache.js file and set the threshold to 240. If hoyolab-auto will reach 240, I'll use the stamina anyway and restart the application.

[torikushiii]

for genshin the recoveryTime runs just fine so its "should" be updating correctly as you can see here

but for ZZZ i have zero idea why it isn't doing anything for you. If i get this correctly ZZZ got invalidated once and after that the data become stale and doesn't update right?

also one last thing can you go to the main index.js file which is here https://github.com/torikushiii/hoyolab-auto/blob/main/index.js

and find this code

https://github.com/torikushiii/hoyolab-auto/blob/6856d520ea9756073b57b9a54b8e5ed865b403c2/index.js#L113-L122

then comment it into this:

    // process.on("unhandledRejection", (reason) => {
    //  if (!(reason instanceof Error)) {
    //      return;
    //  }

    //  app.Logger.log("Client", {
    //      message: "Unhandled promise rejection",
    //      args: { reason }
    //  });
    // });

and try to run it again one more time and let me know if it threw any errors

[YozenPL]

I'll look at it tomorrow :)

[YozenPL]

No additional logs were shown after editing index.js.

[torikushiii]

the original cache.js as of this commit? https://github.com/torikushiii/hoyolab-auto/commit/dbfd65d422daa7684f7390baafb8edabfd8109c9

and everything doesn't get updated only after ZZZ threshold got reached and ZZZ data is static until next restart? If so, I'm just gonna put another check at stamina cron