Load cache from local or remote zip

[ESchouten]

Solves https://github.com/Dezinater/osrscachereader/issues/1 Added cache loading via downloading and unzipping remote and local zip files

Prevents https://github.com/Dezinater/osrscachereader/issues/3 Resolves undefined when the xteas file is not found, it crashed on my cache version

Pinned web-worker 1.2 since 1.3.0 gave me an error.

NOTE: package-lock.json not updated since prebuilt binaries for canvas are not available for ARM mac, and npm fails to build it. I succeeded with pnpm, but did not want to include that lock file, so you have to run npm i yourself after this PR

[ESchouten]

@Dezinater You said you have a couple of more ideas for the zip loading, I'm interested, could you reveal them?

[Dezinater]

The idea was basically just being able to load either the latest cache or a specific cache version number from passing a number to it

Maybe latest can be something like this let cache = new RSCache("latest", (x) => { console.log(x) }, "./"); let cache = new RSCache(-1, (x) => { console.log(x) }, "./"); let cache = new RSCache(Infinity, (x) => { console.log(x) }, "./");

And then loading a specific version let cache = new RSCache(219, (x) => { console.log(x) }, "./");

It should just be a small change but this would be very useful for testing I think

Also another somewhat related concept, there is an API on OpenRS2 to directly load cache file data. I was told about it in the OpenRS2 discord and they sent me this code https://github.com/Pazaz/RT5-Server/blob/main/util/OpenRS2.js

[ESchouten]

Interesting concepts, the version number based retrieval indeed seems to be very useful for testing...

As for the dynamic cache loading, interesting as well, might be useful depending on the requirements of your application e.g displaying icons and models on a regular website. Downloading tens of MB for just that is not going to work. I guess for a lot of other applications direct, fast access to a local version of the cache is important for speed and reliability reasons as well as the offline availability of the data.

What I'm thinking is: We could make a cache loading API that supports various cache loading strategies.

1. On demand: only load the files when they are needed and store retrieved items for future usage. Fast initial load and limits the applications data usage. Unreliable for offline usage and suffers from delays when loading new items.
2. Full download: Download and store the full cache. Slow initial load and causes a large data usage. Fully available offline and fast item retrieval.
3. Hybrid: Use the on demand strategy until the full download is complete. Fast initial load and once completed, fully available offline and fast item retrieval

[ESchouten]

We could also support various storage mechanisms behind an interface, e.g. fs like the sample above, but also innodb for in-browser caching. I guess caching should be optional so consumers could do their own caching via e.g. service-workers with workbox

[Dezinater]

I like the idea of having different modes and especially like the hybrid idea of having it use the on demand mode until the cache is loaded. For caching I did originally have something for it but I ended up having to remove it because a user had issues with it. I think now would be a good time to look into it again.

Also I've created a discussions section on this repo so we can post ideas in there now

[ESchouten]

@Dezinater Is this PR good enough to get merged so we can load local and remote zips? Or do you want to implement additional features first?

[Dezinater]

I think it is good to merge but I would like to add the additional features first. I've just been a bit busy but I should be able to add it in this weekend or you can add the changes if you'd like

Also there's multiple caches per version because of how they patch the game. Do we want a feature to grab by closest timestamp as well or should version number only be good enough?

[Dezinater]

I've added a few ways of loading the cache. You can load it by version, date or latest.

For latest it can either be the string 'latest' or undefined let cache = new RSCache(); let cache = new RSCache("latest");

For timestamp it has to be a Date object let cache = new RSCache(new Date(12345));

And version is just a number let cache = new RSCache(220);

I also had to add a new promise in CacheLoader since it wasn't able to use async functions. There would be some async race conditions so I added a cachePromise which resolves whenever CacheLoader is done it's job