Stream orders not resetting after OSRS server reset

[Padje1994]

plug-in fails to set the new stream order after reset. plug-in shows that world 302 has 583 hits for stream order GGGBBB while the stream order is GBGGBB

[jcarbelbide]

Hey, thanks for reporting this. This is unfortunately a limitation with the current system that may occasionally happen. There is a service on the server that monitors a connection to the osrs servers, and when that connection breaks, a server reset is detected, and should wipe the data. Unfortunately, this can fail if the togcrowdsourcing server is down when the reset happens. This may have been what happened.

There are ways to mitigate this, but the solutions I can think of have downsides, and since this is a fairly rare occurrence, I'm inclined to leave it as is for now, unless we get more reports of this.

I'll leave this issue open in case this happens again.

[Killako]

Issue happened again after server reset on Wednesday. https://www.togcrowdsourcing.com/lastreset still says 6/26 was the previous server reset, which is incorrect. Can you manually reset ToG?

Thanks

[jcarbelbide]

Hey, thanks for the follow up. Didn't realize it's been outdated for that long. Sorry about that! I went ahead and reset the data so it should be fresh. I think one thing we can do to mitigate it being outdated for too long is to have a built in once a week reset at least. That'll be the next thing I add

Recently, due to high load, the server has been having a lot of periods of downtime. I think the servers may have reset during this time, which is why the reset was missed. That is also being worked on when I have time

[Killako]

Totally understandable. As far as I am aware, there is a server reset every Wednesday (GMT), I believe.

[Killako]

Hey, thanks for the follow up. Didn't realize it's been outdated for that long. Sorry about that! I went ahead and reset the data so it should be fresh. I think one thing we can do to mitigate it being outdated for too long is to have a built in once a week reset at least. That'll be the next thing I add

Recently, due to high load, the server has been having a lot of periods of downtime. I think the servers may have reset during this time, which is why the reset was missed. That is also being worked on when I have time

Another thought I had was to change the storage method from a JSON file to a hashmap, with the key being the world and the data being the stream order. That way, even if the worlds reset, the stream order should just update to whatever is the latest one sent in.

[mijkolsmith]

It seems to be wrong again. Should be easy to check the hits you get against the data already in the database, if the data is conflicting on the last few submissions (from different sources) you could do an automatic reset maybe?

[jcarbelbide]

Hey @mijkolsmith , @Killako thanks for the notification and suggestions. I've just manually reset it. This time, I don't see a significant period of downtime last night, assuming that is when the servers reset (i don't actually actively play much anymore, so I'm not sure when the update happened). To rule out server load completely, the ideal thing would be to move the js5-monitoring service (the service that monitors for updates) over to a new server, but that costs $$. Something like you mention could work, I'll talk more about why it doesn't do that currently below, but we could possibly alter it.

First, @Killako - I wanted to respond to your comment as well - The data is not actually stored in a JSON file, it only appears as a json to the endpoints. The data is stored in a database, and each world+stream order gets a row. I chose to do that because the endpoint is public, and anybody with malicious intent could wipe the world data if they wanted to by submitting false information. So that's why we need the whole reset process. Additionally, we use a database so that if the server has to reset, we don't lose all the data, where we would if we stored everything in memory. Forgive me if I'm misunderstanding your suggestion though!

@mijkolsmith you suggested resetting if the last few submissions are different. At first, I wanted to avoid giving anyone power to reset worlds, so I went with the osrs server monitor. Your suggestion is good, but the drawback is that if someone wanted, they could send a bunch of incorrect data to the server and constantly induce a data reset. We do block repeat requests from the same IP, but I don't think spoofing IPs is too difficult for a determined person. We can try this, and it's possible that no one is interested in messing with this tool and everything will be fine.

I'm going to start with a few other things I had planned to reduce server load first, and manually monitor the resets moving forward. I'll look into some free services we could put the monitor on, but I don't think anything that fits our current needs is free.

Edit: I will also think of how we can induce a reset if data is wrong. I've saved the data from the last reset with outdated information, and I'll try to analyze if theres a reasonable pattern that can that there was a reset that wasn't deteceted.

[Lorentary]

Plugin's still broken

[jcarbelbide]

@Lorentary thanks for the notification. I haven't had time to push any changes this week. I manually reset the servers for now, and will continue with that every Wednesday until it's fixed

[Justin-Folvarcik]

Why not set a cron job to reboot every Wednesday? It would make a pretty easy temp solution.

[mijkolsmith]

If you're worried about abuse, you could just add a cooldown on the reset period, like an hour for example.

[jcarbelbide]

@Justin-Folvarcik @mijkolsmith , thank you for your suggestions. I did set up a cron job, but the server resets unfortunately aren't always predictable. Sometimes the servers will reset in the middle of the week, or the update will happen early/late, so the cron job doesn't work great. Today is an example (looks like there was an issue with the login servers, so they reset the servers).

Adding a cooldown on the reset is still open to abuse, and I could see this logic being a bit complicated. It's not a bad idea, but a few questions come to mind - What would be the trigger? Some x number of worlds have conflicting data? How do we decide x, or an appropriate cooldown? I foresee a lot of playing around with numbers. Definitely possible, but I think the right solution is to fix the server monitor. I apologize I haven't had time to work on this lately. Life and work have been busy

I made some time today to work on this, and it looks like the reason the osrs server monitor broke was due to a change in the JS5 protocol that (to me) appears to have been backwards incompatible with previous versions. JS5 is a Jagex protocol I'm using to check if servers have reset. I pushed a change that restores the connection (when I tested locally) and I hope this will catch the reset tomorrow morning. I'll keep a close eye on it.

[jcarbelbide]

Looks like it reset this morning. I'll leave this issue open for another week before closing it to make sure the fix is solid.

[jcarbelbide]

closing, as the issue of servers not resetting when servers go down appears to have been resolved for now