search

pdinklag / MinecraftStats

A Minecraft player statistics browser for the web - supports 1.13 and later!
https://discord.gg/brH5PGG8By
Other
227 stars 53 forks source link

Error while running update.py player = players[uuid] KeyError: '00000000-0000-0000-0009-01fa34ab1ee1' #111

Closed Leonardo-Fiori closed 3 years ago

Leonardo-Fiori commented 3 years ago 
updating profile for b65a4a00-589d-4259-8984-97c137d0b89b ...
updating profile for 473d9691-d6bf-4494-be13-2abf0b664f33 ...
updating profile for 2436b3f9-35e5-42d2-a1e0-c9b9bee14f70 ...
updating profile for ed5f3352-f43d-4113-8a64-b492b70331fc ...
updating profile for eaa87174-2d28-4666-aefe-988e08e8ab5f ...
updating profile for df713b79-1b7b-4e07-9bda-93cbe0a3b56e ...
Traceback (most recent call last):
  File "update.py", line 578, in <module>
    player = players[uuid]
KeyError: '00000000-0000-0000-0009-01fa34ab1ee1'

Update.py was running since 3/4 hours (i have many player files) and suddenly stopped with this error.

image

Any idea? Some players login trough an amazing piece of sofware called Geyser Proxy, essentially it allows pocket edition (bedrock) players to play on java edition. The authentication happens on my Bungeecord Proxy Server Proxy, with a plugin called floodgate, that generates uuids for them.

Paper version git-Paper-136 Python 3.7.3

pdinklag commented 3 years ago

It's OK that it fails getting data from the Mojang API because those IDs aren't real, but it shouldn't be crashing, that's a bug.

In general, though, I'm not supporting non-Mojang IDs right now. So once the crash is fixed, note that you will only see UUIDs in the browser instead of names. I might work on this to use the player cache names in the future for fake UUIDs, but I'll open another issue on that.

Leonardo-Fiori commented 3 years ago

Maybe as a quick fix before implementing non-mojang uuids, could we simply hide them? It would be really ugly to see uuids :) Then, take your time, and consider implementing it properly if you want!

pdinklag commented 3 years ago

After having a look at the code, I'm a bit puzzled as to why this happens, because apparently what I said should already be happening.

I'll try to explain the problem in a nutshell: one of your bedrock players appears to be the best at some award and is supposed to appear on the front page, i.e., he is part of what I call the "summary". But there seems to be a contradiction I don't understand, because that player's ID seems to be unknown when the summary is created.

So I'd like to find out how exactly he got in the summary and how this can happen. I'll open a branch about it with a version of update.py that I'd like you to try, will get back here in a few minutes,

pdinklag commented 3 years ago

Could you please try running the update.py file in this branch: https://github.com/pdinklag/MinecraftStats/tree/debug_issue_111

  1. It should be printing some lines like "adding [UUID] to summary because ...", and I'd be interested in any line ending with "registered: False".
  2. It should no longer crash like before, but instead write a line starting with "ERROR" that I'd also like to have.

This should hopefully get a working stat page for you (albeit maybe with some "???" entries for the players it failed on) and help me find out what exactly is wrong. :smiley:

Leonardo-Fiori commented 3 years ago

Just replaced update.py with your new version! Thanks for the quick edit :) Testing... for now i see only stuff like updating profile for 00000000-0000-0000-0009-01f5b2c1b32d waiting for it to get stuck on something!

Leonardo-Fiori commented 3 years ago

We got some true, some false and some ERROR, it is currently stuck, looks like it stopped without crash message: https://hastebin.com/huranowaga.coffeescript

pdinklag commented 3 years ago

It should eventually have finished? If you have a lot of players, writing the client data may take some time.

Anyway, it's interesting to see that all and only the 000... UUIDs are affected. You get no (registered: False) line above the snippet you sent, right? I'll have another look tomorrow and try to understand what's going on. Thanks for giving this a try!

Leonardo-Fiori commented 3 years ago

As you can see here, all the registered: False are 000... UUIDs, BUT not all 000... UUIDs are registered: False For regular UUIDs, instead, they all look true

https://hastebin.com/ujalifosum.coffeescript

Thanks for giving this a try!

No problems really, thanks for being so willing to fix issues 😄 usually this geyser proxy brings some incompatibilities here and there also with regular spigot plugins. Some devs refuse to even look at it! As of now 2000+ servers are using Geyser, and the number is steadily growing. Finally the poor bedrock players have a decent way to play the game online 😆

It should eventually have finished?

If i keep re-sending the python3 command over and over every time it stops, i think it will eventually finish! https://www.peacefulvanilla.club/mcstats/index.html my plan is to rename index.html and move it into the root folder

Leonardo-Fiori commented 3 years ago

Looks like it has completed, take a look here: https://www.peacefulvanilla.club/mcstats/index.html#player:00000000-0000-0000-0009-01fa34ab1ee1

This is a bedrock player, it recognizes it's name. Very strange. Other bedrock players are marked as ??? instead

pdinklag commented 3 years ago

thanks for being so willing to fix issues

I'm slow adding new features, but fixing bugs is a priority on this project. :smile: It's good to see everything practically working well with the non-Mojang UUIDs.

But yes, the ??? players are the interesting ones and really have me puzzled. There must be stat files for them and somehow they got at least a bronze medal (rank 3 for some stat), but then again their UUID is not known, which is really a contradiction.

Is it possible for you to drop a stat file for one of those players, e.g., the 00000000-0000-0000-0009-01feaa9e5a20.json file from the stats directory?

pdinklag commented 3 years ago

I also added some more debug output to update.py, there should now be some output lines starting with DEBUG. If any of those contains a UUID for which the script later prints an ERROR, that confirms the alleged contradiction. I still haven't quite figured why this could happen. 🤔

Leonardo-Fiori commented 3 years ago

Is it possible for you to drop a stat file for one of those players, e.g., the 00000000-0000-0000-0009-01feaa9e5a20.json file from the stats directory?

You mean to upload here that file? Here it is! :)

00000000-0000-0000-0009-01feaa9e5a20.zip

pdinklag commented 3 years ago

The file looks totally fine to me, but when I put it on my server I can now reproduce your error exactly, so that's good news. 👍 I'll have a look at this soon enough!

pdinklag commented 3 years ago

Turns out this happened when the Mojang API returns nothing for a UUID (e.g. because it is invalid) and there is also no usercache.json entry. These players will now be completely excluded, because since their user cache entry expired, they have been inactive in a good while anyway.

The contradiction I spoke of happened because even though the player was filtered later, their stats and rankings were processed, making them appear in rankings even though they were excluded. It should now be fixed (master branch).

Leonardo-Fiori commented 3 years ago

updated mojang.py and update.py taking the files from master branch, now the outputs look different but i think it worked! Don't see any "finish" or "process completed" message tho, but it looks done.

Take a look here: https://www.peacefulvanilla.club/mcstats/index.html#players:1

Are bedrock players still supposed to appear there now that i re-run the command? Looks that ??? players have disappeared, and that's nice

pdinklag commented 3 years ago

updated mojang.py and update.py taking the files from master branch, now the outputs look different but i think it worked! Don't see any "finish" or "process completed" message tho, but it looks done.

Great! There's indeed no "finish" message. My idea was initially that if there is no output, everything's fine, since Minecraft Stats is usually invoked in an automated manner and the process return code (0 = success) suffices.

Are bedrock players still supposed to appear there now that i re-run the command? Looks that ??? players have disappeared, and that's nice

Yes, they are supposed to appear. I've had reports of people who wanted to see unregistered UUIDs before (probably cracked clients, though the reporters certainly won't admit), so I made Minecraft Stats look up the server's usercache.json file to get names for those players. Gettings their skins, however, is not possible.

What changed now is that players with unregistered UUIDs are ignored if there is no user cache entry, which happens if they have been away for a good while.

However, once they are in the Minecraft Stats database (in the data directory) with a valid name, they won't disappear unless you delete the data directory. Think of it as an extended user cache. 😃

Leonardo-Fiori commented 3 years ago

Wow, Amazing explaination! 😊 Thanks for all the good work. For now I will look into configuring a cronjob. How to get the return code zero tho?

pdinklag commented 3 years ago

Personally, I wrote a wrapper script that calls update.py and I registered a cronjob for the wrapper. I believe cron notifies you via the Linux internal mail daemon when a job failed (return code other than zero), but I'm no expert for cron, nor is this the place to discuss this. Feel free to contact me via mail or Discord.

Leonardo-Fiori commented 3 years ago

Sent you a request on ds! I am Hosom#3615