Open Gatsik opened 2 years ago
test_ladder_game_draw_bug
and test_game_ended_broadcasts_rating_update
are failing probably because of customized precision
Do you want this custom precision only for queue_pop_time_delta
field in matchamker_info
message?
Hey, nice work! I think it makes sense to use the same precision for everything, so those tests probably need to be adjusted to have higher tolerance for error.
The second one is interesting because it means there will be some edge cases where the rating change is so small that it doesn’t appear to change at all. I think that’s ok though since the full change history will still be available through the replay details page and the rating graph.
One thing we have to keep in mind is that if we’re messing with the json encoding, we need to be very careful of the performance impact since this is one of the hottest functions in the entire code base. It would be amazing if we could just override the float serialization code and leave the rest of the implementation untouched. I found a little snippet that does this although it uses the mock.patch
functionality to override private functions from the Json library. Still, this may be a good source of inspiration:
https://gist.github.com/Sukonnik-Illia/ed9b2bec1821cad437d1b8adb17406a3
It looks like we can't do much without creating custom python package that will use compiled C code. I did some comparison between different methods using slightly modified script taken from https://developpaper.com/speed-comparison-of-five-json-libraries-in-python/ and here are the results of encoding on my machine:
patching (example given above): 107.800
current (this pr's code): 28.629
Default serialization
simplejson: 6.981
ujson: 3.503
stdlib json: 4.500
Round floats approach (https://stackoverflow.com/a/53798633):
simplejson: 14.196
ujson: 10.481
stdlib json: 11.109
Overriding iterencode
function, which contains floatstr
method, directly without patching:
15.991
Compiling custom mix of json
and simplejson
(most of the code is a copy from https://github.com/python/cpython/pull/13233/, built with setup.py
from simplejson
):
6.589
Time is in seconds, the data for encoding is:
{
"command": "game_info",
"visibility": "public",
"password_protected": True,
"uid": 13,
"title": "someone's game",
"state": "playing",
"game_type": "custom",
"featured_mod": "faf",
"sim_mods": {},
"mapname": "scmp_009",
"map_file_path": "maps/scmp_009.zip",
"host": "Foo",
"num_players": 2,
"launched_at": 1111111111.1112312312312,
"rating_type": "faf",
"rating_min": None,
"rating_max": None,
"enforce_rating_range": False,
"team_ids": [
{
"team_id": 1,
"player_ids": [1],
},
{
"team_id": 2,
"player_ids": [2],
},
],
"teams": {
1: ["Foo"],
2: ["Bar"],
},
}
There is also a package called orjson
which is super fast, but requires dict
keys to be strings, so with modified initial data, in which all keys are strings, the result is:
0.624
Otherwise, using "round floats" approach in which we also convert dict
keys to str
:
8.457
Closes #875