Valve requestRules return weird chars

[podrivo]

Hello! I'm getting weird chars in some of the Valve requestRules response. Here's the response I got:

"rules": {
  "\u0001\u0003": "\u0001\u0001\u0001\u0002\u0001\u0002\u0001\u0002\b�ж\u0017\u0004\u0001\u0002\b'k\u0012WindstrideClothing\u001a�\"�\u0004���\u0016Advanced Weapon Scopesq(��\u0004\u0011�}�\u000eMortys Weapons��\u0013\u0004��̒\u0007Esseker\u0006\u0015�\u0004f\u0010�}\u0006DayOne",
  "\u0002\u0003": "����\u0004�'9]\u0016Community Online Tools�\n\\\u0010\u0004\u0004��\\\u0013Community Framework����\u0013Ԓ\u0011\u000bLivonia DLC\n\u000bcftoolsRoot\u0004dayz\nEsseker0_5\u000bEsseker0_51\u000bEsse",
  "\u0003\u0003": "ker0_52\u0006Inkota\u000eJacob_Mango_V3\tMandibull\u0003TTC\nWindstride",
  "allowedBuild": "0",
  "dedicated": "1",
  "island": "esseker",
  "language": "65545",
  "platform": "win",
  "requiredBuild": "0",
  "requiredVersion": "112",
  "timeLeft": "15"
}

I was wondering how to convert it? Thank you! :)

[mmorrisontx]

This definitely looks broken. What game causes this? Do you have an example server ip?

[podrivo]

Ohhh, sorry about that... It's a DayZ server query. This is the server in the example above: https://www.battlemetrics.com/servers/dayz/11359652

Here are others IPs you can test: 208.52.153.250:2405, 147.135.31.183:2305, 104.152.143.10:27016. You can check more examples here: https://www.battlemetrics.com/servers/dayz

For context, the weird chars part is related to the Mods that server has.

"rules": {
  "\u0001\u0003": "\u0001\u0001\u0001\u0002\u0001\u0002\u0001\u0002\b�ж\u0017\u0004\u0001\u0002\b'k\u0012WindstrideClothing\u001a�\"�\u0004���\u0016Advanced Weapon Scopesq(��\u0004\u0011�}�\u000eMortys Weapons��\u0013\u0004��̒\u0007Esseker\u0006\u0015�\u0004f\u0010�}\u0006DayOne",
  "\u0002\u0003": "����\u0004�'9]\u0016Community Online Tools�\n\\\u0010\u0004\u0004��\\\u0013Community Framework����\u0013Ԓ\u0011\u000bLivonia DLC\n\u000bcftoolsRoot\u0004dayz\nEsseker0_5\u000bEsseker0_51\u000bEsse",
  "\u0003\u0003": "ker0_52\u0006Inkota\u000eJacob_Mango_V3\tMandibull\u0003TTC\nWindstride",
}

So in this example, you have the mods:

• WindstrideClothing
• Advanced Weapon Scopes
• Mortys Weapons
• Esseker
• DayOne
• Community Online Tools
• Community Framework

I believe the weird chars have also the Steam Workshop ID's, because that's how Steam identify them in their system. WindstrideClothing would be https://steamcommunity.com/workshop/filedetails/?id=1797720064

And maybe there's more extra info, because I see some Mods Author names, like in this line:

"\u0003\u0003": "ker0_52\u0006Inkota\u000eJacob_Mango_V3\tMandibull\u0003TTC\nWindstride"

Jacob_Mango, Inkota, ker0_52, Mandibull, Windstride

[mmorrisontx]

This was definitely an adventure to figure out. There's some... special encoding in there. I mostly just blindly reverse engineered it, so hopefully it's stable :). There's still an unknown field on each mod that I couldn't figure out, but you're right that the workshop id was in there.

[podrivo]

That's awesome, @mmorrisontx! Thank you so much for putting the effort! Did some testing with different servers and it worked great on most of them!

However, I found one server where I got a similar response with weird chars. This is the server I tested: 198.251.83.51:2405

Here's part of the response I got:

{
  "unknown": 73122578,
  "workshopId": 274674285,
  "title": "inter ChernaruszTz$\u0004]�k�\u0013Rearmed Server Pack��\u001d�\u0004�\u0007��\u0012Helicrash Licensed\u0016i�Y\u0004�b\t�\u0013Rearm"
}

I did some exploring about how to handle this kind of response and found a different method. I don't understand the code so sorry I can't help, but maybe this can help with any insights! You can take a look here: https://github.com/Jadfii/dayzmagiclauncher/blob/fc83a80fb461bdb26e6f83f5f4423a9c3ff0aa8a/src/main/query.js#L50-L157

———

Also, I had a conversation with a DayZ modder, that suggested using a second method to retrieve DayZ mods. It turns out, in most modded servers there's a tool that provides a response with mods + ids, by pointing to IP:GAME_PORT+10. So for the example server with the weird chars, you can get a response with all mods here: 198.251.83.51:2412

———

And about the unknown field, do you think it could be related to the mod author? I say this, because in one of the servers that I got a perfect response with title and workshopId, I also got the mod author's names. This is the server I tested: 135.148.136.66:2405

Here's the dayzMods response I got:

[{
        "unknown": 3930091765,
        "workshopId": 1819514788,
        "title": "Ear Plugs"
    },
    {
        "unknown": 347648180,
        "workshopId": 1999076953,
        "title": "Notes"
    },
    {
        "unknown": 397856978,
        "workshopId": 1797720064,
        "title": "WindstrideClothing"
    },
    {
        "unknown": 3592609562,
        "workshopId": 2143128974,
        "title": "Advanced Weapon Scopes"
    },
    {
        "unknown": 3687731737,
        "workshopId": 2141780688,
        "title": "ArmA 2 ACR Weapon Pack"
    },
    {
        "unknown": 1604478857,
        "workshopId": 1793351435,
        "title": "[Remastered] Arma Weapon Pack"
    },
    {
        "unknown": 343755824,
        "workshopId": 1665663702,
        "title": "MoreGuns"
    },
    {
        "unknown": 1458572568,
        "workshopId": 2110459075,
        "title": "DayOne"
    },
    {
        "unknown": 3600277669,
        "workshopId": 1564026768,
        "title": "Community Online Tools"
    },
    {
        "unknown": 274467515,
        "workshopId": 1559212036,
        "title": "Community Framework"
    },
    {
        "unknown": 3397756556,
        "workshopId": 294835219,
        "title": "Livonia DLC"
    },
    {"title": "A2Weaponsv3"},
    {"title": "cftoolsRoot"},
    {"title": "CooltrainV3"},
    {"title": "dayz"},
    {"title": "Inkota"},
    {"title": "Jacob_Mango_V3"},
    {"title": "Mandibull"},
    {"title": "NotABananaV3"},
    {"title": "Wardog.v3"},
    {"title": "Windstride"}
]

Let me know if any of this makes sense! Thanks for all the help! 😄 ✨

[mmorrisontx]

@podrivo Give 3.0.7 a shot, it should fix the corruption issue on that server. I'm reluctant to add a secondary port request for dayz here, since most servers won't need it and we're already getting these through the main rules request. Regarding the "mystery" number, I actually also suspected something author-related, but it didn't seem to correlate to any steamids or anything for the mod creators I could find. The dayzmagiclauncher you linked refers to it as a "mod hash", so it's probably not very useful and I've removed it from dayzMods in 3.0.7.

[podrivo]

OMG @mmorrisontx! Response is looking so much better now! 😍

I was testing on a more simple server, with less mods and less complex information, and got this response. This is the server I tested: 147.135.31.183:2305

"dayzMods": [
    {
        "workshopId": 1797720064,
        "title": "WindstrideClothing"
    },
    {
        "workshopId": 2143128974,
        "title": "Advanced Weapon Scopes"
    },
    {
        "workshopId": 2155726353,
        "title": "Mortys Weapons"
    },
    {
        "workshopId": 2462896799,
        "title": "Esseker"
    },
    {
        "workshopId": 2110460006,
        "title": "DayOne"
    },
    {
        "workshopId": 1564026768,
        "title": "Community Online Tools"
    },
    {
        "workshopId": 1559212036,
        "title": "Community Framework"
    },
    {
        "workshopId": 294835219,
        "title": "Livonia DLC"
    },
    {"title": "cftoolsRoot"},
    {"title": "dayz"},
    {"title": "Esseker0_5"},
    {"title": "Esseker0_51"},
    {"title": "Esseker0_52"},
    {"title": "Esseker0_55"},
    {"title": "Inkota"},
    {"title": "Jacob_Mango_V3"},
    {"title": "Mandibull"},
    {"title": "TTC"},
    {"title": "Windstride"}
],

And I realized that we also have author names, like {"title": "Windstride"} . So I was wondering, do you think we can link the author names with their mods? For example, we would have something like:

 {
  "workshopId": 1797720064,
  "title": "WindstrideClothing",
  "author": "Windstride"
}

What do you think?

———

I 100% agree with not adding a secondary port query. This is not specific to a valve server and not always will be present. And it makes sense to remove the mod hash. Nice! 🙌

[mmorrisontx]

I don't /think/ those are author names. Otherwise, I'd expect them to line up 1:1 with the workshop mods. Is it possible there's a mod, or part of a mod, that's being registered that's literally just called Windstride?

[podrivo]

This specific example {"title": "Windstride"} is definitely the author/publisher's name. You can check the "WindstrideClothing" example here: https://steamcommunity.com/workshop/filedetails/?id=1797720064

But I'm not sure that's the only thing in the response, because these examples look like patch versions for this mod, called Esseker: https://steamcommunity.com/workshop/filedetails/?id=2462896799

{"title": "Esseker0_5"},
{"title": "Esseker0_51"},
{"title": "Esseker0_52"},
{"title": "Esseker0_55"},

And by checking each workshopId, I don't see all author's names in the response. For example, this one: {"workshopId": 2155726353, "title": "Mortys Weapons"} the author's name is Morty, but it's not included in the response.

So I'm guessing there's some extra filtering in the response to get all data? Let me know if it makes sense!

[mmorrisontx]

All of those "mods" without a workshopId are presented by the server in a simple single flat list of strings. There's nothing that would identify them as an author, patch, or otherwise, so I'm not sure how else we could filter it.

[podrivo]

Hmmm, that's unfortunate. This info is probably from the mod itself, and maybe will depend on a specific file included in the mod's folder, that may or may not be there. I got this sense from the conversation I had with the DayZ modder I linked above. It would be cool to have this bind together, but since you managed to grab the name + id, I think that's all we need!

So maybe let's remove this altogether? It's not adding any useful info related to mods and with the mod id, we can use an alternative service to grab the full mod info, like description, releases, authors etc.

[aclist]

Excellent work on this implementation. I wanted to make you aware that this does not support the rules key on DayZ Experimental servers (https://dayz.fandom.com/wiki/Experimental).

"Experimental" servers are a separate testing branch that is available to server operators. These load Steam Workshop mods in the typical fashion, but the A2S response is encoded differently as far as the rules key is concerned and is getting interpreted as a UTF-8 string:

https://www.battlemetrics.com/servers/dayz/5485515

"name": "DayZ US - LA 3599 (Experimental/Unstable)",
  "map": "chernarusplus",
  "password": false,
  "raw": {
    "protocol": 17,
    "folder": "dayz",
    "game": "DayZ Exp",
    "appId": 1024020,
    "numplayers": 16,
    "numbots": 0,
    "listentype": "d",
    "environment": "w",
    "secure": 1,
    "version": "1.21.156165",
    "steamid": "90172469402110980",
    "tags": [
      "battleye",
      "shard000",
      "lqs0",
      "etm6.200000",
      "entm1.700000",
      "07:18"
    ],
    "rules": {
      "\u0001\u0001": "\u0001\u0001\u0001\u0002\u0001\u0002\u0001\u0002\u0001\u0002\u0001\u0001\u0004dayz",
      "allowedBuild": "0",
      "dedicated": "1",
      "island": "chernarusplus",
      "language": "65545",
      "platform": "win",
      "requiredBuild": "0",
      "requiredVersion": "121" "timeLeft": "15"
    }