ValvePython / vdf

📜 Package for working with Valve's text and binary KeyValue format
https://pypi.org/project/vdf/
MIT License
165 stars 30 forks source link

Error when parsing Dota 2 item schema #39

Open carloslbello opened 3 years ago

carloslbello commented 3 years ago

I'm trying to write a script that uses information from Dota 2's item schema, but when I try to use the vdf module to parse the schema it gives me this error:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.9/site-packages/vdf/__init__.py", line 208, in load
    return parse(fp, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/vdf/__init__.py", line 175, in parse
    stack[-1][key] = _unescape(val) if escaped else val
TypeError: 'str' object does not support item assignment

I downloaded the schema from here (and I got that URL from Steam's WebAPI), and I get the same error on the PyPI and GitHub versions of the vdf module.

I appreciate any insight or help!

rossengeorgiev commented 3 years ago

That file most likely has entries with duplicate keys. I suggest using mapper=vdf.VDFDict, and check the readme on how that handles duplicate keys.

carloslbello commented 3 years ago

Thank you for the response!

I tried using the mapper=vdf.VDFDict parameter when loading the file and still get the same error.

rossengeorgiev commented 3 years ago

My bad, you have to also set merge_duplicate_keys=False. I've also pushed patch as that exception should not happen.

The problem section in the file (line 564618):

"12187"
{
    "name"      "Cavernite Dire Creeps"
    ....
    "visuals"
    {
       ....
        "asset_modifier"
        {
            "type"      "entity_clientside_model"
            "asset"     "npc_dota_creep_badguys_melee_upgraded_mega"
            "modifier"      "models/creeps/lane_creeps/creep_bad_melee/creep_bad_melee_cavern_mega.vmdl"
        }
        "asset_modifier"
        {
            "type"      "portrait_game"
            "asset"     "npc_dota_creep_badguys_melee"
            "modifier"
            {
                "PortraitLightPosition"     "29.379999 12.250000 162.360001"
                "PortraitLightAngles"       "73.760002 10.190000 0.000000"
                "PortraitLightFOV"      "75"
                "PortraitLightDistance"     "115"
...

Notice how asset_modifier and modifier are duplicate. This is not possible to respent in JSON. By default we the parser tries to merge them together, which creates a awful mashup, but potentially its good enough.

In [1]: import vdf, json

In [2]: _ = vdf.load(open('items_game.txt'))

In [3]: _['items_game']['items']['12187']['visuals']
Out[3]:
{'asset_modifier': {'type': 'portrait_game',
  'asset': 'npc_dota_creep_badguys_ranged_upgraded_mega',
  'modifier': {'PortraitLightPosition': '57.590000 -15.800000 176.649994',
   'PortraitLightAngles': '68.419998 172.600006 0.000000',
   'PortraitLightFOV': '90',
   'PortraitLightDistance': '99',
   'PortraitLightColor': '254 248 242',
   'PortraitShadowColor': '74 74 74',
   'PortraitShadowScale': '5',
   'PortraitAmbientColor': '79 108 108',
   'PortraitAmbientScale': '5',
   'PortraitSpecularColor': '251 74 84',
   'PortraitBackgroundTexture': 'materials/vgui/hud/heroportraits/portraitbackground_darkclouds.vmat',
   'PortraitBackgroundColor1': '0.700000 0.340000 0.200000',
   'PortraitBackgroundColor2': '0.700000 0.340000 0.200000',
   'PortraitBackgroundColor3': '0.700000 0.340000 0.200000',
   'PortraitBackgroundColor4': '0.700000 0.340000 0.200000',
   'PortraitLightScale': '4.500000',
   'PortraitGroundShadowScale': '1.500000',
   'PortraitAmbientDirection': '-79.070 -84.150 -25.320',
   'PortraitAnimationActivity': 'ACT_DOTA_IDLE',
   'cameras': {'default': {'PortraitPosition': '259.155487 112.627571 16.826363',
     'PortraitAngles': '346.670013 203.990051 0.000000',
     'PortraitFOV': '23',
     'PortraitFar': '1000'}},
   'PortraitSpecularDirection': '0.000000 0.000000 -1.000000',
   'PortraitSpecularPower': '16',
   'PortraitAnimationCycle': '0',
   'PortraitAnimationRate': '1',
   'PortraitHideHero': '0',
   'PortraitHideParticles': '0',
   'PortraitHideDropShadow': '0',
   'PortraitDesaturateParticles': '0',
   'PortraitDesaturateHero': '1'}}}

With VDFDict and merge_duplicate_keys=False you can preserve the order and duplication if you need them:

In [4]: _ = vdf.load(open('items_game.txt'), merge_duplicate_keys=False, mapper=vdf.VDFDict)

In [5]: print(json.dumps(_['items_game']['items']['12187']['visuals'], indent=2))
{
  "asset_modifier": {
    "type": "entity_clientside_model",
    "asset": "npc_dota_creep_badguys_melee",
    "modifier": "models/creeps/lane_creeps/creep_bad_melee/creep_bad_melee_cavern.vmdl"
  },
  "asset_modifier": {
    "type": "entity_clientside_model",
    "asset": "npc_dota_creep_badguys_melee_upgraded",
    "modifier": "models/creeps/lane_creeps/creep_bad_melee/creep_bad_melee_cavern.vmdl"
  },
  "asset_modifier": {
    "type": "entity_clientside_model",
    "asset": "npc_dota_creep_badguys_melee_upgraded_mega",
    "modifier": "models/creeps/lane_creeps/creep_bad_melee/creep_bad_melee_cavern_mega.vmdl"
  },
  "asset_modifier": {
    "type": "portrait_game",
    "asset": "npc_dota_creep_badguys_melee",
    "modifier": {
      "PortraitLightPosition": "29.379999 12.250000 162.360001",
      "PortraitLightAngles": "73.760002 10.190000 0.000000",
      "PortraitLightFOV": "75",
      "PortraitLightDistance": "115",
      "PortraitLightColor": "191 240 254",
      "PortraitShadowColor": "48 48 48",
...