Potential improvments to repository

[VATICAN-PSYCHO]

Hey, i found this tool very helpful and it's amazing. However, I'm not Windows user and this does not work out of the box. The thing that need changing is abandoning backslashes (\) in favor of slashes (/). Since this (slashes) also works in Windows, it would be nice.

Another things are mods and textures. Every of them follow some kind schema (that is not os-agnostic).

For example textures list from map_data.yaml:

<texture-entry-name>:
    texture_root: mod_root
    texture_path: |-
        <steam-id</mods/<mod-id>/media/texturepacks
    texture_file_patterns: ['.*[.]pack']

This could be replaced with:

<texture-entry-name>:
    steam_id: <steam-id>
    mod_id: <mod_id>

Why? Because every texture so far follow same schema. Field texture_file_patterns is same everywhere, so it be either removed (and hardcoded) or set as default to (better option). Same goes for texture_root. texture_path can be build from steam_id and mod_id. Same could be done for maps.

Better option would be separating mod maps (and textures) from config at all. Since map can (but not must) contain own textures, so they can be easily detected (simple by looking for texturepacks directory inside media. Dependencies should be defined inside mod.info file so there is no need for explicit defining them (it up to the mod publisher to provide working mod, not you as code maintainer). That way every map mod can be submitted to this repository.

[ethano8225]

You had the example:

For example textures list from map_data.yaml: < texture-entry-name >: steam_id: < steam-id > # had to add spaces bc formatting problems on git mod_id: < mod_id >

Sure every mod is in mod_root, currently. But if you're developing a mod that isn't on the workshop, and you wanted to add that mod to the map to check if it's compatible / working properly, you'd have to make a fake modID to throw into your workshop folder in order to load it (with how you set it up).

So instead, maybe texture_root can be set to mod_root by default, only changed if the user provides a specific path (and actually, texture_file_patterns could be pre-set too) ie

<texture-entry-name>:
    #texture_root: mod_root                         #implicit unless texture_root is intentionally set by user
    texture_path: |-
        <steam-id</mods/<mod-id>/media/texturepacks 
    #texture_file_patterns: ['.*[.]pack']           #implicit

That code would be simplified to this:

<texture-entry-name>:
    texture_path: |-
        <steam-id</mods/<mod-id>/media/texturepacks 

This would, in my opinion, be simpler than providing both the mod_id and steam_id seperately, and would prevent errors in the case that the texturepack files are not in /media/texturepacks.

And, to quickly change this to work with MacOS as well, you can use find and replace to find "\" and replace with "/". Many if not all file editors support this (i think even notepad)

[cff29546]

Here are some thoughts after reading the comments:

1. replace backslash: I see no problem. (no code change needed)
2. default texture_file_patterns: I see no problem. (no code change needed)
   • current code already defaults to ['.*[.]pack'].
3. use steam-id, mod-id, and mod-name and follow the path template: redesign config parsing logic
   • use a default template, for example '{steam-id}/mods/{mod-id}/media/maps/{map-name}'
   • user can overwrite the default template or exact path by manual config
4. config files structure: redesign for easy mod map description
   • Spread configuration into multiple files
   • one configuration file describes a few maps and a few textures
   • unify the texture and map description: allow map or texture or both described in the same entry

A configuration section example:

<entry_name>:
    steam_id: 1234567
    mod_id: 12345
    map_name: Name    # has map named `Name`, default is '' meaning not a map
    texture: true     # has textures, default is false meaning no private texture

    #values with default
    #
    #texture_path_pattern: '{mod_root}/{steam_id}/mods/{mod_id}/media/texturepacks'
    #texture_path: ''   # force override texture_path if needed. example: 'test/texturepacks'
    #texture_file_patterns: ['.*[.]pack']
    #texture_files: []   # select texture packs and ignore texture_file_patterns. example: ['a.pack', 'b.pack']
    #map_path_pattern: '{mod_root}/{steam_id}/mods/{mod_id}/maps/{map_name}'
    #map_path: '' # force override map_path if needed
    #    example: 'media/maps/Muldraugh, KY'
    #encoding: utf8
    #depend_textures: ['default']    # Auto include private texture if exist
    ...

[VATICAN-PSYCHO]

Thank you for your response. I'm making similar project to yours, but focuses on performance (and self-learning). There is a piece of code that handles dependencies, based on mod.info. You could implement this feature here.

[cff29546]

Thank you for your response. I'm making similar project to yours, but focuses on performance (and self-learning). There is a piece of code that handles dependencies, based on mod.info. You could implement this feature here.

Not all mod have a well written mod.info. My solution is to retrieve the Steam workshop page and extract the dependencies using regex matching.

get_mod_dep.py:

import requests
import re

URL_TEMPLATE = 'https://steamcommunity.com/sharedfiles/filedetails/?id={mod_id}'
TITLE_PATTERN = re.compile('<div class="workshopItemTitle">([^<>]*)</div>')
DEP_PATTERN = re.compile('<a href="([^"]*)"[^>]*>\\s*<div class="requiredItem">\\s*(.*?)\\s*</div>\\s*</a>', re.MULTILINE)

def get_info(mod_id):
    url = URL_TEMPLATE.format(mod_id=mod_id)
    rsp = requests.get(url)

    title = None
    match = TITLE_PATTERN.search(rsp.text)
    if match:
        title = match.group(1)

    depend = []
    for match in DEP_PATTERN.finditer(rsp.text):
        if match:
            dep_id = match.group(1).split('=')[-1]
            dep_name = match.group(2)
            depend.append((dep_id, dep_name))

    return title, depend

if __name__ == '__main__':
    import argparse
    parser = argparse.ArgumentParser(description='mod dep getter')
    parser.add_argument('ids', nargs=argparse.REMAINDER)
    args = parser.parse_args()

    for steam_id in args.ids:
        title, dep = get_info(steam_id)
        print(title, dep)

An output example:

>get_mod_dep.py 2820363371 2915656059
Ashenwood [('2337452747', 'Daddy Dirkie dirks tiles'), ('2599752664', "Dylan's Tile Pack"), ('2384329562', "throttlekitty's tiles")]
Cedar Hill [('2844685624', "TryHonesty's Tiles"), ('2774834715', "ExtraNoise's Newburbs Tiles"), ('2337452747', 'Daddy Dirkie dirks tiles')]

[VATICAN-PSYCHO]

I thought the same about this issue, but if for example, said mod doesn't have defined dependencies properly in mod.info, then this will not auto load them. You can test this by by loading (in-game) map that requires some tiles and vice versa try to unload tile mod.

You can also skip dependency checking and load all mods that are map (they have maps subdir), textures (texturepacks) and mixed (both maps and texturepacks directory present).

[ethano8225]

Quick update: I added support to include some depend_textures when running collect_mod_map_data, although some steamID's will still throw an error (everything in PZ map mod works). here is the collect_map_mod_data.py and get_mod_dep.py (the code I added is inbetween ###'s)

I optimized it so it can save all dependencies to \pzmap2dzi-main\scripts\dependsave\ (you must make the folder) for re-runs, takes compute time down from ~40 seconds to 0.05s (once the dependencies have been saved). add the txt file \scripts\dependsave\refreshDepends.txt with either a 1 or 0 in it (nothing else) 1 indicates it will refresh the dependencies, 0 means it will use already-written files. I added this to my fork of pzmap, if this is unclear just cd ~/pzmap2dzi-main to run the code (once the files have been created, refreshDepends gets set to 0, so set it to 1 if you want to re-refresh) optimized collect_mod_map_data.py

additional question: can map names that are placed under "maps:" be called for depend_textures? For example, Otr (over the river) has a texturepack file, but it isn't under texturepacks, can it be called by another map under depend_textures so that map can use Otr's textures as well?

edit: Made a script that automatically gets "fake" dependencies (no texturepacks files), am planning to incorporate that into collect_mod_map_data.py so it can ignore all of these fake dependencies automatically when creating map_mod.yaml, rather than having the list of fake dependencies being hard-coded into the file the list of fake dependencies gets exported to pzmap2dzi-main\scripts\dependsave\fakeDependencies.txt

[cff29546]

I have updated the configuration format recently. Here's an example:

LittleTownship:
  map_name: LittleTownship
  mod_name: LittleTownship
  steam_id: '2542249811'

All default values are defined in default.txt. I have also improved the script for collecting descriptions, which can now reuse collected dependencies. For more details, please refer to Map description generation.