Feature/changelog tags & versioning prep

[Surxe]

Main changes

• Changelog output now supports tagging lines with relevant tags. That line will appear on each wiki page corresponding to each tag. For example, "Abrams' Siphon Life now deals 5 more damage", would automatically have tags Heroes, Abrams, Siphon Life, and Abilities. This would cause the line to appear on wiki pages: /Heroes#Update history, /Abrams#Update History, /Siphon Life#Update history, and /Abilities#Update history. We can add support for more tags from their headers in heading_to_tag, or parse a specific tag from relevant data such as weapon items being assigned "Weapon Items" tag, or add them manually, lua modules and deadbot uploader is planned to do the rest.
• Changelogs now saved in the repository by their Valve-assigned build # (from the patch note url) instead of the date.
• The date and link for each valve-assigned build are also now parsed to output-data/changelogs/changelogs.json. This in preparation for a future PR that will focus on saving changelogs in deadbot by their Steam assigned manifest id. See the #Get old version numbers thread in the community discord on this process thus far.

Other changes

• hyphen prefixes replaced with asterisks for html bullet point integration
• ": " appearing in a line is no longer removed, as sometimes it removes needed context & its safer to leave each line as close to the original as possible

The {{PageRef}} has an initial version but is still very subject to change as it is planned to prioritize svg icons that are both light and dark theme compatible, which is waiting on dark theme release.

I added complete documentation on how I expect the Changelog editing process (after uploader) to go at [[Module:Changelog/doc]], as well as parsing for the list of Changelog Date pages displayed on [[Changelog Dates]], and some initial (front-facing) preparations at [[Changelog Tags]]

[Surxe]

Ready for review. There are 300+ files in the filediff because of the storage change. Instead, to see the changes, run main.sh with Changelog on, then view output-data/changelogs directory, fetch_changelogs. and parse_changelogs

[Surxe]

Really great work! How we plan on deploying this? As we will have to make sure any existing contributions aren't just paved over (assuming they aren't handled by the parsed logs)

Great question! Essentially, I am just going to need to spend a weekend merging the parsed data from here and the special formats that are done from the existing update pages. It will mostly be stuff like Shadow Transformation being nicknamed to Shadow Explosion as Valve renamed it. I have the main cases that require human-intervention documented on [[Module:Changelogs]] too!

[hassayag]

Closed so i can reopen and trigger the deadlock-data PR 🤞

[hassayag]

Okay this is the one

[Surxe]

Summary of changes since last review:

• Added tons more tags, frontend-friendly list visible at [[Template:Changelogs Navbox]] (formatting is placeholder)
• Extracted lots of code within parse_changelogs.run() to separate functions for readability sake. Elaborated on their uses in depth in comments.
• Remap-source is now used as alt_name parameter for {{PageRef}}, allowing for more embedments and removing the need to adjust for case sensitivity/plural. Example: "Fixed Rejuv buff" -> "Fixed {{PageRef|Rejuvenator|alt_name=Rejuv}} buff"
• Added self.tag_parents for assigning a tag's many child <-- parent relationship's in a more streamlined fashion. Detecting these relationships for say, resources such as Siphon Life <-- Abrams, still requires parsing however. Though, the relationships between group tags is verbosely defined with this map.
• Moved all maps/lists for tags to the top of the parser for easier reference, in order of their application. I thought of moving the maps to maps.py, but then the maps would be separated from the lists, whereas currently its very convenient.
• Lines that are headings are now parsed adequately as having no tags, making them appear only on an update page i.e. [[Update:05-03-2024]] (fake example), and not a tag page's update section.
• Added hero lab's changelog in a similar format to ./input-data/raw-changelogs/herolab_2024_10_29.txt and /output-data/changelogs/changelogs.json manually. Added some comments to fetch_changelogs for this process, but we need a more streamlined way to communicate this requirement documentation wise. Any ideas?

[Surxe]

Today's changes

• Tag renames
• • Base Defenses -> Objective, removing Trooper from the group
• • Soul -> Souls for matching translation
• • Orb -> Soul Orb for more verboseness since there is no official translation
• Fixed edge case for nested PageRef calls
• All plural tags are now singular tags for consistency (except Souls)

[Surxe]

@hassayag it won’t let me reply to your comment at https://github.com/deadlock-wiki/deadbot/pull/84#discussion_r1825075373 so I’ll reply here

The optimal (but would never be agreed upon) route is to have ability pages at /Ability/<abilityname>, hero pages at /Hero/<heroname>, etc. Instead, when we spot dupes during the uploader script, we will have it create the pages as /<entityname> (Ability) and /<entityname> (Item). I'm actually leaning towards no parentheses simply for better localization support, so /<entityname> Ability

It would also create an ambiguation page at /<entityname>

[Surxe]

Summary of changes since last review:

1. More tags added

   • Base Guardian
   • Sapphire Hand / Amber Hand
   • Cosmic Veil
   • Crate
   • Powerup
   • Breakable (now parent to many previously added tags too)
   • Sandbox
   • Shop
   • Minimap (merged into Map tag)
   • Parry
   • Light Melee, Heavy Melee, Melee
   • Idol (merged into Urn)
   • Pause
   • Sinner's Sacrifice (Vault)
   • Urn tag renamed to Soul Urn
   • Flex Slot
   • Hero lab related changes now use tag HeroLab <heroname>

2. Hero lab entries now added manually via input-data rather than output-data. Created a readme for this process as well, as it will be manual unless Valve uploads hero lab notes to the forums, or some image recognition is applied to read the ones from in game. Cleaned up some old hero lab specific files that were meant to be removed earlier.

3. Tag child/parent hierarchy scripted into dictionary format at tag_tree.json, uploaded to [[Data:Changelog Tag Tree.json]], and used by [[Module:Changelog]] for use on [[Template:Changelogs Navbox]] (though admittedly its not actually a navbox yet, maybe som1 will have ideas for it down the line)

[Surxe]

Changes since last review:

• self.parents being defined then transformed into self.tag_tree is removed. Now, self.tag_tree is defined, then transformed into self.parent_lookup (structure identical to the old self.parents).
• Since this format now more directly corresponds to the outputted tag_tree.json dict (as opposed to the old tag_groups.json list), group tags are no longer needed. As such, everything related to group tags they were completely removed, greatly simplifying the comments needed to explain several processes. The only downside is that now instead of specifying is_group_tag parameter when its parsed, you have to add it to self.tag_tree even if it has no children tags. Though I think this was worth the complexity reduction.
• Removed the need for HeroLab related tags to be specified in self.tag_tree manually
• Added indicators to tag_tree.json where instance tags would appear (like Abrams, Basic Magazine, etc.), via <Hero Name>, <Ability Name>, etc.
• Fixed tags that were accidentally in the tag_tree in two different locations. They now appear in only the furthest depth.
• self.tag_tree and therefore tag_tree.json now use a custom order that isn't restricted to alphabetical like before.
• Today's HeroLabs patch added to input-data

Specific tag related changes:

• Guardian, Base Guardian, and Walker tags are now only children to Objective, and not NPC. This is not only because they are arguably more robot than an NPC, but also to reinforce the ideal state of tags only appearing in the tag_tree once.
• Parry tag is now a child to the Melee tag
• Gun tag added, and as a child to the Hero tag
• Lane names added as tags (i.e. Greenwich)
• Many map locations added as tags as they appear on signs (i.e. Bridge, Hotel) as children to the Map tag

[Surxe]

Changes since last review:

• Improved the changelog readme

• _write_tag_tree moved to tags.py

• Removed the duplicate Denizen tag

• Sapphire Hand and Amber Hand now children to Team tag

• Gun tag renamed to Weapon

• Skip problematic disabled items rather than all disabled items

• Check for resources in order of the length of their name. I.e. if the text is Smoke Bomb, the tag assigned will now reliably be Smoke Bomb instead of Smoke (a disabled item)

• Lane name tags moved under Map tag