Support for Wildfire maps

[ja2-derek]

I was bored with the vanilla maps, so I tried playing on Wildfire maps with Straciatella, and I got a working proof-of-concept. I probably is not the first one to try, but I wonder how much interest there is. I may be able to help make this into a JA2S mod.

I am able to get in-game with WF maps, though I haven't completed a play-through nor tested extensively. I am reporting my progress so far, hoping some of you would also be interested:

Using Wildfire maps

These are what I did so far:

• Export assets from Wildfire data directory, I am using the Steam version. Extract the below files and place in a mod directory
  • Data/BinaryData/JA2set.dat
  • Data/Interface/b_map.sti, convert to 8-bit pcx as b_map.pcx (See #712)
  • Data/Maps/**
  • Data/Radarmaps/**
  • Data/TileSets/**
• The maps should be loading fine at this point once the mod is enabled
• There are more town sectors in Wildfire maps. To enable those, edit:
  • InitializeStrategicMapSectorTownNames(void), add A1,B1,C14,F2 as town sectors
  • GARRISON_GROUP, add garrison groups to those sectors
  • g_traverse_we, g_traverse_ns and g_traverse_through, change those to town sectors

Known issues

• Some sectors crashes the Map Editor. For example I6 (Estoni). I haven't been able to figure out why.
• Wildfire has different items, so you will be getting wrong pre-placed items(https://github.com/ja2-stracciatella/ja2-stracciatella/issues/321#issuecomment-221561534)
• Some rooms and key locations moved, which means:
  • Some NPC AI movement scripts fail, e.g. Kyle in San Mona. The NPC scripts, and any hard-coded gridNo needs to be updated with WF ones.
• Some civilians outside of San Mona are in Kingpin's faction. For example some civilians may attack you in Drassen Airport if Kingpin is hostile.

What can be done

If there is interest, we can make it into a modpack and provide basic (and experimental) support of Vanilla-like experience on WF maps.

Regarding items differences, I don't know if we can ignore or somehow replace those WF items. I don't know how JA2 1.13 handle those issues.

For the strategic map differences, we can externalize Sector Info and Garrison Groups and make them mod-overridable.

Screenshots

[lynxlynxlynx]

Cool stuff, nice investigation!

We can script the data extraction with https://github.com/ja2-stracciatella/ja2-open-toolset and other python tools if needed.

If extended, it could also take care of the item swapping. Though I don't understand the problem, are their indexes different or why do vanilla ones show up?

Point three is the easiest.

[ja2-derek]

I think new items were added in WF. Some items, such as ammos that exist in Vanilla, are working. I am yet to check if WF is only adding new items, or if it also replaces existing ones.

There are indeed WF-only items, for example this one in Drassen SAM site. In Wildfire, it gives 2 "LAW rockets". This is the "ammo" for the re-usable and reload-able LAW only in WF.

In JA2S, it becomes "SMALL NAME". And the game glitches here sometimes. I have got a crash once, and another time seen the item appearing like a used LAW, which I suppose is some kind of default fallback.

[lynxlynxlynx]

Good, so it sounds like we'd just need to extend our externalized json tables and probably add some existence checks to the code.

[ja2-derek]

I have looked at the ITEMDESC.edt. It looks like WF added some new items, replaced some weapons (e.g. Rocket Rifle removed, Colt .45 replaced with UMP). Keys and other plot-significant items seem to be unchanged.

Wildfire on the left ; Vanilla on the Right Both files are the Steam versions (WF v6.04).

Do you know of any tools that can export .EDT as text or json, so I can produce a full diff?

[lynxlynxlynx]

Seven said: EdtMegaEditor

[ja2-derek]

I have been using EdtMegaEditor, but I didn't realized I can save as .TXT 🤦‍♂

Anyway, full diff below. WF has only used a few empty slots for new weapons. Some useless items (cigar, golf club) are replaced by weapon items. So if JA2S can ignore undefined items, it should not break gameplay too much. But of course, game balance will be a different issue.

Expand to see the diff between Vanilla("<") and Wildfire(">"))

```diff 2,7c2,7 < 1,Glock 17 < 2,Glock 18 < 3,Beretta 92F < 4,Beretta 93R < 5,.38 S&W < 6,Barracuda --- > 1,Calico M950 > 2,Micro Uzi > 3,USP > 4,Beretta 92F > 5,Colt 1991 > 6,Colt Python 9c9 < 8,Colt .45 --- > 8,UMP 12,15c12,15 < 11,Thompson < 12,Commando < 13,MP53 < 14,AKSU-74 --- > 11,Uzi > 12,M4 > 13,MP5 > 14,Colt Commando 17,20c17,20 < 16,Type-85 < 17,SKS < 18,Dragunov < 19,M24 --- > 16,HK 53 > 17,Mini-30 > 18,SVD > 19,M16 22c22 < 21,G41 --- > 21,HK 33 24,25c24,25 < 23,C-7 < 24,FA-MAS --- > 23,G36 > 24,MSG90 28,31c28,31 < 27,M-14 < 28,FN-FAL < 29,G3A3 < 30,G11 --- > 27,M14 > 28,FAL > 29,G3 > 30,AN-94 36,37c36,37 < 35,RPK-74 < 36,H&K 21 --- > 35,RPK > 36,HK 21 56,57c56,57 < 55,Rocket Rifle < 56,Automag --- > 55,SVU > 56,MP7 66,69c66,69 < 65,A. R. Rifle < 66,'small name' < 67,'small name' < 68,'small name' --- > 65,VAL > 66,VSS > 67,V-94 > 68,F2000 72,80c72,80 < 71,9mm mag < 72,9mm SMG mag < 73,9mm mag AP < 74,9mm SMG mag AP < 75,9mm mag HP < 76,9mm SMG mag HP < 77,.38 S-load < 78,.38 S-load AP < 79,.38 S-load HP --- > 71,9mm mag AP > 72,9mm SMG mag AP > 73,9mm mag AP 50 > 74,9mm mag HP > 75,9mm SMG mag HP > 76,9mm mag HP 50 > 77,4.6mm mag > 78,7.62*54mm 10 AP > 79,7.62*54mm 10 HP 87,106c87,106 < 86,.357 S-load < 87,.357 mag < 88,.357 S-load AP < 89,.357 mag AP < 90,.357 S-load HP < 91,.357 mag HP < 92,5.45mm mag < 93,5.45mm mag HP < 94,5.56mm mag < 95,5.56mm mag HP < 96,7.62mm WP 10 < 97,7.62mm WP 30 < 98,7.62mm WP 10 < 99,7.62mm WP 30 < 100,7.62mm NATO 5 < 101,7.62mm NATO 20 < 102,7.62mm NATO 5 < 103,7.62mm NATO 20 < 104,4.7mm mag < 105,5.7mm mag --- > 86,.357 S-load AP > 87,.357 mag AP > 88,.357 S-load HP > 89,.357 mag HP > 90,5.45mm mag > 91,5.45mm mag HP > 92,5.56mm mag > 93,5.56mm 100 AP > 94,5.56mm mag HP > 95,5.56mm 100 HP > 96,7.62mm WP 30 AP > 97,7.62mm WP 75 AP > 98,7.62mm WP 30 HP > 99,7.62mm WP 75 HP > 100,7.62mm NATO 20 > 101,7.62mm NATO 100 > 102,7.62mm NATO 20 > 103,7.62mm NATO 100 > 104,9*39mm 10 > 105,9*39mm 20 112,114c112,114 < 111,Minirockets, AP < 112,Minirockets, HE < 113,Minirockets, HEAP --- > 111,12.7mm, AP > 112,12.7mm, HE > 113,12.7mm, HEAP 161c161 < 160,'small name' --- > 160,LAW rocket 203c203 < 202,Med Kit --- > 202,Hospital 208c208 < 207,Silencer --- > 207,Silencer 9mm 248c248 < 247,Discarded LAW --- > 247,Empty LAW 266c266 < 265,Golf Clubs --- > 265,Silencer .45 270c270 < 269,Cigars --- > 269,Silencer P90 306,308c306,308 < 305,Rod < 306,Spring < 307,Rod & Spring --- > 305,Silencer MP7 > 306,Silencer Commando > 307,Collimator ```

[ja2-derek]

Implemented the installation script. You can try Wildfire Maps using this:

https://github.com/ja2-derek/ja2-stracciatella/tree/wildfire-maps-mod/assets/mods/wildfire-maps

[lynxlynxlynx]

Wow, nice job! If you're interested in finishing this, I suggest we keep this bug as the meta overview one and you open new subreports as needed.

The city sector externalizatoin is the easy part. Weapons we already have the same way (eg. assets/externalized/weapons.json), so it's mostly a matter of checking if any new hardcoded needs are there or changed, then adapting the externalized files (I guess also the merchant inventories). If it's not the case yet, ensuring mod data overrides these files as well, since it's likely we can't ship it by default (ok, it could be a separate task to generalize this out).

Also, if you want, we can move the repo under this github organisation.

[ja2-derek]

There are indeed quite a lot more to work on, to get this fully working. I am going to use this comment to track the work breakdown as you said. I will come back and update this as I go deeper.

Not sure how much further I can go. I don't think I can complete everything all alone. But the externalization should benefit the project anyways.

Data files exports from Wildfire

• [x] Data/BinaryData/JA2set.dat
• [x] Data/Interface/b_map.sti, convert as b_map.pcx
• [x] Data/Interface/MilitiaMaps.sti
• [x] Data/Maps/**
• [x] Data/Npcdata/
  • [x] 093.npc (SPIKE, changed gridNo)
  • [x] 095.npc (KYLE, patched Vanilla version, changed pathing)
  • [x] 117.npc (HANS, changed gridNo)
  • [x] 069.npc (VINCE, Cambria hospital) (https://github.com/ja2-derek/mod-wildfire-maps/commit/357e5ca7ac7bfa8f3a3c552eb3538af2459bdaeb)
  • [x] 080.npc (STEVE, Cambria hospital)
  • [x] 132.npc (JENNY, Cambria hospital)
• [x] Data/Radarmaps/**
• [x] Data/TileSets/**
  • [x] Excluding item sprites (smguns.sti, SMITEMS.STI, smp1items.sti, smp2items.sti, SMP3ITEMS.STI)

Data in codebase to be externalized

With the suggested name, location in codebase and updates for WF maps (see also #665):

• [x] army-garrison-groups.json army-patrol-group.json army-compositions.json - StrategicAI.cc: gOrigGarrisonGroup[]; Add garrison groups to new town and roadblock sectors (#1025)
• [x] strategic-map-towns.json - StrategicMap.cc: InitializeStrategicMapSectorTownNames(void); Map_Screen_Interface_Map.cc: sBaseSectorList[]; A1,B1,C14,F2 are now town sectors (#1024)
• [x] strategic-map-movement-costs.json - Strategic_Movement_Costs.cc: g_traverse_we, g_traverse_ns and g_traverse_through; A1,B1,C14,F2 are now town sectors (#1021)
• [x] strategic-bloodcat-placements.json - Game_Init.cc: InitBloodCatSectors, missing placements at C3, D4, G6, G12, K6 ,K10. (#1018)
• [x] strategic-map-npc-placements.json - Strategic_Event_Handler::HandleEarlyMorningEvents; Missing vehicle placement (Hamous and his truck) at D3 and G6.dat (#1029)
• [x] strategic-map-sam-sites.json - pSamGridNoAList[], pSamGridNoBList[], update the SAM controller computer gridNo; pSamList[] not changed in WF (#1059)
• [x] strategic-map-underground-sectors.json: Campaign_Init.cc: BuildUndergroundSectorInfoList. New UG sectors in Wildfire: f8_b1.dat, h13_b1.dat, i13_b1.dat, m2_b1.dat, m3_b1.dat, m4_b1.dat, n4_b1.dat, n5_b1.dat, o4_b1.dat (#1074)
• [x] tactical-npc-action-params.json strategic-fact-params.json - Interface_Dialogue.cc: HandleNPCDoAction(); Quests.cc: CheckFact(), CheckNPCAt() (#1061)

Non-blocker issues

Noticable but do not cause game to malfunction, crash or stuck. Nice to get fixed but not show-stopper

• [x] shipping-destinations.json: Arriving items lie in weird locations: BOBBYR_SHIPPING_DEST_GRIDNO (#1077)
• [x] Weapons and items on the Tactical Map look like the WF items, which is wrong. For example, a Colt .45 lying on the ground looks like an UMP. (Handled in install script)
• [x] AI does not seem to trigger Panic Buttons, e.g. at Tixa and Alma (H13) -- #1080 for J9; threshold is 70% enemies killed in Wildfire H13
• [x] strategic-map-creature-lairs.json: Creature insertion (town attack) and warp insertion (Travel to surface prompt) have the wrong gridNo. Refer to GetWarpOutOfMineCode and gsCreatureInsertionGridNo in Creature_Spreading.cc
• [ ] tactical-gridnos.json: List of various special gridNos, identified by a combination of sectorID and a string name
  • [ ] captureItems at I13 and N7: Captured/Interrogated soldiers' item are placed on an open tile. It is supposed to be crate/locker grids. (sAlmaCaptureItemsGridNo => [8552,8552,8552] and sInterrogationItemGridNo => 13506)
  • [ ] staircase at P3: Queen goes down to basement at the incorrect gridNo. STAIRCASE_GRIDNO should be changed to 11727.

Other issues to be addressed

• [x] BuildListOfTownSectors causes buffer overrun when it goes past the last element (#1016)
• [x] Handling of pre-placed Wildfire-only items (#1031, #1042)
• [x] I6 crashes the editor and game. Soldier placed at far bottom left corner (e.g. 24719) seems to cause a crash in FindBestPath (#1064)
• [x] Had a load game crash at Tactical/Keys.cc: synchronizeDoorStatusToStructureData(). base was null at the last line. Worked around with a null check. Need to check why. The sector was G8. (#1065)
• [x] Dead loop loading G9 in game. Loading is stuck at Buildings.cc:GenerateBuilding. It is stuck at the while(1) loop, with sCurrGridNo=25487, sRightGridNo=25486,bDirection=4,sStartGridNo=20056 (#1063)

Other gridNo references in code

To be investigated one by one. Ignore if not changed in Wildfire maps

• [x] ~Boxing.cc: gsBoxerGridNo~same gridNos in WF
• [x] ~End_Game.cc: DoesO3SectorStatueExistHere()~ - same gridNos in WF
• [x] ~GameScreen.cc: Near IN_DEIDRANNA_ENDGAME~ Tested, no problems with end game scene
• [x] ~Handle_Doors.cc: (Brothel doors in D4)~ - The pSoldier->gridNo checks in ProcessImplicationsOfPCMessingWithDoor does not seem to do anything. There is a room check already
• [x] ~Interface_Dialogue.cc: sBasementEnterGridNos[] , sBasementExitGridNos[], NPC_ACTION_GOTO_HIDEOUT(ExitGrid.usGridNo)~ No changes needed for Rebels' Hideout
• [x] ~Interface_Dialogue.cc: NPCOpenThing() for ARMAND~ same gridNo in WF
• [x] ~Overhead.cc: HandleJohnArrival~ Works fine on WF map
• [x] ~PanicButtons.cc:~ WARDEN_ALARM_GRIDNO, WARDEN_GAS_GRIDNO both unused.
• [x] Queen_Command.cc: sAlmaCaptureGridNos, sAlmaCaptureItemsGridNo, sInterrogationItemGridNo - AlmaCaptureGridNos are good; AlmaCaptureItems grid is an open tile right at a door; Queen Interrogation works, but items are placed in the open outside of the facility (sInterrogationItemGridNo)
• [x] ~Strategic_Movement.cc: AddCorpsesToBloodcatLair()~ cosmetics only
• [x] ~Map_Screen_Interface.cc: START_SECTOR_LEAVE_EQUIP_GRIDNO, 10433.~ It is some random open tile on map. Does not really matter.

Out of Scope

Some Wildfire features or differences here. Some may be interesting to have, but should not be in scope for a map mod. They could be a separate WF gameplay mod in the future.

• [ ] Externalizing enemy soldier creation. Wildfire enemy regular army soldiers are in blue.
• [ ] Wildfire storyline: Email, NPC scripts and dialogues. There is no Enrico Chivaldori, for example. Some or most should work, but still need to check them one by one
• [ ] Tactical AI. Wildfire has much tougher enemies. Probably impossible without seeing the WF source code.
• [ ] In Wildfire, civilians are unhappy (lowers loyalty) or become hostiles if you rob their stuff.
• [ ] No placement of Sergeant Krott. There might be yet some more unplaced / disabled NPCs

[lynxlynxlynx]

I don't have WF, so can only pontificate. What don't I understand about 095.npc? Doesn't just replacing the file work?

[ja2-derek]

I don't have WF, so can only pontificate. What don't I understand about 095.npc? Doesn't just replacing the file work?

~Oh yes. How come I didn't think of that? Updated.~

EDIT: Wildfire Kyle has slightly different behavior. He does not go to the back room, but gives the money right away. No NPC_ACTION_KYLE_GETS_MONEY in Wildfire, I suppose. It should be easier to patch GridNo in the NPC script, than altering the NPC interaction code.

(Ref: https://youtu.be/ZocmwO8vXVg?t=4443)

[ja2-derek]

OK, the packaging is done for the installer. https://github.com/ja2-derek/mod-wildfire-maps/releases/tag/0.1.0

[lynxlynxlynx]

Nice. Did you consider the installer taking an optional path to WF, so it could copy slfs itself?

[ja2-derek]

Installer updated: https://github.com/ja2-derek/mod-wildfire-maps/releases/tag/0.2.0 (README.txt)

• Installer now accepts --src_dir command arg, or launches a GUI prompt if display is available
• Installer outputs are now highlighted with colors
• Excludes item sprites from Tilesets, so items will look correct in Tactical
• Includes all the externalized JSON assets so far. Already customized for WF maps, but still need further data verification

[lynxlynxlynx]

Cool. Looks like just a bit more is needed to obviate the need for point 3 in the README.

[lynxlynxlynx]

What's with the other npc files? I thought that just preserving the version from Wildfire is enough and we don't need to care.

[ja2-derek]

I am hand-picking which NPC files to use. It is because some NPCs have dependencies on WF version of quotes, speeches or code logic. I want to keep this mod to only maps if possible.

So by default I am using the vanilla NPC. If there are path changes, use the Wildfire version. There is one exception case KYLE, which I am patching the vanilla file to work with the WF map

[lynxlynxlynx]

So I guess that's blocked by the interpreter discussion, ok. Can you then verify the other grid todos before jumping to UB?

[ja2-derek]

Yes I am still on it. I am going to finish a playthrough with WF maps, and verify each and every one of the Grid TODOs.

If there are no blocking issues with GridNos, I will update the installer. Then it should be ready for release targeting 0.17.

[ja2-derek]

Current status

• No outstanding blocker issues
• Installer script (mod-wildfire-maps) updated, ready for 0.17 release
• Completed played through, covered all significant parts that I can think of.
• A couple of open non-blocker issues:
  • Creature lair externalization is half done and not tested. I don't have time to finish that just now
  • I don't really want to do the "tactical gridNo" one. I'd rather wait for scripted / externalized quest logic

[ja2-derek]

I'm going to close this issue, as all of the necessary changes are merge and good for 0.17 release.

Creature Lair externalization will be handled with #1087.

Tactical gridNo stuff will be deferred

[ja2-derek]

Latest version of installer script: https://github.com/ja2-derek/mod-wildfire-maps/releases/tag/0.2.1

Can that repo be moved into the official ja2-stracciatella org?

[lynxlynxlynx]

Re issue: yeah, please open a new one and migrate what's left to do for a full WF experience.

Repo: yes, please. Do you have the option already?

I would like to invite you to become part of the github team — it's well deserved and might be needed for the migration. @flaviojs any objections?

[flaviojs]

No objections. 👍

[lynxlynxlynx]

Ok, done. Welcome aboard! :fireworks:

In case it wasn't obvious @ja2-derek, we all work on the main repo indirectly through PRs (master branch is marked as protected).

I'll write something for the website features page as soon as you move the repo, so the links won't break.

[ja2-derek]

Thank you!

Moved repo: https://github.com/ja2-stracciatella/mod-wildfire-maps

[lynxlynxlynx]

Done on my side as well. So then please just open the new issue and this one is done done.

https://ja2-stracciatella.github.io/features/#bundled-optional-mini-mods