search

Mutagen-Modding / Mutagen

A .NET library for analyzing, creating, and manipulating Bethesda mods
GNU General Public License v3.0
119 stars 31 forks source link

feat: add empty definitions for almost all missing major records #481

Closed FlayaN closed 1 year ago

FlayaN commented 1 year ago

UPDATE: updated the PR with a check against Starfield.esm. Still 2 separate commits. Now it's only these left:

AOPF -> exists, just need to update name MRPH -> exists, just need to update name

PGRE -> better to do manually due to in previous mutagen games this is grouped in trap etc PHZD -> better to do manually due to in previous game this is grouped in trap etc

SMBN -> better to do manually due to in previous game this is grouped in StoryManagerNodes.xml etc SMQN-> better to do manually due to in previous game this is grouped in StoryManagerNodes.xml etc SMEN-> better to do manually due to in previous game this is grouped in StoryManagerNodes.xml etc

Hello! I did a python script to generate empty definitions for basically all major records (only added, discarded if a definition already exists)

One nice part about this is to minimize conflicts in the future if multiple people create new records (due to one of the ids being incremental)

You can decide if this is needed or not! No bad feelings, only some effort to fill in the script with the record and their names

Using this template: (named TEMPLATE.xml)

<?xml version="1.0" encoding="us-ascii"?>
<Loqui xmlns="http://tempuri.org/LoquiSource.xsd">
  <Object name="[NAME]" recordType="[RECORD_TYPE]" objType="Record" baseClass="StarfieldMajorRecord">
    <Fields>
    </Fields>
  </Object>
</Loqui>

I ran this python script

records = {
    #"TES4": "Main File Header",      #// 01
    #"GRUP": "",      #// 02
    "GMST": "GameSetting",      #// 03
    "KYWD": "Keyword",      #// 04 BGSKeyword
    "FFKW": "Form Folder Keyword List",      #// 05 BGSFormFolderKeywordList
    "LCRT": "LocationReferenceType",      #// 06 BGSLocationRefType
    "AACT": "ActionRecord",      #// 07 BGSAction
    "TRNS": "Transform",      #// 08 BGSTransform
    "TXST": "TextureSet",      #// 09 BGSTextureSet
    "GLOB": "Global",      #// 0A TESGlobal
    "DMGT": "DamageType",      #// 0B BGSDamageType
    "CLAS": "Class",      #// 0C TESClass
    "FACT": "Faction",      #// 0D TESFaction
    "AFFE": "AffinityEvent",      #// 0E BGSAffinityEvent
    "HDPT": "HeadPart",      #// 0F BGSHeadPart
    "EYES": "Eyes",      #// 10 TESEyes
    "RACE": "Race",      #// 11 TESRace
    "SOUN": "SoundMarker",      #// 12 TESSound
    "SECH": "Sound Echo Marker",      #// 13 BGSSoundEcho
    "ASPC": "AcousticSpace",      #// 14 BGSAcousticSpace
    ###################"AOPF": "Audio Occlusion Primitive",      #// 15 BGSAudioOcclusionPrimitive
    "SKIL": "SkillRecord",      #// 16
    "MGEF": "MagicEffect",      #// 17 EffectSetting
    "SCPT": "Script",      #// 18 Script
    "LTEX": "LandscapeTexture",      #// 19 TESLandTexture
    "PDCL": "Projected Decal",      #// 1A BGSProjectedDecal
    "ENCH": "ObjectEffect",      #// 1B EnchantmentItem
    "SPEL": "Spell",      #// 1C SpellItem
    "SCRL": "Scroll",      #// 1D ScrollItem
    "ACTI": "Activator",      #// 1E TESObjectACTI
    "TACT": "TalkingActivator",      #// 1F BGSTalkingActivator
    "CURV": "Curve Table",      #// 20 BGSCurveForm
    "CUR3": "Curve 3D",      #// 21 BGSCurve3DForm
    "ARMO": "Armor",      #// 22 TESObjectARMO
    "BOOK": "Book",      #// 23 TESObjectBOOK
    "CONT": "Container",      #// 24 TESObjectCONT
    "DOOR": "Door",      #// 25 TESObjectDOOR
    "INGR": "Ingredient",      #// 26 IngredientItem
    "LIGH": "Light",      #// 27 TESObjectLIGH
    "MISC": "MiscItem",      #// 28 TESObjectMISC
    "STAT": "Static",      #// 29 TESObjectSTAT
    "SCOL": "StaticCollection",      #// 2A BGSStaticCollection
    "PKIN": "PackIn",      #// 2B BGSPackIn
    "MSTT": "MoveableStatic",      #// 2C BGSMovableStatic
    "GRAS": "Grass",      #// 2D TESGrass
    "FLOR": "Flora",      #// 2E TESFlora
    "FURN": "Furniture",      #// 2F TESFurniture
    "WEAP": "Weapon",      #// 30 TESObjectWEAP
    "AMMO": "Ammunition",      #// 31 TESAmmo
    "NPC_": "Npc",      #// 32 TESNPC
    "LVLN": "LeveledNpc",      #// 33 TESLevCharacter
    "LVLP": "Leveled Pack In",      #// 34 BGSLevPackIn
    "KEYM": "Key",      #// 35 TESKey
    "ALCH": "Ingestible",      #// 36 AlchemyItem
    "IDLM": "IdleMarker",      #// 37 BGSIdleMarker
    "BMMO": "Biome Marker",      #// 38 BGSBiomeMarkerObject
    "NOTE": "Note",      #// 39 BGSNote        # HOLOTAPE IN FO4 mutagen?
    "PROJ": "Projectile",      #// 3A BGSProjectile
    "HAZD": "Hazard",      #// 3B BGSHazard
    "BNDS": "BendableSpline",      #// 3C BGSBendableSpline
    "SLGM": "SoulGem",      #// 3D TESSoulGem
    "TERM": "Terminal",      #// 3E BGSTerminal
    "LVLI": "Leveled Item",      #// 3F TESLevItem
    "GBFT": "Generic Base Form Template",      #// 40 BGSGenericBaseFormTemplate
    "GBFM": "Generic Base Form",      #// 41 BGSGenericBaseForm
    "LVLB": "Leveled Base Form",      #// 42 BGSLevGenericBaseForm
    "WTHR": "Weather",      #// 43 TESWeather
    "WTHS": "Weather Settings",      #// 44 BGSWeatherSettingsForm
    "CLMT": "Climate",      #// 45 TESClimate
    "SPGD": "Shader Particle Geometry",      #// 46 BGSShaderParticleGeometryData
    "REGN": "Region",      #// 47 TESRegion
    "NAVI": "Navigation Mesh Info Map",      #// 48 NavMeshInfoMap
    "CELL": "Cell",      #// 49 TESObjectCELL
    "REFR": "PlacedObject",      #// 4A TESObjectREFR
    "ACHR": "PlacedNpc",      #// 4B Actor
    #"PMIS": "PlacedMissile",      #// 4C MissileProjectile   # these are combined into PlacedTrap
    #"PARW": "PlacedArrow",      #// 4D ArrowProjectile       # these are combined into PlacedTrap
    #"PGRE": "PlacedTrap",      #// 4E GrenadeProjectile      # these are combined into PlacedTrap
    #"PBEA": "PlacedBeam",      #// 4F BeamProjectile         # these are combined into PlacedTrap
    #"PFLA": "PlacedFlame",      #// 50 FlameProjectile       # these are combined into PlacedTrap
    #"PCON": "PlacedCone",      #// 51 ConeProjectile         # these are combined into PlacedTrap
    #"PPLA": "PlacedPlasma",      #// 52 PlasmaProjectile                 # these are combined into PlacedTrap
    #"PBAR": "PlacedBarrier",      #// 53 BarrierProjectile   # these are combined into PlacedTrap
    #"PEMI": "PlacedEmitter",      #// 54 EmitterProjectile                # these are combined into PlacedTrap
    #"PHZD": "PlacedHazard",      #// 55 Hazard              # these are combined into PlacedTrap
    "WRLD": "Worldspace",      #// 56 TESWorldSpace
    "NAVM": "NavigationMesh",      #// 57 NavMesh
    "TLOD": "TLODRecord",      #// 58
    "DIAL": "DialogTopic",      #// 59 TESTopic
    "INFO": "DialogResponses",      #// 5A TESTopicInfo
    "QUST": "Quest",      #// 5B TESQuest
    "IDLE": "IdleAnimation",      #// 5C TESIdleForm
    "PACK": "Package",      #// 5D TESPackage
    "CSTY": "CombatStyle",      #// 5E TESCombatStyle
    "LSCR": "LoadScreen",      #// 5F TESLoadScreen
    "LVSP": "LeveledSpell",      #// 60 TESLevSpell
    "ANIO": "Animated Object",      #// 61 TESObjectANIO
    "WATR": "Water",      #// 62 TESWaterForm
    "EFSH": "EffectShader",      #// 63 TESEffectShader
    "TOFT": "TOFTRecord",      #// 64
    "EXPL": "Explosion",      #// 65 BGSExplosion
    "DEBR": "Debris",      #// 66 BGSDebris
    "IMGS": "ImageSpace",      #// 67 TESImageSpace
    "IMAD": "ImageSpaceAdapter",      #// 68 TESImageSpaceModifier
    "FLST": "FormList",      #// 69 BGSListForm
    "PERK": "Perk",      #// 6A BGSPerk
    "BPTD": "BodyPartData",      #// 6B BGSBodyPartData
    "ADDN": "AddonNode",      #// 6C BGSAddonNode
    "AVIF": "ActorValueInformation",      #// 6D ActorValueInfo
    "CAMS": "CameraShot",      #// 6E BGSCameraShot
    "CPTH": "CameraPath",      #// 6F BGSCameraPath
    "VTYP": "VoiceType",      #// 70 BGSVoiceType
    "MATT": "MaterialType",      #// 71 BGSMaterialType
    "IPCT": "Impact",      #// 72 BGSImpactData
    "IPDS": "ImpactDataSet",      #// 73 BGSImpactDataSet
    "ARMA": "ArmorAddon",      #// 74 TESObjectARMA
    "LCTN": "Location",      #// 75 BGSLocation
    "MESG": "Message",      #// 76 BGSMessage
    "RGDL": "RGDLRecord",      #// 77
    "DOBJ": "DefaultObjectManager",      #// 78 BGSDefaultObjectManager
    "DFOB": "DefaultObject",      #// 79 BGSDefaultObject
    "LGTM": "LightingTemplate",      #// 7A BGSLightingTemplate
    "MUSC": "MusicType",      #// 7B BGSMusicType
    "FSTP": "Footstep",      #// 7C BGSFootstep
    "FSTS": "Footstep Set",      #// 7D BGSFootstepSet
    #"SMBN": "StoryManagerBranchNode",      #// 7E BGSStoryManagerBranchNode        #Combined into StoryManagerNodes
    #"SMQN": "StoryManagerQuestNode",      #// 7F BGSStoryManagerQuestNode          #Combined into StoryManagerNodes
    #"SMEN": "StoryManagerEventNode",      #// 80 BGSStoryManagerEventNode          #Combined into StoryManagerNodes
    "DLBR": "DialogBranch",      #// 81 BGSDialogueBranch
    "MUST": "MusicTrack",      #// 82 BGSMusicTrackFormWrapper
    "DLVW": "DialogView",      #// 83
    #"WOOP": "WordOfPower",      #// 84 TESWordOfPower  # Relevant in starfield?? :D
    #"SHOU": "Shout",      #// 85 TESShout               # Relevant in starfield?? :D
    "EQUP": "EquipType",      #// 86 BGSEquipSlot
    "RELA": "Relationship",      #// 87 BGSRelationship
    "SCEN": "Scene",      #// 88 BGSScene
    "ASTP": "AssociationType",      #// 89 BGSAssociationType
    "OTFT": "Outfit",      #// 8A BGSOutfit
    "ARTO": "ArtObject",      #// 8B BGSArtObject
    "MATO": "MaterialObject",      #// 8C BGSMaterialObject
    "MOVT": "MovementType",      #// 8D BGSMovementType
    "DUAL": "DualCastData",      #// 8E BGSDualCastData
    "COLL": "CollisionLayer",      #// 8F BGSCollisionLayer
    "CLFM": "ColorRecord",      #// 90 BGSColorForm
    "REVB": "ReverbParameters",      #// 91 BGSReverbParameters
    "RFGP": "ReferenceGroup",      #// 92 BGSReferenceGroup
    "AMDL": "AimModel",      #// 93 BGSAimModel
    "AAMD": "Aim Assist Model Data",      #// 94 BGSAimAssistModel
    "MAAM": "Melee Aim Assist Model",      #// 95 BGSMeleeAimAssistModel
    "LAYR": "Layer",      #// 96
    "COBJ": "ConstructibleObject",      #// 97 BGSConstructibleObject
    "OMOD": "ObjectModification",      #// 98 BGSMod::Attachment::Mod
    "ZOOM": "Zoom",      #// 99 BGSAimDownSightModel
    "INNR": "InstanceNamingRules",      #// 9A BGSInstanceNamingRules
    "KSSM": "SoundKeywordMapping",      #// 9B BGSSoundKeywordMapping
    "SCCO": "SceneCollection",      #// 9C
    "AORU": "AttractionRule",      #// 9D BGSAttractionRule
    "STAG": "AnimationSoundTagSet",      #// 9E BGSSoundTagSet
    "IRES": "Resource",      #// 9F BGSResource
    "BIOM": "Biome",      #// A0 BGSBiome
    "NOCM": "Navigation Mesh Obstacle Cover Manager",      #// A1 NavMeshObstacleCoverManager
    "LENS": "LensFlare",      #// A2 BGSLensFlare
    "LSPR": "LSPRRecord",      #// A3
    "OVIS": "ObjectVisibilityManager",      #// A4 BGSObjectVisibilityManager
    #"DLYR": "",      #// A5
    "STND": "Snap Template Node",      #// A6 BGSSnapTemplateNode
    "STMP": "Snap Template",      #// A7 BGSSnapTemplate
    "GCVR": "Ground Cover",      #// A8 BGSGroundCover
    #############"MRPH": "MorphableObject",      #// A9 BGSMorphableObject
    "TRAV": "Traversal",      #// AA BGSTraversal
    "RSGD": "Resource Generation Data",      #// AB BGSResourceGenerationData
    "OSWP": "Biome Swap",      #// AC BGSObjectSwap
    "ATMO": "Atmosphere",      #// AD BGSAtmosphere
    "LVSC": "Leveled Space Cell",      #// AE BGSLevSpaceCell
    "SPCH": "Speech Challenge",      #// AF BGSSpeechChallengeObject
    #"RESO": "",      #// B0
    "AAPD": "AimAssistPose",      #// B1 BGSAimAssistPoseData
    "VOLI": "VolumetricLighting",      #// B2 BGSVolumetricLighting
    "SFBK": "Surface Block",      #// B3 BGSSurface::Block
    "SFPC": "Surface Pattern Config",      #// B4
    "SFPT": "Surface Pattern",      #// B5 BGSSurface::Pattern
    "SFTR": "Surface Tree",      #// B6 BGSSurface::Tree
    "PCMT": "Planet Content Manager Tree",      #// B7 BGSPlanetContentManagerTree
    "BMOD": "Bone Modifier",      #// B8 BGSBoneModifier
    "STBH": "Snap Template Behavior",      #// B9 BGSSnapBehavior
    "PNDT": "Planet Data",      #// BA BGSPlanet::PlanetData
    #"IUTF": "",      #// BB
    "CNDF": "Condition Form",      #// BC BGSConditionForm
    "PCBN": "Planet Content Manager Branch Node",      #// BD BGSPlanetContentManagerBranchNode
    "PCCN": "Planet Content Manager Content Node",      #// BE BGSPlanetContentManagerContentNode
    "STDT": "Galaxy Star Data",      #// BF BSGalaxy::BGSStar
    "WWED": "WWise Event Data",      #// C0 BGSWwiseEventForm
    "RSPJ": "Research Project",      #// C1 BGSResearchProjectForm
    "AOPS": "Aim Optical Sight Marker",      #// C2 BGSAimOpticalSightModel
    "AMBS": "Ambience Set",      #// C3 BGSAmbienceSet
    "WBAR": "WBARRecord",      #// C4 BGSWeaponBarrelModel
    "PTST": "Surface Pattern Style",      #// C5 BGSSurface::PatternStyle
    "LMSW": "Layered Material Swap",      #// C6 BGSLayeredMaterialSwap
    "FORC": "Force Data",      #// C7 BGSForceData
    "TMLM": "Terminal Menu",      #// C8 BGSTerminalMenu
    "EFSQ": "Effect Sequence",      #// C9 BGSEffectSequenceForm
    "SDLT": "Secondary Damage List",      #// CA BGSSecondaryDamageList
    "MTPT": "Material Path",      #// CB BGSMaterialPathForm
    "CLDF": "Clouds",      #// CC BGSCloudForm
    "FOGV": "Fog Volume",      #// CD BGSFogVolumeForm
    "WKMF": "WWise Keyword Mapping",      #// CE BGSWwiseKeywordMapping
    "LGDI": "Legendary Item",      #// CF BGSLegendaryItem
    "PSDC": "Particle System Define Collision",      #// D0 BGSParticleSystemDefineCollection
    "SUNP": "Galaxy Sun Preset",      #// D1 BSGalaxy::BGSSunPresetForm
    "PMFT": "Photo Mode Feature",      #// D2 BGSPhotoModeFeature
    "TODD": "Time Of Day Data",      #// D3 BGSTimeOfDayData
    "AVMD": "Actor Value Modulations",      #// D4 BGSAVMData
    #"PERS": "",      #// D5 TESDataHandlerPersistentCreatedUtil::BGSPersistentIDsForm
    "CHAL": "Challenge",      #// D6 BGSChallengeForm
}

# dumped from starfield.esm
starfield_esm_records = [
    "GMST",
    "KYWD",
    "FFKW",
    "LCRT",
    "AACT",
    "TRNS",
    "TXST",
    "GLOB",
    "DMGT",
    "CLAS",
    "FACT",
    "AFFE",
    "HDPT",
    "RACE",
    "SOUN",
    "SECH",
    "ASPC",
    "AOPF",
    "MGEF",
    "LTEX",
    "PDCL",
    "ENCH",
    "SPEL",
    "ACTI",
    "CURV",
    "CUR3",
    "ARMO",
    "BOOK",
    "CONT",
    "DOOR",
    "LIGH",
    "MISC",
    "STAT",
    "SCOL",
    "PKIN",
    "MSTT",
    "GRAS",
    "FLOR",
    "FURN",
    "WEAP",
    "AMMO",
    "NPC_",
    "LVLN",
    "LVLP",
    "KEYM",
    "ALCH",
    "IDLM",
    "BMMO",
    "PROJ",
    "HAZD",
    "BNDS",
    "TERM",
    "LVLI",
    "GBFT",
    "GBFM",
    "LVLB",
    "WTHR",
    "WTHS",
    "CLMT",
    "SPGD",
    "REGN",
    "NAVI",
    "CELL",
    "REFR",
    "PGRE",
    "ACHR",
    "NAVM",
    "PHZD",
    "WRLD",
    "QUST",
    "SCEN",
    "DIAL",
    "INFO",
    "DLBR",
    "IDLE",
    "PACK",
    "CSTY",
    "LSCR",
    "ANIO",
    "WATR",
    "EFSH",
    "EXPL",
    "DEBR",
    "IMGS",
    "IMAD",
    "FLST",
    "PERK",
    "BPTD",
    "ADDN",
    "AVIF",
    "CAMS",
    "CPTH",
    "VTYP",
    "MATT",
    "IPCT",
    "IPDS",
    "ARMA",
    "LCTN",
    "MESG",
    "DOBJ",
    "DFOB",
    "LGTM",
    "MUSC",
    "FSTP",
    "FSTS",
    "SMBN",
    "SMQN",
    "SMEN",
    "MUST",
    "EQUP",
    "OTFT",
    "ARTO",
    "MOVT",
    "COLL",
    "CLFM",
    "REVB",
    "RFGP",
    "AMDL",
    "AAMD",
    "MAAM",
    "LAYR",
    "COBJ",
    "OMOD",
    "ZOOM",
    "INNR",
    "KSSM",
    "AORU",
    "STAG",
    "IRES",
    "BIOM",
    "NOCM",
    "LENS",
    "OVIS",
    "STND",
    "STMP",
    "GCVR",
    "MRPH",
    "TRAV",
    "RSGD",
    "OSWP",
    "ATMO",
    "LVSC",
    "SPCH",
    "AAPD",
    "VOLI",
    "SFBK",
    "SFPC",
    "SFPT",
    "SFTR",
    "PCMT",
    "BMOD",
    "STBH",
    "PNDT",
    "CNDF",
    "PCBN",
    "PCCN",
    "STDT",
    "WWED",
    "RSPJ",
    "AOPS",
    "AMBS",
    "WBAR",
    "PTST",
    "LMSW",
    "FORC",
    "TMLM",
    "EFSQ",
    "SDLT",
    "MTPT",
    "CLDF",
    "FOGV",
    "WKMF",
    "LGDI",
    "PSDC",
    "SUNP",
    "PMFT",
    "TODD",
    "AVMD",
    "CHAL"
]

filtered_records = dict((k, records[k]) for k in starfield_esm_records if k in records)

missing_records = [x for x in starfield_esm_records if x not in records.keys()]
# AOPF
# PGRE
# PHZD
# SMBN
# SMQN
# SMEN
# MRPH

for key in filtered_records:
    record = filtered_records[key].replace(" ", "") #remove whitespace from filename
    with open("TEMPLATE.xml", 'r') as templatefile:
        template = templatefile.read()
    with open(f"{record}.xml", 'w+') as writefile:
        writefile.write(template.replace('[NAME]', record).replace('[RECORD_TYPE]', key))

Then manually removed the files tagged as modified instead of added!

The key for the dict and the comment on that row was taken from here https://github.com/Starfield-Reverse-Engineering/CommonLibSF/blob/main/CommonLibSF/include/RE/F/FormTypes.h

Then the value part of the dict was first manually fetched from xEdit wbDefinitionsSF1.pas

The values still having a space in it is from the name defined in xEdit. If it's PascalCasing I took the name you've already used for that form in FO4/OBL/SKYRIM

I also commented out some line with some extra comment added to it

Hopefully this helps somehow! Like I said I don't care if you take the script and run it manually locally

FlayaN commented 1 year ago

https://github.com/Mutagen-Modding/Mutagen.Bethesda.Generation.Tools/blob/main/Mutagen.Bethesda.Generation.Tools/ContentAnalyzers/DumpSubrecords.cs

Thanks for the link! I basically did a local change of that to remove the -m filter and print all major records found in the output so I could paste that into the python script i posted here!