Fallout 4 Basic Support

lojack5 commented 8 years ago

Fallout 4 is out!

Things Bash should be able to do:

[x] Detect and load seeing Fallout 4 as a valid game
[x] Read load order information - Probably plugins.txt determined like Skyrim?
[x] Read very basic information from ESP/ESM files.
[x] Read directory structures from the B2A format archives. This is needed so we know the basic Data structure layout.
[x] Figure out what the CDX file format is for. Is it just like BSL for Skyrim? If so, no extra work needed. Update: It appears to be a "Cell Reource Index File List", according to Fallout4.ini, so probably not needed to be understood until we want the Bashed Patch to work on CELLs.
[x] Read basic information from ESS files (did they change the name? no idea, I haven't actually played yet).

Work on this will be ongoing in my lojack-f04 branch. If you want to pitch in let me know so I know not to rewrite history on that branch. The goal of this is to get very minimal Fallout 4 Support, so at the very least BAIN and load order management works.

lojack5 commented 8 years ago

Note: the ESM/ESP format looks almost unchanged, differences will be in allowed record/subrecord types most likely

Hardest part will be decyphering the new B2A format, and figuring out what Fallout 4.cdx is for (from an initial inspection at least). Hopeful guess is it's similar to Skyrim's BSL files (and hence unneeded for actual modding), but need's investigation (added to list).

lojack5 commented 8 years ago

initial dump of stats for Fallout4.esm, seems it's the exact same base format as Skyrim/FO3, so getting bash to read it will be very easy. I didn't even have to change any code in TESDumpStats to get it to run correctly, so that's encouraging. More work on this tomorrow night probably.

Utumno commented 8 years ago

Welcome back :) Thanks for working on this - feel free to rewrite history, not my area of expertise, won't mess with it (and anyway I am immune to history rewrite :D).

One thing I would like us to do though is rework the game.template - I want the patcher stuff out and into game/<game>/patcher/__init__.py (see #124) so this is a nice opportunity to clean it up.

Sharlikran commented 8 years ago

@lojack5 There are no plugins so having a Bash Patch for Leveled lists and other things isn't really needed at this time. However, if you are starting to fiddle around with things and start getting BAIN working then I have one piece of information from Zilav and Hlp.

The file header has a mysterious spacer or padding in it. It is not preceded by a 4 letter record signature.

[Beggining of the header is the same] CNAM [data] 2x TNAM [data] 4 Bytes INTV [data] INCC [data]

I will have to get my information from them since I don't have a copy of the game or a computer that can run it. My sons computer might work just to get it installed so I can copy it on a flash drive and at least help decode the files but not sure if that's even possible yet.

Arthmoor commented 8 years ago

If I may suggest - Jon has a utility built to extract the new archive formats : http://afkmods.com/index.php?/topic/4300-bae-bethesda-archive-extractor/ He can probably answer any technical questions about this. As can the SKSE/F4SE team who have their own command line tool for it as well.

He has the game, and now so do I after finding a good deal on a key reseller site. So I can be of some LIMITED assistance with getting things tested and maybe some minor coding but don't count on too much more than that as my Python never got past the Baby's First Program stage.

lojack5 commented 8 years ago

aact_records

The file header has a mysterious spacer or padding in it. It is not preceded by a 4 letter record signature.

[Beggining of the header is the same] CNAM [data] 2x TNAM [data] 4 Bytes INTV [data] INCC [data]

That's not been my experience. Looking at Fallout4.esm I see the exact same structure and layout as with Skryim's ESP/ESMs. So much the same that I literally needed no code changes in order to dump the text file I linked above. Inspecting the file in a Hex Editor also reveals that yep, the record type's are still in fact there:

Example of the AACT GRUP: aact_records

And here of course is the TES4 record at the beginning: tes4_record

So it looks like the ESM/ESP format will be identical in structure, it will just be which subrecords are required, format of the subrecords, and which records are possible that has changed. Other things that may play a roll is the .CDX file, so more info to be found for sure, but at least the ESP/ESM format is pretty much the same.

If I may suggest - Jon has a utility built to extract the new archive formats : http://afkmods.com/index.php?/topic/4300-bae-bethesda-archive-extractor/ He can probably answer any technical questions about this. As can the SKSE/F4SE team who have their own command line tool for it as well.

Thanks, I'll stop by that thread when I'm ready to start working on the "BAIN being able to see inside of B2A files" part of this. In the meantime I'll just use the tool to get the "allowed" Data folders to install into.

I won't really have time to actually do much work on this 'til the weekend, I'm just gathering any info that's easy to get at the moment. A version that can load up and see the plugins present and operate BAIN should be very easy to be implemented in a short amount of time. The B2A stuff may take a little while longer, depending if I can get either John/F4SE team to do a command line tool that lists contents and CRCs of the B2A (unless theirs can already do that). If not I'll have to code something like that myself, but it should only take a few hours if I get detailed info from them.

Edit @Utumno : I plan on documenting what is needed to get at least basic support for Fallout 4 working, that way we have a sort of guide for new games. In doing that (Since I'll have to do it anyway for FO4), I'll cleanup the template files as well.

lojack5 commented 8 years ago

Note: If someone could do me a favor and see if loose files override B2A files, that'd be super. IE: if they don't, then BAIN will be almost unnecessary, other than to install ESP/ESM/B2A files. Most likely loose files still override I'd think though. Also, any know if mods will still be installed directly to the data folder, or if bethesda actually finally moved that to a /mods folder or something?

zilav commented 8 years ago

Loose files can override, but for now you need to manually add subfolder names in Data folder for this like here http://www.nexusmods.com/fallout4/mods/130/ Maybe it would change after the GECK release.

Ortham commented 8 years ago

FWIW I'll be looking at Fallout 4's load order system. I don't have the game, but @pStyl3 has agreed to Guinea pig for me.

Going off screenshots of Fallout4.esm in a hex editor, it does seem to use Skyrim's format (which actually goes back to Fallout 3) like Lojack says, though I sent pStyl3 a test parser and it failed for some reason. That's probably just my code though.

Update: Yep, it was my code.

Sharlikran commented 8 years ago

@lojack5 The screenshots don't work anymore. Not sure the GitHub resource can be linked like that. Also sent you e-mail and CCd Zilav and Hlp to the old e-mail account you gave me once before.

Hugues92 commented 8 years ago

Links to a folder on my google drive that contains what I extracted so far: https://drive.google.com/folderview?id=0Bwcnayjp3CyNZWx3eXdCVkJXVGM&usp=sharing

There are 159 MainRecord signatures known to the runtime.

lojack5 commented 8 years ago

@zilav : thanks, that means BAIN will need to do all its normal duties. @Sharlikran : try now, I think I know what went wrong there, and I'll read through that email once I can, like I said, not much time. @Hugues92 : nice. I only counted 139 types actually present in Fallout4.esm, but if you're decoding from the runtime I guess that means there's more types that are at least possible (so I guess we'll see if the GECK can create them?)

Sharlikran commented 8 years ago

@lojack5 Although I don't have a computer to run this I might be able to jury rig things by installing it on a friends computer and copying it to an SSD just to have the files. If so I'll let you know which branch on my fork has the changes. I'll leave it up to you to decide whether or not you prefer to pull from me as a downstream or work from my fork. I already have the very most basic of changes for FO4Dump while I work on a way to get the files.

Ortham commented 8 years ago

Could anyone decoding the plugin structure please document their findings in the FOPDoc repository? I imagine much of it can be copy/pasted from the existing files. I'll be adding what I've learned once I've had time to cross-reference it with Lojack's stats dump.

In other news, the load order system seems to work exactly the same as Skyrim's, though this is obviously subject to change (plugins.txt being overwritten on launch may point to this). I haven't checked the encoding that plugins.txt uses or the encoding used in TES4's MAST subrecord yet, but I'm not holding my breath for Unicode support.

@zilav @Sharlikran @lojack5 @Utumno @matortheeternal: On the subject of load order, does anyone have any suggestions for improvements to / replacement of our use of loadorder.txt to store load order including inactive plugins? TES5Edit, Wrye Bash, SkyProc and Mator Smash are the utilities that come to mind for which having inactive plugins amongst active plugins (as opposed to listed after them) is beneficial to users, so what do you think?

zilav commented 8 years ago

I don't think that anything would be better than the current system, it works and it is supported by all mod managers, and easy to handle even with custom code if the need arises. However I know that Beth could in theory listen to community and introduce changes with official patches on the early stages of the game. The optimal variant will be storing inactive plugins right inside plugins.txt with some prefix like '-', but that requires someone close with gstaff to speak with them privately.

2015-11-13 0:20 GMT+03:00 Oliver Hamlet notifications@github.com:

Could anyone decoding the plugin structure please document their findings in the FOPDoc https://github.com/TES5Edit/fopdoc repository? I imagine much of it can be copy/pasted from the existing files. I'll be adding what I've learned once I've had time to cross-reference it with Lojack's stats dump.

In other news, the load order system seems to work exactly the same as Skyrim's, though this is obviously subject to change. I haven't checked the encoding that plugins.txt uses or the encoding used in TES4's MAST subrecord yet, but I'm not holding my breath for Unicode support.

@zilav https://github.com/zilav @Sharlikran https://github.com/Sharlikran @lojack5 https://github.com/lojack5 @Utumno https://github.com/Utumno @matortheeternal https://github.com/matortheeternal: On the subject of load order, does anyone have any suggestions for improvements to / replacement of our use of loadorder.txt to store load order including inactive plugins? TES5Edit, Wrye Bash, SkyProc and Mator Smash are the utilities that come to mind for which having inactive plugins amongst active plugins (as opposed to listed after them) is beneficial to users, so what do you think?

— Reply to this email directly or view it on GitHub https://github.com/wrye-bash/wrye-bash/issues/251#issuecomment-156237714 .

Utumno commented 8 years ago

loadorder.txt is fine - but I think libloadorder needs improvements - I have forked it here: https://github.com/Utumno/libloadorder/ and Bash 306 ships with the libloadorder.dll from this fork. Better open a separate issue to discuss this though.

Ortham commented 8 years ago

loadorder.txt is fine - but I think libloadorder needs improvements - I have forked it here: https://github.com/Utumno/libloadorder/ and Bash 306 ships with the libloadorder.dll from this fork. Better open a separate issue to discuss this though.

Open an issue in libloadorder's issue tracker to discuss it then: you may wish to take a look at the changes I'm making in my refactor branch.

lojack5 commented 8 years ago

Agreed. I think the idea of loadorder.txt is great, and the specifics of how it works are good. The only way we could really do without something like that is like @zilav said, which is if we could convince the Devs to not override comments in plugins.txt somehow, so we could use comments for inactive plugins - in orther words, unlikely. Anyway, I'll get to work on this tomorrow, so at least Bash should be able to load and BAIN stuff then, I'm not going into much detail on the plugin format itself yet, just enough so Bash can understand the bare minimum.

lojack5 commented 8 years ago

So got hit with a bad stomach flu over the weekend and into this week, I'll have to work on this this coming weekend.

Utumno commented 8 years ago

@lojack5 - hope you get better soon !

Be sure to rebase on dev and then merge or let me do it myself. Also do have a look in https://github.com/wrye-bash/wrye-bash/commit/8a4b329904ba58819e357c40c75e190a2bcc0375 - if you can hash up a constants.template based on the changes in this merge it would be grand

Sharlikran commented 8 years ago

@lojack5 I have basic Fallout 4 support started in a branch named dev-fallout4-installer. I'm pretty sure I have missed something.

[ ] Need loadorder32.dll to be updated for FO4 (If that is even required) @Utumno
[ ] Need loot FO4 support (I suspect this is done already) @WrinklyNinja

About the only patcher active is for the Merge Patches because there were a number of records that were unchanged. That will give Wrye Bash at least one patcher. I don't know if it needs an active patcher but I assume it might. It might not like an empty list or set but I'm really not sure.

Ortham commented 8 years ago

LOOT doesn't yet have FO4 support, but I'll let you know when it does. Though my executables are still incompatible with Bash's XP compatibility requirement, so I'm not sure how much good that will do you.

Arthmoor commented 8 years ago

How exactly is this supposed to work? Fallout 4 requires a 64 bit copy of Windows 7 or higher in order to run. I can't see this working out very well if Bash is going to continue to hamstring itself by sticking with XP.

Sharlikran commented 8 years ago

Wrye Bash does not have specific XP support that I am aware of. I am sure the Skryim/Oblivion version runs in a 64 bit OS and I know Wrye Bash isn't planed on becoming a 64 bit program. FO4Edit is a 32 bit program because the ZLib dll files are not 64 bit. I know what you are getting at but don't understand why you are concerned about it.

Sharlikran commented 8 years ago

Wrye Bash starting
Using Wrye Bash Version 307
Python version: 2.7.8
wxPython version: 2.8.12.1 (msw-unicode)
input encoding: None; output encoding: None; locale: ('en_US', 'cp1252')
filesystem encoding: mbcs 
Searching for game to manage:
bush.py   87 _supportedGames: Detected the following supported games via Windows Registry:
bush.py   89 _supportedGames:  fallout3: C:\Games\steamapps\common\Fallout 3 goty
bush.py   89 _supportedGames:  falloutnv: C:\Games\steamapps\common\fallout new vegas
bush.py   89 _supportedGames:  oblivion: C:\Games\steamapps\common\Oblivion
bush.py   89 _supportedGames:  skyrim: F:\Games\steamapps\common\Skyrim
bush.py   89 _supportedGames:  fallout4: C:\Games\SteamApps\common\Fallout 4
bush.py  147 _detectGames: Detecting games via the -o argument, bash.ini and relative path:
bush.py  153 _detectGames: Set game mode to fallout4 found in parent directory of Mopy:  C:\Games\SteamApps\common\Fallout 4
bush.py  187 setGame: No preferred game specified.
bush.py  168 __setGame:  Using fallout4 game: C:\Games\SteamApps\common\Fallout 4
testing UAC
__init__.py 5262 __init__: Using libbsa API version: 2.0.0
Traceback (most recent call last):
  File "Wrye Bash Launcher.pyw", line 88, in <module>
    bash.main()
  File "bash\bash.py", line 364, in main
    opts.oblivionPath, bashIni)
  File "bash\bosh\__init__.py", line 10259, in initBosh
    initDirs(bashIni,personal,localAppData, oblivionPath)
  File "bash\bosh\__init__.py", line 10056, in initDirs
    configHelpers = ConfigHelpers()
  File "bash\bosh\__init__.py", line 5264, in __init__
    loot.Init(dirs['compiled'].s)
  File "bash\loot.py", line 155, in Init
    loot_game_fo4 = c_uint.in_dll(LootApi,'loot_game_fo4').value
ValueError: symbol 'loot_game_fo4' not found

The above error is expected, I'm just posting it.

LOOT doesn't yet have FO4 support, but I'll let you know when it does.

@WrinklyNinja No problem thanks for responding. I have no idea what else I have to change for Fallout 4 support. I have Wrye Bash running up to a certain point. I know you are busy and Wrye may need more changes so whenever is fine. Besides I have a lot to do all over the place anyway.

Arthmoor commented 8 years ago

I'm concerned about it because XP holds things back, and I'd rather not see Bash become one of those things held back. From what I can see, the trend from game developers isn't even including Vista, likely because Microsoft terminated support for that too.

No, we don't need to specifically go out of our way to cripple it, but I don't think it should remain a consideration for future development either.

Also, I probably overemphasized the 64 bit aspect, the main point was Windows 7 and up.

Utumno commented 8 years ago

Point number one here is that Bash should gracefully degrade if Loot failed to load for whatever reason - as in function without loot. Patch this first.

Ortham commented 8 years ago

From what I can see, the trend from game developers isn't even including Vista, likely because Microsoft terminated support for that too.

Vista is still supported, and from what I know there's no technical reason not to support it (there's no Windows API or toolset differences I've come across, at least), I think it's just because hardly anyone uses it so it's not worth mentioning.

lojack5 commented 8 years ago

I'm working on this currently as I type. I've got Bash almost starting, just noticed some new notifications so stopped by here.

I already patched Bash so it starts gracefully if the LOOT API does not support a game. It will however still crash if the LOOT API just fails to be present, etc.
Fun fact: the Fallout4.exe embeded icons are ALL Skyrim, except for the last one, hah!
I've already patched libloadorder (branched from Utumno's) to work with Fallout 4. This is just a stopgap measure until we have things fully implemented by @WrinklyNinja

@Sharlikran wish I had had seen you had some work done before I stared, but I'm almost there now myself, I just need a couple small tweaks and it should be ok.

lojack5 commented 8 years ago

Ok @wrye-bash/bashers , try out my branch here. It has very basic minimal support for Fallout 4.

It loads, it sees plugins and read basic info from them. BAIN seems to work fine.

Things it still cannot do are:

Read info from save game files
Any sort of Bashed Patching (no record definitions at all @Sharlikran once we get basic support feel free to start putting whichever records are good into the Fallout 4 definitions)
Editing of plugin headers
Bashed tags/dirty detection (not needed yet, but from LOOT API)

Assuming this all works well, I'll ask @Utumno for a quick code review then merge. After that I can start populating some Issues for what Fallout 4 needs fleshed out. For example, to ensure loose files actually override, an INI setting is needed. Ideally Bash would enable that setting automatically or at least prompt the user to do so, right now it does not, and simply provides INI Tweak file that will perform required edits.

Also @Utumno, check out 622f8c3b9c56a4fcf4e9fd6fbe286613b5132ca9, it fixes a bug in basher that was giving me issues at first. Dunno if you want to cherry pick it or just wait for the merge.

Edit: also, not sure if we want to reach out to the forums for more testers.

Utumno commented 8 years ago

I'll ask @Utumno for a quick code review then merge.

Will do ASAP - thanks for working on this ! Preliminary: please do hash up a constants.template based on https://github.com/wrye-bash/wrye-bash/commit/8a4b329904ba58819e357c40c75e190a2bcc0375 - just to say (if it's not clear) that when you see commits with off hand messages they are temporary (in my branches FFF stands for fixup, RRR for rename, EEE for edit, REV for review etc) - but this merge is pretty much ready for dev. Note in particular gmstEids , GmstTweaks etc are in constants (potentially huge) and also - no star imports (from records import *) - explicitly import. The comment that needs to be here refers to a stage in the refactoring that this import would blow if in the beginning of the file due to cyclic imports - I think it would still blow. <game>/constants.pys should be # no imports files. New code should be formatted (as in space after comma, wrapped at 79, newline at EOF etc)

I've already patched libloadorder (branched from Utumno's) to work with Fallout 4

That will be a bit harder - commented there. We should ideally commit the dll once (binary commits - yak) - and take this opportunity to merge in upstream liblo master.

I already patched Bash so it starts gracefully if the LOOT API does not support a game. It will however still crash if the LOOT API just fails to be present, etc.

Since you are there I would appreciate your giving some more thought on it - we should decouple Bash from Loot (for engineering (!) as well as linux compatibility reasons). I had a lot of exceptions thrown at me while writing 306 (and in skyrim patchers thread) but had never the time to look at it seriously.

Sharlikran commented 8 years ago

@lojack5 @Utumno can someone help me please? I am wondering if you can direct me to where the strings file names are handled. For Skyrim it is Skyrim_English.DLSTRINGS but for Fallout 4 it is Fallout4_En.DLSTRINGS. I am not too good with Python can you provide a code snipit if possible?

I need it to be something like this.

If Filename = var+'English'+FileExt or var+'En'+FileExt then begin

Var is the full path and Filename "C:\Games\SteamApps\common\Fallout 4\Data\Strings\Fallout4_" and FileExt is one of the DLSTRINGS, ILSTRINGS, STRINGS.

it does not have to match what I am asking for it just needs to know that it is either English or En, and French or Fr etc.

lojack5 commented 8 years ago

@Utumno : I wanted this branch focused on just getting fallout support going, so I wanted minimal refactoring type changes here. I can do another to clean up the template file, etc though. I had some ideas while working on this.

@Sharlikran : handled in multiple places. StringTable handles the grunt work of loading the strings files. For determining the filename, that's done via reading the game's main INI file, ie Fallout4.ini or Skyrim.ini, so no code changes are needed for that to work, since everything needed is gleaned from the INI file.

Sharlikran commented 8 years ago

@lojack5 Thanks. I can't run the game on my LGA 775 Quad core. I was gifted a copy of the game. So before I release an experemental version can you check your .ini file. In fallout4.ini you mean you literally have sLanguage=EN under [General] ?

lojack5 commented 8 years ago

Yes, the game launcher automatically sets that INI entry based on your language settings. Bash reads this to determine which Strings files to open.

Sharlikran commented 8 years ago

@lojack5 Would you be willing to click this link, download the file and test that with Fallout 4 please?

Sharlikran commented 8 years ago

@lojack5

Traceback (most recent call last):
  File "bash\basher\__init__.py", line 3435, in Execute
    panel = globals()[className](Link.Frame.notebook)
  File "bash\basher\__init__.py", line 2026, in __init__
    bush.game.displayName)
bash.bolt.BoltError: Wrye Bash cannot read save games for Fallout 4.

EDIT: Quoting Arthmoor from the Beth Forum, I should clarify that the tab is indeed not displayed, and I wasn't trying to access it. The error window was popping up on its own with a long list of what Bash believed to be corrupted headers. They obviously are not corrupt, it just doesn't know what to do with them.

It looks like both Arthmoor and EssArrBee both get that error. At leaset with Arthmoor he isn't even trying to go to the saves tab at all.

EssArrBee tested the installers tab and this was the response.

Clicking on the Installers tab gives this error:

Traceback (most recent call last):
  File "bash\balt.pyo", line 1796, in _conversation_wrapper
  File "bash\basher\__init__.pyo", line 3953, in RefreshData
  File "bash\bosh\__init__.pyo", line 4977, in refresh
  File "bash\bosh\__init__.pyo", line 3894, in refresh
  File "bash\bosh\__init__.pyo", line 3111, in sameAs
AttributeError: 'NoneType' object has no attribute 'size'

At this stage in development some people might have only one plugin or no plugins and just graphical mods like texture replacers.

I don't know how to resolve that and I don't know whether or not if it's from the Skyrim/Oblivion version. The testers can't really try Utumno's version with Fallout 4 so you will have to use my version from the link I already provided above.

Arthmoor tried the Clean Data procedure and it removed Fallout4 - Geometry.csg. I saw that when looking at your file list and saying to myself Fallout4 - Geometry.ba2, hmm... I don't have that file. I can fix that though.

Sharlikran commented 8 years ago

@lojack5 @Utumno It looks like you can get to the Saves tab another way. The error about the saves headers seems to come up on it's own however, the contextual menu has the option.

savestab

Any way to use canReadBasic from class ess or some other existing variable to remove that from the contextual menu? If not then is there a way to use fsName remove that from the Tabs drop down menu?

Utumno commented 8 years ago

@Sharlikran: not a priority now - hopefully saves will be working soon if not will look into it then.

@lojack5:

I can do another to clean up the template file, etc though. I had some ideas while working on this.

Interested in seeing/discussing them - but better in #65 and/or #124

I wanted this branch focused on just getting fallout support going, so I wanted minimal refactoring type changes here

Alright will just edit those before I merge - will notify you before ofc and discuss - priority now is the dll - see https://github.com/Utumno/libloadorder/commits/utumno-merge-upstream-master%2Bfov4. Be sure to keep pushing any updates

Sharlikran commented 8 years ago

Okay. Well wasn't looking to have anyone decode the save game files. I was just reporting the users errors and was trying to get help so that Wrye would quit throwing an error. They get the error even when they don't try to go to the saves tab. Also there is an small issue with the Clean Data routine and the installer folders. However, with the exception of the Save Game files I have it all fixed. I'll just take care of things on my fork. No worries.

Thanks for the help guys and Thanks lojack I could not have disabled LOOT the way you did.

Sharlikran commented 8 years ago

@lojack5 You have some appreciative users. Thanks again for adding code so it runs until LOOT is updated. I made sure they knew you did the code rewrite. I could have never done that. Everyone will appreciate it I'm sure.

lojack5 commented 8 years ago

@Sharlikran : as you've discovered, it isn't that the Saves Tab is enabled, it's that the SavesInfos still get refreshed even when saves cannot be properly read. Something to note on my list of things to eventually get fixed. As for "fixing" it, for now you can just change the extension that Bash thinks is the save extension, to something silly, like ".NOTASAVE" or something, then Bash will be looking for the wrong saves. Re: Clean Data - yeah typo on my part for that one file, otherwise should be good. I'll do a fix sometime this next week in my branch for it.

As far as the Record definitions go, I'll only keep the very minimal for this branch so when 307 goes beta there's the least complexity to possibly cause issues. I just want Bash to be a basic Load Order Manager / Mod Installer for the moment. But definitely keep working on the record definitions where it's obvious they're the same, that way once the GECK comes out we can finalize much of it and possibly have more functionality than Skyrim did at the beginning.

@Utumno : Yep, lots to look at. I think after this is set and complete for minimal support, I'll look more into the package / template layout first. I'll also comment on the liblo part, but I think with your bump to the version number there's nothing else vital to get it working with Fallout 4 at least. Just an offiical build from @WrinklyNinja with your cache upgrades, etc.

So: once I have time (Weds probably): I need to:

[ ] Populate new open Issues for Fallout 4 specific things.
[ ] Write up a wiki page detailing the process and common(?) issues when coding support for a new game. (edit: utumno #65)
[x] Possibly code FOS decoding routines. It's likely someone will have at least basic info decoded for reading by that time.
[x] Look more into libloadorder stuff - see if @WrinklyNinja has any thoughts on that regarding the most recent commits from his code, etc. (edit: utumno - done with it for now I'd say)
[x] Work on refactoring game packages as well as the associated template. (edit: utumno discuss this in #151 as the branch there contains the relevant work - and in #65)

Sharlikran commented 8 years ago

@lojack5 I asked Hlp to look at the Fallout 4 save game headers since he already has code (incomplete and not fully functional) for editing save games with xEdit. Even with the WIPz code it might help resolve that error.

lojack5 commented 8 years ago

Well for avoiding the error, changing the expected file name is enough. For actually reading information though to populate (and enable usage of) the Saves Tab, that will be useful information. My quick glance at a save file in a hex editor seemed to indicate the format didn't change much, I just need the changes to be able to make Bash read them.

Editing Saves information though will probably be a while coming at the very least, but that's not a huge issue.

Sharlikran commented 8 years ago

It's fine whatever you and Utumno want to do go ahead. I normally just rebase my changes and try to follow the code. Sometimes I come across updates I can make so I'll just keep you up to date. Thanks again for your time and help.

Utumno commented 8 years ago

Sharl the installers tab error is related to saves - that's why I told you it's not a priority, I should have been more clear. Thanks for your continued support !

@lojack5: game.oblivion.ess.ext = u'.unsupported' :D

Don't wear your energy - I think that what I have here: https://github.com/wrye-bash/wrye-bash/commit/0d49807576f397e249e5ea5c536b0befa8dd33a1 is pretty much what we need for now re templates (smoothing out needed, comments etc welcome but I put the thought - except of course if you have some radical new idea)

I am in need of good coders as ever - and you moreover know bash. There is very interesting stuff to work on - one that I need done is splitting the InstallersPanel to InstallersPanel and InstallersDetails. Metallic tried but I think it's beyond him.

One other is the patchers package layout

One other is breaking cint into a package (we load all the fallout record definitions needlessly lol)

Etc

Sharlikran commented 8 years ago

Traceback (most recent call last):
  File "bash\balt.pyo", line 1796, in _conversation_wrapper
  File "bash\basher\__init__.pyo", line 3953, in RefreshData
  File "bash\bosh\__init__.pyo", line 4977, in refresh
  File "bash\bosh\__init__.pyo", line 3894, in refresh
  File "bash\bosh\__init__.pyo", line 3111, in sameAs
AttributeError: 'NoneType' object has no attribute 'size'

Yeah knowing that the error is related to the saves games is more helpful because I can post that under known issues so users aren't reporting it to me.

lojack5 commented 8 years ago

@Sharlikran : I have Bash reading save game info now. If you rebase/cherry pick my latest commit you should be good to go. Note I had to do some conflict resolution when rebasing on current dev, so you may have to as well.

@Utumno : I have everything I want with this branch complete now, how do you want to handle libloadorder? Should I compile from my branch with the open pull request, or attempt to get your merged with upstream masters branch to work?

Utumno commented 8 years ago

Replied to you in https://github.com/Utumno/libloadorder/pull/13 - so basically the second one - my merge branch merges work before the refactoring and should be harmless but testing is needed yes when it comes to merging a new dll. If you think that the merge does not work compile from your pull - but if we can get the merge to work (it should work already!) then we have all changes (apart some unneeded IIUC validity checks) in upstream before the upstream liblo refactoring merged in bash's dll - which is good and tidy.

Sharlikran commented 8 years ago

@lojack5 @Utumno I know you guys are busy but could you see if you can update the Making A Release wiki. I had a hell of a time getting GitPython installed once before. Now that I have reinstalled windows I can't get it to work. I might have it installed correctly but I'm overlooking something else.

What I did was I copied my C:\Python27\ folder to a backup location. I then let Wrye Bash install the python dependencies and I installed the additional requirements listed in the Wiki using installers. Then I compared the old and the new folder using a diff program and made sure they were exactly the same. So I know the files I had installed are the same ones I was using now.

Just FYI, the link in the wiki to GitPython has version 1.0.1 and 3.7 and neither work. Also one of the requirements seems to be gitdb as well. The old versions I was using are gitdb-0.5.4-py2.7-win32.egg and gitpython-0.3.2.rc1-py2.7.egg which neither are available anymore. There are newer versions that probably work just fine. Last time I installed using pip, a python 3 script ''py ...'', or ezinstall, or ez_setup, but I forget what I did exactly.

The instructions don't work for a novice like myself. I say that because the script is looking for what looks like a git.exe. There is no git.exe in the GitPython folder just py files. I use this to make the experimental release and my Wrye Flash releases for Valda's versions so I'd like to get it working again as soon as I can but with in reason since you two are busy.

I think I mentioned once before that I rarely update my PATH statement. I don't put a bunch stuff in it that doesn't need to be there. I am happy to add something you think will fix this but only what is required.

My Path

C:\Python27\Scripts\;C:\Python27\;C:\Python35\Scripts\;C:\Python35\;C:\Program Files (x86)\NVIDIA Corporation\PhysX\Common;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\

The Error:

No release version specified, please enter it now.
>307.0.4
2.7.8 (default, Jun 30 2014, 16:03:49) [MSC v.1500 32 bit (Intel)]
ERROR: Could not locate git.  Try adding the path to your git directory to the P
ATH environment variable.
GitPython is not set up correctly, aborting.

 All Done!
Press any key to continue . . .

My site-packages: python27

That is what I have so far. Any suggestions?

wrye-bash / wrye-bash

Fallout 4 Basic Support #251