Legendary Edition package dumper fails to finish when dumping LE2

[SterlingVix]

ME3Tweaks tool: Legendary Explorer (from Beta nightlies, Jul 4) -> Package Dumper

ME game: LE2

Bug

When trying to dump all of LE2 using the Package Dumper the process consistently hangs a little over half way through. I cancelled the first attempt and tried again; memory is maxed out but HD seems to still have activity. Pic attached:

To Reproduce

1. From ME3Tweaks mod manager (Beta channel), open the Legendary Explorer (from nightly build July 4+)
2. Open the Package Dumper util
3. Select LE2 as the game
4. Attempt to dump the game. For me, the process fails to complete, stalling out around 50% completion.

Expected behavior

The package dumper should dump all game files to .txt files in the designated output directory. The process should complete after all files are written.

Version information:

Sorry, I'm not on my gaming PC right now, but this is the nightly build from July 4th.

Stack trace

No error or trace

Other information

None, but I can provide more if asked. My system specs should be adequate to complete this process.

Mgamerz confirmed there is a mem leak on Discord (also suggested a thread cap of 6):

yeah i can see there is memory leak. for some reason it doesn't trigger in debug build

[mirh]

Sounds a bit like #217

[SterlingVix]

@mirh - it does sound very similar. I think the main differences between #217 and this are:

• My process did not complete (#217 did).

• 217 is dumping from an Xbox ISO (I'm on PC (Steam) Legendary Edition).

• 217 dumped the "CookedXenon" folder; I used the 1-click "dump game" option.

• Once I force-quit the process I did not see any lingering memory issues as described in #217. ** But, if my process had completed, I may have had the same issue; no way to know since it can't complete.

It may be the same underlying problem. I don't work with memory allocation / threading / garbage collection / etc. enough for me to hazard a guess, unfortunately.

Your call if you'd like to close this and work on #217 canonically. Thanks.

[SterlingVix]

I tried dumping again today with a smaller subset of files.

You can see a steady climb in memory; it's hit 100% but hasn't stalled.

What I have noticed is, at this point, most of the files are stuck open. They aren't moving; the remaining files are (2 or 3 of the entries here) are opening files, processing them, closing them, and the hung files remain hung.

The process nearly hung on these last 4 files (one of them just cleared):

These don't appear to be unusually large files:

• BioD_KroPrL_300HospEnt.pcc is under 9M
• BioP_HorCr1.pcc is just under 19M
• BioD_TwrMwA_207HangarAFight4.pcc is about 10M

[SterlingVix]

I tried dumping the individual files that failed:

• BioD_KroPrL_300HospEnt.pcc started running up memory and didn't finish. Just this one file was clearly on its way to eating up all my resources without completing.
• Same with BioP_HorCr1.cpp
• And same with BioD_TwrMwA_207HangarAFight4.cpp.

So the memory leak appears to be file-specific, which makes me think that a parser or something is choking unexpectedly.

The files listed above ^^^ are all stock from LE2, copied directly from the CookedPC dir into a backup dir I'm using for modding. All the other files copied in this same way processed OK.

I'm attaching BioD_KroPrL_300HospEnt.pcc, which is the smallest of the files in question (zipped so Github will accept the upload).

BioD_KroPrL_300HospEnt.zip

[Mgamerz]

Yea this is a definite memory leak. We haven't got around to doing leak checks yet. From my preliminary tests this doesn't seem to happen on LE3 dump for me, at least in debug mode. We will probably look into this this weekend.

[SirCxyrtyx]

Thanks for investigating further and finding problem files! It was trying to use to the wrong decompiler for unrealscript functions, which blew up fairly spectacularly. That's fixed in the latest nightly. I'm leaving this issue open for now though as I want to do more thorough testing of the PackageDumper to confirm there's no more issues, and maybe speed it up a bit.

[SirCxyrtyx]

Sped PackageDumper up a bit, and it seems to work fine now. I'll close this, but if you're still having issues, let us know!

[SterlingVix]

Sped PackageDumper up a bit, and it seems to work fine now. I'll close this, but if you're still having issues, let us know!

Amazing, thanks @SirCxyrtyx , all!!! I'll try dumping the problem files today and update you.

[SterlingVix]

🎉 The 3 problem files all dumped! Thanks all!