Frodo45127 / rpfm

Rusted PackFile Manager (RPFM) is a... reimplementation in Rust and Qt5 of PackFile Manager (PFM), one of the best modding tools for Total War Games.
MIT License
356 stars 57 forks source link

Cannot use mouse or keyboard after clicking to save imported files to mod pack #249

Open Exeterchiefs2023 opened 1 year ago

Exeterchiefs2023 commented 1 year ago

Hi I recently started re doing a barbarian units pack for ancient empires mod for attila as my last one got corrupted when I had to turn pc off due to unresponsive mouse and keyboard when saving something in my pack file.

I basically had to import data from ancient empires mod for textures and rigid model files for things like armour to tunics and skins etc and when I clicked save it was taking ages to sort itself out I am using latest version of rusted pfm.

Now I had to try saving stuff again in a new pack file and I cannot use the mouse or keyboard at all it's like it's frozen the PC.

Any ideas why this happens?

Frodo45127 commented 1 year ago

mmmm that sounds like you're running out of ram. When RPFM saves a pack, it loads it to ram, then writes in to disk. If your pack is 4gb, it'll you'll see a 4gb ram usage the first time you save said pack, for example. If you're close to your ram limit, windows will do windows things, and start using your disk as ram, which slows the whole system down to a crawl and sometimes freeze it. When saving, check your ram usage in task manager and see if its that.

Exeterchiefs2023 commented 1 year ago

In that case should I use that lazy loading setting to reduce ram usage my pc ram is 16bg I started a new pack and trying to just import the stuff I used imported all skins and beards to hair and helmets as well as cloaks etc.I guess I need to get myself extra ram maybe but not sure if i my pc can have extra ramOn 14 Apr 2023 19:38, Frodo45127 @.***> wrote: mmmm that sounds like you're running out of ram. When RPFM saves a pack, it loads it to ram, then writes in to disk. If your pack is 4gb, it'll you'll see a 4gb ram usage the first time you save said pack, for example. If you're close to your ram limit, windows will do windows things, and start using your disk as ram, which slows the whole system down to a crawl and sometimes freeze it. When saving, check your ram usage in task manager and see if its that.

—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you authored the thread.Message ID: @.***>

Exeterchiefs2023 commented 1 year ago

Keyboard lights were off and mouse was not working at all so had to turn pc off and file got corrupted twice it's happened so now I'm trying to not import too much and only get what I use in my unit meshes On 14 Apr 2023 19:38, Frodo45127 @.***> wrote: mmmm that sounds like you're running out of ram. When RPFM saves a pack, it loads it to ram, then writes in to disk. If your pack is 4gb, it'll you'll see a 4gb ram usage the first time you save said pack, for example. If you're close to your ram limit, windows will do windows things, and start using your disk as ram, which slows the whole system down to a crawl and sometimes freeze it. When saving, check your ram usage in task manager and see if its that.

—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you authored the thread.Message ID: @.***>

Frodo45127 commented 1 year ago

Unless you're doing weird things or have a lot of programs open, 16gb is usually enough. Use task manager to check if it's a ram problem (see if ram usage skyrockets and paginated file starts being used when saving) or another thing.

Exeterchiefs2023 commented 1 year ago

my ram usage is sometimes high but also disk usage is my data d drive is a HDD drive where as my other drive is SSD but that drive has less memory so i had to move the install to data d drive which is where my games are installed.

i am wondering if its worth getting tad more ram to improve performance but want to find out what type of ram and if my pc can have more in it without needing to replace other stuff.

I do not know too much about replacing parts etc I know that some things are not compatible and replacing certain parts makes pc generate more heat and needs good cooling system to stop overheating etc. I would be worried i mess things up.

Exeterchiefs2023 commented 1 year ago

i did not have many programs running when this occured

Exeterchiefs2023 commented 1 year ago

It's using my hard disk drive and usage is quite high now mouse not working keyboard still lit up however grrr this is infuriating all I told it to do is save some files that I imported to mod pack from thrones of britannia for beards and hair only.Started mucking about again all my games are installed on my hard disk drive my SSD does not have the room for them On 14 Apr 2023 19:38, Frodo45127 @.***> wrote: mmmm that sounds like you're running out of ram. When RPFM saves a pack, it loads it to ram, then writes in to disk. If your pack is 4gb, it'll you'll see a 4gb ram usage the first time you save said pack, for example. If you're close to your ram limit, windows will do windows things, and start using your disk as ram, which slows the whole system down to a crawl and sometimes freeze it. When saving, check your ram usage in task manager and see if its that.

—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you authored the thread.Message ID: @.***>

Frodo45127 commented 1 year ago

if you're saving to an hard disk, that may also be the cause. HDD has a very, VERY low write speed, so writing a 500mb pack will take 8-10 seconds, while on an ssd it'll take 1. And while that happens, windows tends to become non responsive.

Exeterchiefs2023 commented 1 year ago

Bugger so what do you recommend I do get myself another SSD drive or something to then install the games on another drive will be such a pain to transfer them all as all the ones I have on there is on steamOn 14 Apr 2023 23:52, Frodo45127 @.***> wrote: if you're saving to an hard disk, that may also be the cause. HDD has a very, VERY low write speed, so writing a 500mb pack will take 8-10 seconds, while on an ssd it'll take 1. And while that happens, windows tends to become non responsive.

—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you authored the thread.Message ID: @.***>

Frodo45127 commented 1 year ago

try saving the pack to the ssd first. Just the pack. If the situation improves.... it's the hdd. In that case.... either get another ssd, or make place in the one you have.

Exeterchiefs2023 commented 1 year ago

Ok thanks I try that save the pack on the SSD drive next time I'm looking into purchasing a new SSD drive but not sure if it's got space for another one and many things like windows software is on both drives.HDD has most memory about 931gb I think has 180 something free space left I think thanks for the help.Just frustrating why this kept happening think any mod I make have to save on SSD drive and then on HDD once it's completed On 15 Apr 2023 00:19, Frodo45127 @.***> wrote: try saving the pack to the ssd first. Just the pack. If the situation improves.... it's the hdd. In that case.... either get another ssd, or make place in the one you have.

—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you authored the thread.Message ID: @.***>

Exeterchiefs2023 commented 1 year ago

I cut and pasted the pack into my SSD drive and just importing some of the armours and textures to the pack click save and it's taking a while still and now I cannot use mouse or keyboard so looks like I need more ram

Exeterchiefs2023 commented 1 year ago

Would it be a good idea to improve ram first to see if that helps?

Frodo45127 commented 1 year ago

No, first you need to identify the problem, because even with 8gb of ram rpfm runs fine most of the time. Open task manager and see if your ram usage skyrockets when saving, and windows starts paginating, or if the hdd activity goes to 100%. Also, how big is the saved pack?

Exeterchiefs2023 commented 1 year ago

When importing the files to the pack i imported all armour's to tunics and textures from classical empires pack and two models pack of ancient empires forgotten size of the pack.Kept getting issues and then they got corrupted so had to keep starting over so what I done this time is started new pack and just import stuff my units are using and leave everything else be I did import all the skins,beards and hair files textures etc got patterns and shields done.Going through my units and just importing stuff they use my usage is at 40 something percent when the pack is open.When I had other packs open its bit less but notice my ram usage is higher while pack file manager is running if it's not open usage is low same for disk CPU etcOn 16 Apr 2023 22:32, Frodo45127 @.***> wrote: No, first you need to identify the problem, because even with 8gb of ram rpfm runs fine most of the time. Open task manager and see if your ram usage skyrockets when saving, and windows starts paginating, or if the hdd activity goes to 100%. Also, how big is the saved pack?

—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you authored the thread.Message ID: @.***>

Frodo45127 commented 1 year ago

" i imported all armour's to tunics and textures from classical empires pack and two models pack of ancient empires forgotten size of the pack" This doesn't tell me the pack size. I mean in MB. If what you want to do is a submod, I recommend you just setting those packs you import from as dependencies of yours, instead of importing all of them in.

Exeterchiefs2023 commented 1 year ago

Not sure what the file size is but the packs that no longer worked were over 7gb I thinkOn 16 Apr 2023 22:57, Frodo45127 @.***> wrote: " i imported all armour's to tunics and textures from classical empires pack and two models pack of ancient empires forgotten size of the pack" This doesn't tell me the pack size. I mean in MB. If what you want to do is a submod, I recommend you just setting those packs you import from as dependencies of yours, instead of importing all of them in.

—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you authored the thread.Message ID: @.***>

Frodo45127 commented 1 year ago

If it's over 7gb it's not rare it hangs the pc then. RPFM does some heavy lifting to save packs as fast as possible, using all available resources (ram and cpu), but when the pack is so big that sometimes causes hangs. There's little it can do, other than making pack-saving risky and long (taking a lot more time to save and risking that, if an error is detected when reading the pack from disk, the whole thing becomes corrupted). That's way too big. I recommend you either doing a submod, or splitting the pack somehow.

Exeterchiefs2023 commented 1 year ago

I'm making a barbarians unit pack which is a submod but I started a new pack under different name and only importing stuff my units are using and leave everything else the mod uses out so not to make a submod pack too big.Plus eventually when I managed to make the pack it will be incorporated into main mod classical empires by the modder Garviel loken On 16 Apr 2023 23:39, Frodo45127 @.***> wrote: If it's over 7gb it's not rare it hangs the pc then. RPFM does some heavy lifting to save packs as fast as possible, but when the pack is so big... there's little it can do. That's way too big. I recommend you either doing a submod, or splitting the pack somehow.

—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you authored the thread.Message ID: @.***>

gdwitt commented 1 year ago

This sounds like my issue. I am trying to run Ancient Empires. If I load their model pack into RPFM, everything seems fine at first. But every 10-15 minutes, my pc freezes up and I can't do anything with keyboard and mouse for 3 mintues. I checked task manager it told me that my memory was at 99% and that RPFM was using 6.7 GB of ram. I get this with the Vanilla Attila Models3 pack as well.

Exeterchiefs2023 commented 1 year ago

Starting to think could it be rusted pfm that's the problem I ordered more ram for my system to help improve performance especially when using rusted pfm.My ram usage kept sky rocketing and then it would freeze and take ages to load stuff just so I could view it.Is there any other tool I could use similar to rusted pfm that's fairly reliable and stable?Been trying to do a unit pack for ancient empires but keep having issues and one I done is not working at all may have to start from scratch again for like 7th time On 25 Apr 2023 00:31, Geoff @.***> wrote: This sounds like my issue. I am trying to run Ancient Empires. If I load their model pack into RPFM, everything seems fine at first. But every 10-15 minutes, my pc freezes up and I can't do anything with keyboard and mouse for 3 mintues. I checked task manager it told me that my memory was at 99% and that RPFM was using 6.7 GB of ram. I get this with the Vanilla Attila Models3 pack as well.

—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you authored the thread.Message ID: @.***>

Frodo45127 commented 1 year ago

The problem is that you are making a pack way too big, and when saving or autosaving (every few minutes), RPFM has to load it to ram somehow and it fills up the ram. As I said, the solution is not really more ram, but rather to split the pack. Also, if you're making a submod, you're not supposed to import the parent mod files. You are supposed to add said mod as dependency of your own and make sure your mod ONLY CONTAINS YOUR FILES, not the ones from the parent mod.

As for model3, you're not supposed to edit any vanilla pack either. That's how people end up breaking their games. You're supposed to work only with mod/movie packs.

Give this a read to learn more about submodding: https://tw-modding.com/index.php/Tutorial:Submodding

Exeterchiefs2023 commented 1 year ago

I never touch vanilla model files I leave them be and I tried to stop my mod being too big but needed alot of files for skins to tunics etc.I do not know how to split the mod up either and it don't work anyway so I have to start from scratch again.But even just viewing other mods for instance ram usage still high when using Rusted PFMOn 25 Apr 2023 03:33, Frodo45127 @.***> wrote: The problem is that you are making a pack way too big, and when saving or autosaving (every few minutes), RPFM has to load it to ram somehow and it fills up the ram. As I said, the solution is not really more ram, but rather to split the pack. Also, if you're making a submod, you're not supposed to import the parent mod files. You are supposed to add said mod as dependency of your own and make sure your mod ONLY CONTAINS YOUR FILES, not the ones from the parent mod. As for model3, you're not supposed to edit any vanilla pack either. That's how people end up breaking their games. You're supposed to work only with mod/movie packs. Give this a read to learn more about submodding: https://tw-modding.com/index.php/Tutorial:Submodding

—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you authored the thread.Message ID: @.***>

Exeterchiefs2023 commented 1 year ago

What I was making was just a unit pack On 25 Apr 2023 03:33, Frodo45127 @.***> wrote: The problem is that you are making a pack way too big, and when saving or autosaving (every few minutes), RPFM has to load it to ram somehow and it fills up the ram. As I said, the solution is not really more ram, but rather to split the pack. Also, if you're making a submod, you're not supposed to import the parent mod files. You are supposed to add said mod as dependency of your own and make sure your mod ONLY CONTAINS YOUR FILES, not the ones from the parent mod. As for model3, you're not supposed to edit any vanilla pack either. That's how people end up breaking their games. You're supposed to work only with mod/movie packs. Give this a read to learn more about submodding: https://tw-modding.com/index.php/Tutorial:Submodding

—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you authored the thread.Message ID: @.***>

Frodo45127 commented 1 year ago

Ok, if you just want to make a unit pack, first:

Exeterchiefs2023 commented 1 year ago

I have latest version I tried to import only stuff I used some of my units used vanilla weapons for instance I thought I had to have the textures etc for them in the pack.As one time when I had a pack that was working I made an double handed axe unit and weapon they had was missing despite using correct attach point in the variant editor.And selecting to say they carried an axeOn 25 Apr 2023 08:03, Frodo45127 @.***> wrote: Ok, if you just want to make a unit pack, first: Make sure you really need to include those files. If you're not going to modify them in any way, shape of form, you do not need to include them. Even if your units use them. For reference, most unit packs that I've come across don't ever reach 1gb, even with custom models and textures.A mod can be formed of multiple packs. You do not need all the files in the same pack. This is what I mean with splitting the pack. Just add the files you need to a pack until it reaches... don't know, 1 or 2 gb, then leave that pack as is and use another for the rest of the files. And if you fill it, leave it as is and another new pack.Also, a bit of an extra, but make sure you have RPFM up to date. Some earlier versions of 4.0 had bugs in the autosave system that caused them to try and autosave the vanilla files (this is what may cause the ram usage spikes when viewing model3.pack you mentioned).And regarding high ram usage, define "high". Usually, it's 1gb (for dependencies cache, diagnostics,...) or so + the size of your pack. RPFM 4.0 has reduced a lot of ram usage compared to 3.0 (not sure what version you have), but I cannot do miracles.

—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you authored the thread.Message ID: @.***>

Exeterchiefs2023 commented 1 year ago

The pack would also require the textures and various files for the units visuals otherwise they are bugged and stuff missing just like other unit packs I done for other mods like age of justinian mod On 25 Apr 2023 08:03, Frodo45127 @.***> wrote: Ok, if you just want to make a unit pack, first: Make sure you really need to include those files. If you're not going to modify them in any way, shape of form, you do not need to include them. Even if your units use them. For reference, most unit packs that I've come across don't ever reach 1gb, even with custom models and textures.A mod can be formed of multiple packs. You do not need all the files in the same pack. This is what I mean with splitting the pack. Just add the files you need to a pack until it reaches... don't know, 1 or 2 gb, then leave that pack as is and use another for the rest of the files. And if you fill it, leave it as is and another new pack.Also, a bit of an extra, but make sure you have RPFM up to date. Some earlier versions of 4.0 had bugs in the autosave system that caused them to try and autosave the vanilla files (this is what may cause the ram usage spikes when viewing model3.pack you mentioned).And regarding high ram usage, define "high". Usually, it's 1gb (for dependencies cache, diagnostics,...) or so + the size of your pack. RPFM 4.0 has reduced a lot of ram usage compared to 3.0 (not sure what version you have), but I cannot do miracles.

—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you authored the thread.Message ID: @.***>

gdwitt commented 1 year ago

I've been using RPFM almost since the beginning. Thank you for your continued hard work. I think the last release might have a problem. For the cases I presented above, I wasn't modding the files. I was opening them as a reference for my submod. It appears that I can't do this with RPFM anymore and must revert to PFM to view larger files since RPFM is running some kind of script that disables my pc every few minutes.

Frodo45127 commented 1 year ago

Ok, I seemed to miss that there were two people talking here (though both were the same user). I'll re-read later this whole thread when I feel a bit less ill, because it seems I misunderstood and mixed the messages.

Regarding the last update, the autosave system was fixed (was broken before due to a few bugs). It triggers on mods, movie packs.... or anything if you have the "Allow edition of CA Packs" setting enabled, which is disabled by default. I'll probably add a more restrictive check for it on a future update (restricting it so it doesn't trigger on vanilla packs), but you can try disabling that setting.

Also, you can try using the dependencies system marking on your submod ancient empires as dependency. That way it'll be loaded as part of the parent dependencies and you can see and import files without having to open it on a separate instance.

Exeterchiefs2023 commented 1 year ago

I only have one account im separate user to other person lol I never touch vanilla files but viewed them as reference before and ram usage went up I not sure what you mean by dependencies I am not too familiar with that term sorry

Sent from Mailhttps://go.microsoft.com/fwlink/?LinkId=550986 for Windows

From: @.> Sent: 25 April 2023 17:23 To: @.> Cc: @.>; @.> Subject: Re: [Frodo45127/rpfm] Cannot use mouse or keyboard after clicking to save imported files to mod pack (Issue #249)

Ok, I seemed to miss that there were two people talking here (though both were the same user). I'll re-read later this whole thread when I feel a bit less ill, because it seems I misunderstood and mixed the messages.

Regarding the last update, the autosave system was fixed (was broken before due to a few bugs). It triggers on mods, movie packs.... or anything if you have the "Allow edition of CA Packs" setting enabled, which is disabled by default. I'll probably add a more restrictive check for it on a future update (restricting it so it doesn't trigger on vanilla packs), but you can try disabling that setting.

Also, you can try using the dependencies system marking on your submod ancient empires as dependency. That way it'll be loaded as part of the parent dependencies and you can see and import files without having to open it on a separate instance.

— Reply to this email directly, view it on GitHubhttps://github.com/Frodo45127/rpfm/issues/249#issuecomment-1522079319, or unsubscribehttps://github.com/notifications/unsubscribe-auth/A7FWMVUZGXHEYR7FJJGFL7DXC726JANCNFSM6AAAAAAW6PNDWM. You are receiving this because you authored the thread.Message ID: @.***>

gdwitt commented 1 year ago

On lunch break here. I'm glad to see your response Frodo. How do I "marking on your submod ancient empires as dependency"? FYI, RPFM settings, I have Autosave Interval at 10 min and have all debug settings checked but for Unit Editor. I wonder if I should turn off the debug settings for rigid model editor and possibly other debugs? Does it run a check every time it autosaves? Should that be turned off? The debugs are a great feature of RPFM, but I haven't used them on model packs until recently.

Frodo45127 commented 1 year ago

Ok, both of you try 4.0.17 and see if it works better. The bug that caused autosaves to get trigger on vanilla packs should be fixed now, so it should no longer try to autosave them causing ram saturation.

Exeterchiefs2023 commented 1 year ago

Ok I managed to make a new pack recently and do not have half as much as what my previous pack had and it loads and saves bit faster as I have more ram in my pc nowOn 27 Apr 2023 17:10, Frodo45127 @.***> wrote: Ok, both of you try 4.0.17 and see if it works better. The bug that caused autosaves to get trigger on vanilla packs should be fixed now, so it should no longer try to autosave them causing ram saturation.

—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you authored the thread.Message ID: @.***>

gdwitt commented 1 year ago

Thanks Frodo! I will test this with debug on and off.

Frodo45127 commented 1 year ago

Update to 4.0.18 and check again. It contains more fixes for the autosave system that caused autosaves to trigger when they shouldn't under windows.

gdwitt commented 1 year ago

The update seemed to have fixed the issue. I don't have the RAM and CPU spikes anymore with model packs open. Great job!

Exeterchiefs2023 commented 1 year ago

I not had any issue either my ram can handle opening big files can still take bit of time with ancient empires model files however not as much as before and my new ram is alot better than old ones On 23 May 2023 22:24, Geoff @.***> wrote: The update seemed to have fixed the issue. I don't have the RAM and CPU spikes anymore with model packs open. Great job!

—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you authored the thread.Message ID: @.***>

Exeterchiefs2023 commented 1 year ago

Anyone know how I can un install rusted pfm and reinstall it on my new ssd drive as one its stored on currently is running low thanks and is there an easy to follow guide on how to do this

Sent from Mailhttps://go.microsoft.com/fwlink/?LinkId=550986 for Windows

From: @.> Sent: 23 May 2023 22:24 To: @.> Cc: @.>; @.> Subject: Re: [Frodo45127/rpfm] Cannot use mouse or keyboard after clicking to save imported files to mod pack (Issue #249)

The update seemed to have fixed the issue. I don't have the RAM and CPU spikes anymore with model packs open. Great job!

— Reply to this email directly, view it on GitHubhttps://github.com/Frodo45127/rpfm/issues/249#issuecomment-1560141138, or unsubscribehttps://github.com/notifications/unsubscribe-auth/A7FWMVVKX44YSB2YLGHA2BTXHUTJHANCNFSM6AAAAAAW6PNDWM. You are receiving this because you authored the thread.Message ID: @.***>