chrismaltby / gb-studio

A quick and easy to use drag and drop retro game creator for your favourite handheld video game system
https://www.gbstudio.dev
MIT License
8.51k stars 469 forks source link

Project without Build Cache takes 50 Minutes to Compile #1563

Closed Brushman97 closed 3 weeks ago

Brushman97 commented 3 weeks ago

The Issue

When I'm compiling my game, it doesn't take too long for it to build. However, that's an entirely different story if there's no build cache. image As seen in the screenshot, it took 50 minutes for my project to compile. I've never kept track of the exact build time in the past, but I think it usually takes 20-30 minutes when the build cache has been cleared, something that I guess my computer does sometimes after turning it off.

This has been a problem that I've been having maybe ever since I've been using GB Studio. It's most likely my laptop (a Microsoft Surface Go 2) that's the culprit. Speaking of which, here are the specs, in case it's needed:

Processor Intel(R) Core(TM) m3-8100Y CPU @ 1.10GHz 1.61 GHz Installed RAM 8.00 GB (7.87 GB usable) System type 64-bit operating system, x64-based processor Pen and touch Pen and touch support with 10 touch points

patrickmollohan commented 3 weeks ago

Based on what I'm seeing, I don't think it's the specs of your computer. According to benchmarks (which may only be an estimate and might not show the full picture), your CPU should be about double the speed of mine, and mine usually only takes about 6-10 minutes to build a game after clearing the build cache (even less so after the recent update that skips compiling unused scene types).

My guess is (without knowing more), if you are using too much RAM, that Windows is using the page file as temporary RAM stored on your hard drive, which would be very slow. If your hard drive is fragmented on top of this, it could be causing a lot of thrashing, resulting in really bad performance. Perhaps try closing out any other programs/files you may have open when building, and if you aren't already, run the defragger built into Windows.

Brushman97 commented 3 weeks ago

Based on what I'm seeing, I don't think it's the specs of your computer. According to benchmarks (which may only be an estimate and might not show the full picture), your CPU should be about double the speed of mine, and mine usually only takes about 6-10 minutes to build a game after clearing the build cache (even less so after the recent update that skips compiling unused scene types).

My guess is (without knowing more), if you are using too much RAM, that Windows is using the page file as temporary RAM stored on your hard drive, which would be very slow. If your hard drive is fragmented on top of this, it could be causing a lot of thrashing, resulting in really bad performance. Perhaps try closing out any other programs/files you may have open when building, and if you aren't already, run the defragger built into Windows.

I don't think I'll be able to handle having only one program open at a time. Also, I saw that defragging decreases the lifespan of your device. I don't know what else can be done to fix this.

patrickmollohan commented 3 weeks ago

I don't think I'll be able to handle having only one program open at a time.

Yeah, then I think honestly that's the source of the problem, in which case there's nothing that can be done with GB Studio that could help as far as I'm aware of. After checking, it appears your RAM is soldered on, so unless you can handle BGA soldering, upgrading your RAM won't be a possibility. Your only options are to either close out of the other programs or use a different computer.

Also, I saw that defragging decreases the lifespan of your device.

Yeah, I didn't think to check your device. My apologies. Running the built in defragger won't damage the device (it will TRIM it rather than defrag) but it won't provide the benefits of defragging.

Brushman97 commented 3 weeks ago

I don't think I'll be able to handle having only one program open at a time.

Yeah, then I think honestly that's the source of the problem, in which case there's nothing that can be done with GB Studio that could help as far as I'm aware of. After checking, it appears your RAM is soldered on, so unless you can handle BGA soldering, upgrading your RAM won't be a possibility. Your only options are to either close out of the other programs or use a different computer.

Also, I saw that defragging decreases the lifespan of your device.

Yeah, I didn't think to check your device. My apologies. Running the built in defragger won't damage the device (it will TRIM it rather than defrag) but it won't provide the benefits of defragging.

Oh, I see. Also, I don't usually have that many applications running in the background. Usually, I only have around 5. One other thing, would defragging provide any benefits for my device at all, or is there no point in doing so?

chrismaltby commented 3 weeks ago

Oof! I don't know how you put up with those build times! I find it frustrating when it's over 1 minute!!

  1. Your Surface Go 2 has an SSD drive so defragging will pretty much do nothing, I wouldn't bother if I was you

  2. It would be good to confirm is it something specific about your project or just GB Studio in general. As a benchmark could you try creating a brand new project in 4.1.0 (using the color template), empty your build cache and without making any changes at all just build the project and let us know how long it took.

For reference these are my build times on my development machines:

I also run Windows in a VM on my Synology Network Drive for testing

Given the specs on your Surface it's expect it to be around the same speed as my Windows VM but if it's very different that could give some indication of what's happening.

  1. Is there a specific step that seems to take a long time? If the build log is pausing at a specific step just let me know what the last line of text displayed was. In the build log everything up to "Validating Build Files" is GB Studio generating the source files for your game, and there's potential to optimise the steps before this to maybe improve on RAM usage. If its mostly the steps after this, at that point your project is being compiled by GBDK and there's not a huge amount of improvement I could personally make, I already run the compilation in parallel depending on the amount of CPU cores you have available but your Surface only has a 2-core CPU.

  2. If you want me to check out your specific project you could email it to
    **my first name**.**my last name**@gmail.com I might not be able to check immediately but I can confirm fresh build times on my hardware. I always like getting examples of slower projects too as it helps me benchmark for optimising.

Brushman97 commented 3 weeks ago

Oof! I don't know how you put up with those build times! I find it frustrating when it's over 1 minute!!

1. Your Surface Go 2 has an SSD drive so defragging will pretty much do nothing, I wouldn't bother if I was you

2. It would be good to confirm is it something specific about your project or just GB Studio in general. As a benchmark could you try creating a brand new project in 4.1.0 (using the color template), empty your build cache and without making any changes at all just build the project and let us know how long it took.

For reference these are my build times on my development machines:

* _Mac mini 2020 - M1, 16GB RAM_ : 47.36s

* _MacBook Pro 2023 - M3 Pro, 18GB RAM_ : 37.36s

I also run Windows in a VM on my Synology Network Drive for testing

* _Synology DS1621+ - AMD Ryzen V1500B (technically 4 cores, but I've limited the VM to use 2), 16GB RAM_ : 6m 25s

Given the specs on your Surface it's expect it to be around the same speed as my Windows VM but if it's very different that could give some indication of what's happening.

3. Is there a specific step that seems to take a long time? If the build log is pausing at a specific step just let me know what the last line of text displayed was. In the build log everything up to "Validating Build Files" is GB Studio generating the source files for your game, and there's potential to optimise the steps before this to maybe improve on RAM usage. If its mostly the steps after this, at that point your project is being compiled by GBDK and there's not a huge amount of improvement I could personally make, I already run the compilation in parallel depending on the amount of CPU cores you have available but your Surface only has a 2-core CPU.

4. If you want me to check out your specific project you could email it to
   `**my first name**`.`**my last name**`@gmail.com
   I might not be able to check immediately but I can confirm fresh build times on my hardware. I always like getting examples of slower projects too as it helps me benchmark for optimising.

I had someone test out my project on their end, and they said it took them only 6 minutes for it to compile, so we know it's not my game (but rather something's up with my device possibly). image image

The thing it gets stuck on is presumably the "platform.c", but it used to stop on the "topdown.c" part and stay on that for the majority of the compilation time, though now there's a couple more things after it since I've given my project more plugins. image (Earlier) (My project uses an alpha version of a 4.0.0-updated Platformer Plus, so that's what the playerstates.c, playerstatesb.c, and playerstatesc.c scripts are)

image (More Recently)

I'll do that experiment with the new project later today, though should it be an empty one, or the sample project?

chrismaltby commented 3 weeks ago

Ah I'd just tried that new alpha of Platformer Plus today, it took a VERY long time to compile even on my M3 Macbook Pro. From a fresh project, just adding that version of the plugin increased build times by 6x. Given this isn't a problem with the core of GB Studio I'm going to close this issue.

By the way when it was getting stuck on topdown.c previously I think it was likely actually getting stuck on platform.c. Because you have a 2 core processor the compiler will work on two files at once, unfortunately because that build log doesn't give feedback on when a file finished building it can look like it's getting stuck on a different file than it actually is (so in your screenshot while one CPU got stuck on platform.c the other worked through the next 7 files ). I might have a look at making the build log a little clearer about what's happening :-)

Brushman97 commented 3 weeks ago

Ah I'd just tried that new alpha of Platformer Plus today, it took a VERY long time to compile even on my M3 Macbook Pro. From a fresh project, just adding that version of the plugin increased build times by 6x. Given this isn't a problem with the core of GB Studio I'm going to close this issue.

By the way when it was getting stuck on topdown.c previously I think it was likely actually getting stuck on platform.c. Because you have a 2 core processor the compiler will work on two files at once, unfortunately because that build log doesn't give feedback on when a file finished building it can look like it's getting stuck on a different file than it actually is (so in your screenshot while one CPU got stuck on platform.c the other worked through the next 7 files ). I might have a look at making the build log a little clearer about what's happening :-)

Gotcha. The build log idea sounds like it would be a useful feature.