Unnecessary rebuild all for any core lib file changed

[PaulStoffregen]

When any file within the core library changes, "Build options changed, rebuilding all" prints and all files are unnecessarily recompiled.

I tested just now with arduino-PR-beta1.9-BUILD-22 on Linux 64 bit (Ubuntu 14.04), with Arduino/Genuino Zero Programming Port selected. I have preferences set to show verbose info while compiling. The problem seems to happen with any code, but this is the program I used just now:

void setup() {
  Serial.begin(9600);
}
void loop() {
  static int count=0;
  Serial.print("test ");
  Serial.print(count++);
  delay(100);
}

To produce the problem, I first clicked Verify. Then I run this in a terminal window:

touch ~/.arduino15/packages/arduino/hardware/samd/1.6.16/cores/arduino/Print.cpp

When I click Verify again after this command, the result is "Build options changed, rebuilding all", and all files in the core library are recompiled, even though only Print.cpp has changed.

[facchinm]

We know it and do on purpose to solve a broader problem: with normal caching infrastructure, it you change anything in platform.txt or boards.txt the change is not applied unless you change the target board. Do you suggest we could only apply the this approach to non-source files, while continue trusting .d files for source dependencies?

[PaulStoffregen]

Yes. Definitely everything has to be recompiled if platform.txt or boards.txt change. If any of the settings that make up the fully qualified board name change, of course everything needs to be recompiled.

But for changes only in the source code, whether or not to recompile a particular source file is the whole purpose of those .d files. I believe they are very trustworthy, if you can verify the .d file and .o file are newer than the timestamps on all the files listed within the .d file. Arduino 1.6.5 (the last before switching to arduino-builder) did this quite well, even though it didn't have many of the features arduino-builder has now.

[pedzed]

I'm facing this issue as well. Compiling on macOS with VSCode's Arduino extension takes very long this way. Not sure if that has anything to do with it?

[facchinm]

@pedzed I'm not sure that VSCode extension uses arduino-builder under the hood (I'm pretty sure it doesn't) so you'd better report your issue to their repo (https://github.com/Microsoft/vscode-arduino)

[pedzed]

I'm not sure that VSCode extension uses arduino-builder under the hood (I'm pretty sure it doesn't)

I actually do think it does (based on the terminal with Arduino that opens up in the background).

~Anyway, it seems to be solved in the newer Arduino version (1.8.5). I was using 1.8.3. I should have tried this a long time ago, but I remember not seeing it in the change log which was the reason I did not bother updating.~

Really? I am getting it again. I am not even touching the code, just reuploading.

[facchinm]

@pedzed Arduino IDE != arduino-builder (although the IDE uses the builder internally) . If you launch multiple instances of the IDE you'll likely lose any cache produced by the previous run (if you could paste here the commandline that gets executed I can tell you what is going on).

@PaulStoffregen https://github.com/arduino/arduino-builder/pull/250/commits/9b0a81a95695592857c504fba238b3d48323cf46 should target this issue, since it wipes only if platform.txt or boards.txt has changed. Beta 1.9 already contains the fix if you want to try it out :wink:

[stevstrong]

I am having the same problem. V1.8.5, I am using an external code editor (Metapad). Each time I just press the save button in Metapad to save the INO file, the IDE does rebuild all core files:

Build options changed, rebuilding all
Detecting libraries used...

EDIT: only the INO file is opened in Metapad. Is there any way to solve this issue within this version? I don't want to change to a new (beta) version.

[artua]

I have the same issues running esp8266 board. It is always recompile all, but no boards or platforms file changed. 1.9 beta working fine. Any chance to have the same fix for 1.8.6?

[pedzed]

I switched from Arduino Builder to PlatformIO (in combination with the lovely VSCode editor). Definitely recommended. It solved all issues for me.

[PaulStoffregen]

fwiw, I've been using 1.8.5 with the arduino-builder from 1.9, and the tools-builder stuff it needs. I did have to add another rule into platform.txt for the new preprocessor. With platform.txt updated, it works very well. The parallel build is especially nice when regularly switching between boards.

With 1.9 so far on the horizon, I too hope a stable 1.8.6 release can be made with the updated arduino-builder. There is a huge community of Arduino users who aren't going to use beta software, and are needlessly suffering slow compile times. I know the dev effort is going into 1.9, but can't a little work go into giving so many ordinary non-beta users these very stable improvements while the rest of the new stuff remains in 1.9?

[guino]

@PaulStoffregen care to share what you changed in platform.txt ? I am dealing with full recompile every time on 1.8.5 and have no solution right now.

[PaulStoffregen]

@guino - First, try the nightly build, or copy arduino-builder from the nightly into 1.8.5. The latest nightly has these changes (since this commit: https://github.com/arduino/Arduino/commit/ee1967cd530ceb9a1d638875e385157e90d532e8 ), but (probably) doesn't need the platform.txt change.

But to answer your question, with the caveat that this is for Teensy, here is what I added to platform.txt:

## New Preprocessor for Arduino 1.9
tools.arduino-preprocessor.path={runtime.tools.arduino-preprocessor.path}
tools.arduino-preprocessor.cmd.path={path}/arduino-preprocessor
tools.arduino-preprocessor.pattern="{cmd.path}" "{source_file}" "{codecomplete}" -- -std=gnu++14

This is almost certainly not correct for other non-Teensy boards, since we use C++14 and Teensyduino has a significantly different boards.txt and platform.txt. But you asked what I added, so there it is.

[guino]

@PaulStoffregen I tried the latest nightly compile and it worked to my surprise! Now the compiles are way faster without that full rebuild every time. I am just crossing fingers it doesn't break something else. Thanks for the suggestion and information!

[marcmerlin]

Dear arduino core people, you ask me for donations when I download arduino, I donate, and then we have impacting bugs like this one that go unfixed for close to a year. It's fixed in 1.9 beta, except that one has other serious bugs that cost me a lot of lost time. I just figured had 2 days go down the drain with what I thought were ESP32 problems when it was simply all betas of arduino 1.9 that fail in subtle ways with my program when I build on ESP32. Sure enough, going back to 1.8.5 fixes everything, and now I have 2mn compiles for any line changed, due to everything being rebuilt. So I guess I can start hand patching 1.8.5 as @PaulStoffregen posted above, but you should really release a working 1.8.6 please.

[PaulStoffregen]

now I have 2mn compiles for any line changed

Even old computers without SSD storage aren't that slow. Pretty much only 1 thing causes such slow compiles: Windows anti-virus software doing "real time scan".

[marcmerlin]

@PaulStoffregen you are correct for teensy it's not as bad. For ESP32 and ESP8266, the core libraries are quite large in comparison, hence the minute+ rebuild times. You're right that depending on your computer, it can be faster than that. Anyway, I just heard back on the other thread, they have fixed this in arduino 1.8.6 nightly (which is called hourly builds on https://www.arduino.cc/en/Main/Software , downloads as something that gives no clue it's a 1.8.6 pre-release, but in fact it is).

Sure enough 1.8.6 pre release also fixes the spurious rebuild problem. I just hope they release it really soon, it's been quite a while...

[PaulStoffregen]

I too hope for a 1.8.6 release soon, mostly for Windows users suffering from the OneDrive bug.

[PaulStoffregen]

Regarding library sizes, Teensy has substantially more core library code than ESP32. In CPP files, ESP32 has 2576 lines in 13 files. Teensy has 23 CPP files totaling 4238 lines. In C files (which compile much faster), ESP32 has 4024 lines in 17 files. Teensy has 13421 lines in 28 files. Likewise for headers, ESP32 has 3426 lines in 36 files. Teensy has 34560 lines in 59 files. There is simply a lot more code in Teensy's core library.

Running on a high-end Linux system, a full rebuild takes about 6 seconds. Admittedly, Linux is faster. But even slow Windows machines manage a full rebuilt in ~30 seconds.

If you're seeing 2 minute full rebuild times on Windows, you really should try disabling real-time or heuristic anti-virus scanning.

[marcmerlin]

@PaulStoffregen thanks for correcting me on teensy vs ESP32. My recollection of how long it felt,can't argue with your real data :) I do however use linux on a 4 core (8 threads) higher end laptop with a Fast SSD, and even there building my code that uses IRRemote and FastLED, ends up taking about 40 seconds :( From what I can tell the arduino IDE does not do parallel builds though, so I guess it only uses one thread of my CPU. Building with 1.8.6 takes only 30 seconds the first time, and 15 seconds using cached objects, the second time.

[PaulStoffregen]

Testing so far with 1.8.6 looks like this problem is fixed.

Does anyone see a reason to leave this issue open?

[facchinm]

Indeed, closing as fixed by 1.8.6