Corrupted zip (jar) file and other issues in incremental builds

[artem-zinnatullin]

We're getting few different issues during incremental builds on developer machines where particularly buck kill helps.

Issue 1: Corrupted zip file of a jar from another module

./buckw build //instant:bin_devDebug
Creating action graph: finished in 0.5 sec
Downloaded 113 artifacts, 15.56 Mbytes, 13.2% cache miss
Building: finished in 41.0 sec (100%) 2765/3373 jobs, 174 updated
  Total time: 41.8 sec
Command failed with exit code 1.
stderr: /Users/user/project/mymodule/main/java/com/mymodule/MyClass.java:5: error: cannot access com.mymodule2.SomeClass
import com.mymodule2.SomeClass;
                             ^
  bad class file: /Users/user/project/buck-out/gen/mymodule2/lib__src_main__output/src_main.jar(com/mymodule2/SomeClass.class)
    unable to access file: corrupted zip file
    Please remove or make sure it appears in the correct subdirectory of the classpath.
Errors: 1. Warnings: 0.

    When running <javac>.
    When building rule //mymodule:src_release.

It can happen with any module combination. jar file is actually present in expected path and seems to be valid (javap dumps of classes are fine too).

Running same build command will fail again.

buckw kill fixes it.

---

Issue 2: Java code can't reference Kotlin class from another module

Command failed with exit code 1.
instant-scoop/scoop-activity/src/main/java/com/lyft/android/scoop/app/activity/ScoopActivity.kt:8:27: error: unresolved reference: IRxActivityBinder
import me.lyft.android.rx.IRxActivityBinder
                          ^
instant-scoop/scoop-activity/src/main/java/com/lyft/android/scoop/app/activity/ScoopActivity.kt:9:27: error: unresolved reference: RxActivityBinder
import me.lyft.android.rx.RxActivityBinder
                          ^
instant-scoop/scoop-activity/src/main/java/com/lyft/android/scoop/app/activity/ScoopActivity.kt:13:17: error: unresolved reference: IRxActivityBinder
    val binder: IRxActivityBinder = RxActivityBinder()
                ^
instant-scoop/scoop-activity/src/main/java/com/lyft/android/scoop/app/activity/ScoopActivity.kt:13:37: error: unresolved reference: RxActivityBinder
    val binder: IRxActivityBinder = RxActivityBinder()
                                    ^

    When running <kotlinc -d /Users/user/src/instant-android/buck-out/bin/instant-scoop/scoop-activity/lib__src_release__scratch/classes -classpath /Users/user/Library/Android/sdk/platforms/android-28/android.jar:/Users/user/Library/Android/sdk/platforms/android-28/optional/android.test.base.jar:/Users/user/Library/Android/sdk/platforms/android-28/optional/android.test.mock.jar:/Users/user/Library/Android/sdk/

IRxActivityBinder was a new class, added to to the module. Module was already in dependency list of target module.

Running the build again will fail. buck kill fixes the issue.

Looks like something is off about the Buck daemon 🤔

---

Environment:

• Buck https://github.com/facebook/buck/commit/f252d4859e9a25511d2a69ef23b7c43375e09454
• macOS 10.13.6
• Oracle JDK/JRE 1.8.0_181-b13
• Watchman 4.9.0_2

[raviagarwal7]

do you have action graph caching enabled? if so try skipping the action graph cache and see if it works - https://buckbuild.com/files-and-dirs/buckconfig.html#client.skip-action-graph-cache

[artem-zinnatullin]

good call @raviagarwal7, I'll try 👍

[artem-zinnatullin]

We're still seeing it with action graph cache disabled :(

[artem-zinnatullin]

Maybe anyone from Buck team can shine a light on this issue? It seems that daemon corrupts its in-memory state

[artem-zinnatullin]

We're still getting this ~4 times per developer per day, had to add automatic workaround and retry but unclear what exactly goes wrong :(

[Blisse]

I want to say that we're also experiencing this issue and it's really crippling to our developer velocity. Essentially to the point that we have a git hook to run ./buckw kill after changing anything.

Any suggestions to help us diagnose this issue would be appreciated.

[jbarr21]

We're seeing this as well. These occurred mostly after rebases. One particular reproducible instance was pulling master and rebasing to pickup a new inner enum defined in a Java scope consumed by Kotlin