Static archiver call does not use command file

[JuergenHaug]

The "old native build" fails to create a static archive due the 32k command line limit.

Program 'arm-none-eabi-ar.exe' failed to run: The filename or extension is too longAt line:1 char:1

The invocation of the compiler and linker use command files, can that be activated for the archiver too ? I did some digging in the code and using the GccOptionsFileArgsWriter for ArStaticLibraryArchiver seems to do the trick.

Unfortunately we are cross-compiling a C application. My impression is, that is it is too early to migrate to the new native build for this case.

Expected Behavior

Call to static archiver of gcc should use command file.

Current Behavior

Arguments are passed via command line and will eventually hit the 32k limit.

Context

Creating a static archive from ~25 object files. The files are passed using absolute paths, which makes the command line call too large.

Your Environment

cross compiling c application using arm gcc, gradle 5.5, Win10

[lacasseio]

Thanks for reporting this issue and I agree the ar could use a command file. The broader question is if every ar tool support command file. I think we can try to make the change and see where the tests may be failing. Would you be willing to help with this? Here are the steps I think we can so to solve this issue:

• Write an integration test in language native for all the language that would generate a 32k command line and make sure it's failing.
• Modify the ArStaticLibraryArchiver constructor to take useCommandFile like his GccCompatibleNativeCompiler sibling. Pass the variable to the superclass constructor instead of false.
• Pass false for useCommandFile in SwiftPlatformToolProvider (we can try to pass true but let's leave Swift outside the scope of this issue).
• Pass useCommandFile variable inside GccPlatformToolProvider.
• Replace ArStaticLibraryArchiver#addOptionsFileArgs method content with GccCompatibleNativeCompiler#addOptionsFileArgs content. We can deal with the code duplication later.
• Run the unit and integration test for the platform-native and language-native: ./gradlew :platformNative:check :languageNative:check --continue.
• Fix any issues that are coming from the fact that we are now using the command file instead of the raw command line.
• Make sure the integration test for the 32k command line is now passing.
• Submit PR :-)

That would be really helpful. I can take care of porting the integration test to the new native plugins.

[jbelle]

Hi, I'd like to help resolve this issue. I implemented an integration test that generates many source files and builds a library with the cpp-library plugin. The command line for the ar static library archiver then exceeds the windows command line length limit and the test fails on windows. With your proposed changes to ArStaticLibraryArchiver the test passes. As far as I can tell from the test results of platform-native and language-native, these changes don't introduce any further issues. Please let me know how I can proceed.

[BigPapa314]

This issue still exists in gradle 7.5.1 ist there any plan to proceed?

[jbelle]

It's not clear whether all static library archivers support command files and how we can make sure not to break existing builds. My pull request got closed after some time due to inactivity and I don't know how to proceed. I guess even some configuration option that allows to enable command file usage would help, for example in GccCommandLineToolConfiguration?

It is possible to modify the command line arguments using CommandLineToolConfiguration#withArguments to work around this issue:

model {
  toolChains {
    withType(GccCompatibleToolChain) {
      eachPlatform {
        // Use a command file for the static library archiver to avoid the windows command line length limit.
        staticLibArchiver.withArguments { args ->
          // We need a different location for the command file for each task because Gradle executes tasks in parallel.
          def tmpDir = java.nio.file.Files.createTempDirectory(file("${buildDir}/tmp").toPath(), "createStaticLibrary")
          // Write the arguments to a command file and modify the command line arguments accordingly.
          new org.gradle.nativeplatform.toolchain.internal.gcc.GccOptionsFileArgsWriter(tmpDir.toFile()).execute(args)
        }
      }
    }
  }
}

[BigPapa314]

You made my day!

Here is my setup of the staticLibArchiver:

// Use a command file for the static library archiver to avoid the windows command line length limit.
staticLibArchiver.withArguments { args ->
  // We need a different location for the command file for each task because Gradle executes tasks in parallel.
  def tmpDir = java.nio.file.Files.createTempDirectory(file("${buildDir}/tmp").toPath(), "createStaticLibrary")

  // Write the arguments to a command file and modify the command line arguments accordingly.
  new org.gradle.nativeplatform.toolchain.internal.gcc.GccOptionsFileArgsWriter(tmpDir.toFile()).execute(args)

  // arm-none-eabi-gcc-ar needs at least one command line argument
  def withDummyArgument = ['-v'] + args
  args.clear()
  args.addAll(withDummyArgument)
}