apply_patch failure - Githubissues

h0tw1r3 commented 1 year ago

When git is selected to apply a patch and there is a config directory in the base of the source being built, the following error occurs:

warning: unable to access 'config': Is a directory
warning: unable to access 'config': Is a directory
fatal: unknown error occurred while reading the configuration files

flavorjones commented 1 year ago

Hi! I'm having trouble reproducing what you're seeing. I've got an existing project (nokogiri). I'm in a docker container where I've removed patch. I create a directory named "config" in the project root. I then run "bundle exec rake compile" which does a bunch of things including applying patches to libxml2 using git apply:

# bundle exec rake compile
mkdir -p tmp/x86_64-linux/nokogiri/3.1.2
cd tmp/x86_64-linux/nokogiri/3.1.2
/usr/local/bin/ruby -I. -r.rake-compiler-siteconf.rb ../../../../ext/nokogiri/extconf.rb 
checking for whether -std=c99 is accepted as CFLAGS... yes
checking for whether -Wno-declaration-after-statement is accepted as CFLAGS... yes
checking for whether -O2 is accepted as CFLAGS... yes
checking for whether -g is accepted as CFLAGS... yes
checking for whether -Winline is accepted as CFLAGS... yes
checking for whether -Wmissing-noreturn is accepted as CFLAGS... yes
Building nokogiri using packaged libraries.
Static linking is enabled.
Cross build is disabled.
Using mini_portile version 2.8.0
checking for iconv... yes
---------- IMPORTANT NOTICE ----------
Building Nokogiri with a packaged version of libxml2-2.10.3.
Configuration options: --host\=x86_64-pc-linux-gnu --enable-static --disable-shared --libdir\=/nokogiri/ports/x86_64-linux/libxml2/2.10.3/lib --with-iconv\=yes --disable-dependency-tracking --without-python --without-readline --with-c14n --with-debug --with-threads --disable-shared --enable-static CFLAGS\=-O2\ -U_FORTIFY_SOURCE\ -g\ -fPIC
The following patches are being applied:
  - 0001-Remove-script-macro-support.patch
  - 0002-Update-entities-to-remove-handling-of-ssi.patch
  - 0003-libxml2.la-is-in-top_builddir.patch
  - 0009-allow-wildcard-namespaces.patch

The Nokogiri maintainers intend to provide timely security updates, but if
this is a concern for you and want to use your OS/distro system library
instead, then abort this installation process and install nokogiri as
instructed at:

  https://nokogiri.org/tutorials/installing_nokogiri.html#installing-using-standard-system-libraries

Note, however, that nokogiri cannot guarantee compatibility with every
version of libxml2 that may be provided by OS/package vendors.

Extracting libxml2-2.10.3.tar.xz into tmp/x86_64-pc-linux-gnu/ports/libxml2/2.10.3... OK
Running git apply with /nokogiri/patches/libxml2/0001-Remove-script-macro-support.patch... OK
Running git apply with /nokogiri/patches/libxml2/0002-Update-entities-to-remove-handling-of-ssi.patch... OK
Running git apply with /nokogiri/patches/libxml2/0003-libxml2.la-is-in-top_builddir.patch... OK
Running git apply with /nokogiri/patches/libxml2/0009-allow-wildcard-namespaces.patch... OK
Running 'configure' for libxml2 2.10.3... OK
Running 'compile' for libxml2 2.10.3...
...

What's different about your setup? How can I reproduce this?

h0tw1r3 commented 1 year ago

I'm wondering if it depends on the version of git (2.37.1) or possibly my local ~/.gitconfig?

flavorjones commented 1 year ago

Lots of possible variables, for sure, which is what makes this challenging to approach as a maintainer. I absolutely believe that the change in #120 fixes your problem, but I'd like to write a test and understand what's going wrong.

A couple of suggestions (choose one) for a reasonable next step:

Can you add a test to the mini_portile suite that reproduces this? There is an examples directory that tries to apply patches to libiconv, can you alter the project directory or test to make this happen?
Or can you set up a new public repo that runs mini_portile and reproduces what you're seeing?

Or let me know if I can help in any other way?

flavorjones commented 1 year ago

Correction: the examples suite isn't exercising patches (but it will shortly! see https://github.com/flavorjones/mini_portile/pull/121), but the unit test suite does, in test/test_cook.rb.

h0tw1r3 commented 1 year ago

@flavorjones haven't had time to dig into tests yet, but the issue came up when I was working on building another gem. https://github.com/winebarrel/ruby-jq/pull/20

flavorjones commented 1 year ago

I still need help reproducing this.

h0tw1r3 commented 1 year ago

@flavorjones commands to reproduce, tested on OSX 12.5 (M1) and Debian 11 x86_64.

wget https://github.com/btolab/ruby-jq/archive/refs/heads/master.tar.gz
tar xzf master.tar.gz
cd ruby-jq-master
bundle exec rake compile

output

mkdir -p tmp/x86_64-linux-gnu/jq_core/2.7.4
cd tmp/x86_64-linux-gnu/jq_core/2.7.4
/usr/bin/ruby2.7 -I. ../../../../ext/extconf.rb
Buildling jq using packaged libraries.
Downloading onigmo-6.2.0.tar.gz (100%) 
Extracting onigmo-6.2.0.tar.gz into tmp/x86_64-linux-gnu/ports/onigmo/6.2.0... OK
Running 'autoreconf' for onigmo 6.2.0... OK
Running 'configure' for onigmo 6.2.0... OK
Running 'compile' for onigmo 6.2.0... OK
Running 'install' for onigmo 6.2.0... OK
Activating onigmo 6.2.0 (from ports/x86_64-linux-gnu/onigmo/6.2.0)...
Downloading cff5336ec71b6fee396a95bb0e4bea365e0cd1e8.tar.gz (100%) 
Extracting cff5336ec71b6fee396a95bb0e4bea365e0cd1e8.tar.gz into tmp/x86_64-linux-gnu/ports/jq/1.6... OK
Running git apply with /home/h0tw1r3/x/ruby-jq-master/ext/jq-onigmo.patch... ERROR, review '/home/h0tw1r3/x/ruby-jq-master/tmp/x86_64-linux-gnu/jq_core/2.7.4/tmp/x86_64-linux-gnu/ports/jq/1.6/patch.log' to see what happened. Last lines are:
========================================================================
warning: unable to access 'config': Is a directory
warning: unable to access 'config': Is a directory
fatal: unknown error occurred while reading the configuration files
========================================================================
*** ../../../../ext/extconf.rb failed ***
Could not create Makefile due to some reason, probably lack of necessary
libraries and/or headers.  Check the mkmf.log file for more details.  You may
need configuration options.

Provided configuration options:
    --with-opt-dir
    --without-opt-dir
    --with-opt-include
    --without-opt-include=${opt-dir}/include
    --with-opt-lib
    --without-opt-lib=${opt-dir}/lib
    --with-make-prog
    --without-make-prog
    --srcdir=../../../../ext
    --curdir
    --ruby=/usr/bin/$(RUBY_BASE_NAME)2.7
    --use-system-libraries
Traceback (most recent call last):
    9: from ../../../../ext/extconf.rb:46:in `<main>'
    8: from /var/lib/gems/2.7.0/gems/mini_portile2-2.8.0/lib/mini_portile2/mini_portile.rb:184:in `cook'
    7: from /var/lib/gems/2.7.0/gems/mini_portile2-2.8.0/lib/mini_portile2/mini_portile.rb:118:in `patch'
    6: from /var/lib/gems/2.7.0/gems/mini_portile2-2.8.0/lib/mini_portile2/mini_portile.rb:118:in `each'
    5: from /var/lib/gems/2.7.0/gems/mini_portile2-2.8.0/lib/mini_portile2/mini_portile.rb:120:in `block in patch'
    4: from /var/lib/gems/2.7.0/gems/mini_portile2-2.8.0/lib/mini_portile2/mini_portile.rb:114:in `apply_patch'
    3: from /var/lib/gems/2.7.0/gems/mini_portile2-2.8.0/lib/mini_portile2/mini_portile.rb:104:in `block in apply_patch'
    2: from /var/lib/gems/2.7.0/gems/mini_portile2-2.8.0/lib/mini_portile2/mini_portile.rb:426:in `execute'
    1: from /var/lib/gems/2.7.0/gems/mini_portile2-2.8.0/lib/mini_portile2/mini_portile.rb:426:in `chdir'
/var/lib/gems/2.7.0/gems/mini_portile2-2.8.0/lib/mini_portile2/mini_portile.rb:460:in `block in execute': Failed to complete patch task (RuntimeError)
rake aborted!
Command failed with status (1): [/usr/bin/ruby2.7 -I. ../../../../ext/extco...]
/var/lib/gems/2.7.0/gems/rake-compiler-0.9.9/lib/rake/extensiontask.rb:193:in `block (2 levels) in define_compile_tasks'
/var/lib/gems/2.7.0/gems/rake-compiler-0.9.9/lib/rake/extensiontask.rb:190:in `block in define_compile_tasks'
Tasks: TOP => compile => compile:x86_64-linux-gnu => compile:jq_core:x86_64-linux-gnu => copy:jq_core:x86_64-linux-gnu:2.7.4 => tmp/x86_64-linux-gnu/jq_core/2.7.4/jq_core.so => tmp/x86_64-linux-gnu/jq_core/2.7.4/Makefile
(See full trace by running task with --trace)

flavorjones commented 1 year ago

Thanks for the repro! OK, I understand what's happening now.

Many years ago, https://github.com/flavorjones/mini_portile/issues/69 asked for the ability to work in broken git directories. The fix for that was to specify --git-dir=. in e84075b7a3d318b59e86ed9841f117dec2c3dcc6

However, that was not the right thing to do -- for exactly this reason. The presence of a "config" directory is confusing to git, because it thinks it's in a .git directory and tries to read config info (and fails).

I think the right thing to do is:

revert e84075b7a3d318b59e86ed9841f117dec2c3dcc6
reverse the preference of patch and git apply -- that is, prefer patch if it exists (to address #69)

and then your use case will work. I'll put together a PR shortly.

flavorjones commented 1 year ago

Uuuuugh ... d5fec0a set the preference to git apply back in 2015.

OK, I'm going to do a new PR that's a variation of @h0tw1r3's #120 but with a true empty temp directory.

flavorjones commented 1 year ago

Fixed in https://github.com/flavorjones/mini_portile/releases/tag/v2.8.1

flavorjones / mini_portile

apply_patch failure #119