Closed kenhys closed 1 month ago
bundled gem size was increased. (~5MB)
mainly rdkafka upgrade affect it.( 2.3MB=>7.2MB )
need to relax limitation for a while.
ffi 1.16.3 can't be built on AmazonLinux 2.
https://github.com/fluent/fluent-package-builder/actions/runs/10034848340/job/27729984596
rdkafka 0.16.1 is not correctly built?
If rdkafka was reverted to 0.16.0, package size was not so increased. (may be no need to relax limitation )
rdkafka 0.16.0 is buldable, but not available in colfluent test.
/opt/fluent/bin/gem list |grep rdkafka rdkafka (0.16.0)
rdkafka was installed. but not recognized... :thinking:
rdkafka 0.16.0 was installed at:
~$ find /opt/fluent/ -name '*rdkafka*'
/opt/fluent/lib/ruby/gems/3.2.0/specifications/rdkafka-0.16.0.gemspec
/opt/fluent/lib/ruby/gems/3.2.0/gems/rdkafka-0.16.0
/opt/fluent/lib/ruby/gems/3.2.0/gems/rdkafka-0.16.0/lib/rdkafka
/opt/fluent/lib/ruby/gems/3.2.0/gems/rdkafka-0.16.0/lib/rdkafka.rb
/opt/fluent/lib/ruby/gems/3.2.0/gems/rdkafka-0.16.0/ext/librdkafka.so
/opt/fluent/lib/ruby/gems/3.2.0/gems/rdkafka-0.16.0/rdkafka.gemspec
/opt/fluent/lib/ruby/gems/3.2.0/gems/fluent-plugin-kafka-0.19.2/lib/fluent/plugin/in_rdkafka_group.rb
/opt/fluent/lib/ruby/gems/3.2.0/gems/fluent-plugin-kafka-0.19.2/lib/fluent/plugin/out_rdkafka.rb
/opt/fluent/lib/ruby/gems/3.2.0/gems/fluent-plugin-kafka-0.19.2/lib/fluent/plugin/out_rdkafka2.rb
/opt/fluent/lib/ruby/gems/3.2.0/extensions/x86_64-linux/3.2.0/rdkafka-0.16.0
The rdkafka failure is caused by removing gem.build_complete.
https://github.com/fluent/fluent-package-builder/actions/runs/10036365251/job/27733978175
windows_platforms = [:mingw, :x64_mingw] # :mswin
gem "ffi", "1.16.3", platforms: windows_platforms
Thus, platforms: does not work as expected on AmazonLinux 2.
+ /usr/lib/rpm/brp-python-bytecompile /usr/bin/python 1
Compiling /root/rpmbuild/BUILDROOT/fluent-package-5.1.0-1.amzn2.x86_64/opt/fluent/lib/ruby/gems/3.2.0/gems/ffi-1.16.3/ext/ffi_c/libffi/generate-darwin-source-and-headers.py ...
And during building debuinfo?, unexpectedly bytecompile process was kicked.
And during building debuinfo?, unexpectedly bytecompile process was kicked.
Same result (v1.17.0) on AmazonLinux 2.
https://fedoraproject.org/wiki/Changes/No_more_automagic_Python_bytecompilation
does not work as expected...
container ea60ff1bf4515e0721f4583107bf62c2fee67b8fe97b9435ff8ca5bf4d1a295c encountered an error during hcs::System::Start: context deadline exceeded
it may be resolved with retry.
Check whether rdkafka 0.16.0 => 0.16.1 upgrade can be bundled or not again. (as gem build_complete issue was resolved)
Blocker issue on Windows:
Downloading fluent-plugin-webhdfs-1.6.0 revealed dependencies not in the API or
the lockfile (webhdfs (>= 0.11.0)).
Either installing with `--full-index` or running `bundle update
fluent-plugin-webhdfs` should fix the problem.
rake aborted!
Note: confluent community can't be installed on AmazonLinux (via package management)
Note: confluent community can't be installed on AmazonLinux (via package management)
Precisely, confluent-client repository is not supported.
build failure on windows was fixed in https://github.com/fluent/fluent-package-builder/actions/runs/10052696903/job/27784298154
kafka related test case is bit out-dated. so refactor it now.
NOTE:
Now macOS workflow always fails because of refreshed runner image
https://github.com/fluent/fluent-package-builder/actions/runs/10105868174/job/27947051208 macos-14-arm64 20240722.3 causes failure.
https://github.com/fluent/fluent-package-builder/actions/runs/10091587427/job/27903442530 until version macos-14-arm64 20240714.2, it succeeds.
recently nokogiri 1.16.7 was released, catch up it.
recently nokogiri 1.16.7 was released, catch up it.
done.
We need to tackle windows build failure...
Try to update to ruby 3.2.5.
We need to tackle windows build failure...
Noticed that nokogiri 1.16.7 fails to build on windows, but not reproducible now...
build process on arm works as expected.
Even though some fixes for script are needed, but it works well with switching signing key. checked with test/experimental/5 repository.
Done!
Tracking v5.1.0 issues still in development.