search

eddelbuettel / rprotobuf

R Interface to Protocol Buffers
71 stars 25 forks source link

Protobuf headers not found with default CXXFLAGS and CPPFLAGS, manually trying /usr/local/include #100

Closed sgavathe closed 5 months ago

sgavathe commented 6 months ago

Hello,

I get the below error, when I tried to install RProtobuf using R

R -e "install.packages(c('RProtoBuf'),  dependencies = TRUE)"

I already have a google protobuf dependency installed from the source at a different location which is , instead of using apt-get install

ls /usr/local/include
absl  google  java  utf8_range.h  utf8_validity.h
configure: WARNING: Protobuf headers not found with default CXXFLAGS and CPPFLAGS, manually trying /usr/local/include
configure: WARNING: Unsetting ac_cv_header_google_protobuf_stubs_common_h
checking for google/protobuf/stubs/common.h... no
configure: error: ERROR: ProtoBuf headers required; use '-Iincludedir' in CXXFLAGS for unusual locations.
ERROR: configuration failed for package 'RProtoBuf'
* removing '/workspace/renv/library/R-4.1/x86_64-pc-linux-gnu/RProtoBuf'

The downloaded source packages are in
        '/tmp/Rtmpx0TZrF/downloaded_packages'
Warning message:
In utils::install.packages(c("RProtoBuf"), dependencies = TRUE) :
  installation of package 'RProtoBuf' had non-zero exit status
> 
>

Thanks,

eddelbuettel commented 6 months ago

I think you misunderstand the lines in DESCRIPTION:

https://github.com/eddelbuettel/rprotobuf/blob/5dd25c4b9b029003253b2d02ea8048ccd90a6389/DESCRIPTION#L18-L21

This is assessed in a the configure script created from configure.ac, it relies on pkg-config. So in short you need to make sure pkg-config --exists protobuf returns a valid value:

https://github.com/eddelbuettel/rprotobuf/blob/5dd25c4b9b029003253b2d02ea8048ccd90a6389/configure.ac#L34-L41

If that is not the case on your system you can work around it, see ./configure --help.

Note that the package is 15 years old and has (essentially) always installed this way which works for lots of people.

sgavathe commented 6 months ago

Thanks @eddelbuettel 

pkg-config --exists protobuf && echo "Installed" || echo "Not installed"
Installed

it does return true

How do I install the RProtobuf from source?

I am going to have to try that next. Everything will be on a docker image.

eddelbuettel commented 6 months ago

Please read the next few lines in configure.ac that I quoted an hour ago and also run

~/git/rprotobuf(master)$ pkg-config --cflags protobuf

~/git/rprotobuf(master)$ pkg-config --libs protobuf
-lprotobuf 
~/git/rprotobuf(master)$

I am on Ubuntu, and that is all I need. I.e. when I run ./configure as the package does:

~/git/rprotobuf(master)$ ./configure 
checking whether the C++ compiler works... yes
checking for C++ compiler default output file name... a.out
checking for suffix of executables... 
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether the compiler supports GNU C++... yes
checking whether ccache g++ accepts -g... yes
checking for ccache g++ option to enable C++11 features... none needed
checking how to run the C++ preprocessor... ccache g++ -E
checking for gcc... gcc
checking whether the compiler supports GNU C... yes
checking whether gcc accepts -g... yes
checking for gcc option to enable C11 features... none needed
checking whether the compiler supports GNU C++... (cached) yes
checking whether ccache g++ accepts -g... (cached) yes
checking for ccache g++ option to enable C++11 features... (cached) none needed
checking for pkg-config... yes
checking for protoc... yes
checking for stdio.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for strings.h... yes
checking for sys/stat.h... yes
checking for sys/types.h... yes
checking for unistd.h... yes
checking for google/protobuf/stubs/common.h... yes
checking for google/protobuf/compiler/code_generator.h... yes
checking if ProtoBuf version >= 2.2.0... yes
checking for R... yes
configure: creating ./config.status
config.status: creating src/Makevars

RProtoBuf 0.4.22
================

cflags:    -g -O0 -Wall -pipe -pedantic -fdiagnostics-color=always -Wformat 
libs:     -lprotobuf 

~/git/rprotobuf(master)$

and then

~/git/rprotobuf(master)$ cat src/Makevars
## -*- mode: makefile; -*-

## Make this C++17 so that clang will not complain about [[nodiscard]] in headers
CXX_STD=CXX17

## Configure tells us about locations for ## both Rcpp (ie libRcpp.so and Rcpp.h) and
## ProtoBuf headers and library via these variables
PKG_CPPFLAGS=  -g -O0 -Wall -pipe -pedantic -fdiagnostics-color=always -Wformat 
PKG_LIBS= -lprotobuf 
~/git/rprotobuf(master)$

Note that there may be a binary package of RProtoBuf for you depending on what Linux distro you use (which you never said -- supplying more details generally makes it easier for others to help you).

sgavathe commented 5 months ago

I am putting it all under ubuntu:22.04 docker imager

so what I noticed is, when I install it using

  protobuf-compiler \
  libprotobuf-dev \
  libprotoc-dev \

  RUN R -e "install.packages(c('RProtoBuf'),  dependencies = TRUE)"

I am able to see the RProtobuf works correct.

But when I install the protobuf from source https://github.com/protocolbuffers/protobuf/releases/download/v25.0/protobuf-25.0.tar.gz , it gives the install error.

pkg-config --cflags protobuf
-Wnon-virtual-dtor -DNOMINMAX -Wnon-virtual-dtor -DNOMINMAX -Wnon-virtual-dtor -DNOMINMAX -Wnon-virtual-dtor -DNOMINMAX -Wnon-virtual-dtor -DNOMINMAX -Wnon-virtual-dtor -DNOMINMAX -Wnon-virtual-dtor -DNOMINMAX -Wnon-virtual-dtor -DNOMINMAX -Wnon-virtual-dtor -DNOMINMAX -Wnon-virtual-dtor -DNOMINMAX -Wnon-virtual-dtor -DNOMINMAX -Wnon-virtual-dtor -DNOMINMAX -Wnon-virtual-dtor -DNOMINMAX -Wnon-virtual-dtor -DNOMINMAX -Wnon-virtual-dtor -DNOMINMAX -Wnon-virtual-dtor -DNOMINMAX -Wnon-virtual-dtor -DNOMINMAX -Wnon-virtual-dtor -DNOMINMAX -Wnon-virtual-dtor -DNOMINMAX -Wnon-virtual-dtor -DNOMINMAX -Wnon-virtual-dtor -DNOMINMAX -Wnon-virtual-dtor -DNOMINMAX -Wnon-virtual-dtor -DNOMINMAX -Wnon-virtual-dtor -DNOMINMAX -Wnon-virtual-dtor -DNOMINMAX -Wnon-virtual-dtor -DNOMINMAX -Wnon-virtual-dtor -DNOMINMAX -Wnon-virtual-dtor -DNOMINMAX -Wnon-virtual-dtor -DNOMINMAX -Wnon-virtual-dtor -DNOMINMAX -Wnon-virtual-dtor -DNOMINMAX -Wnon-virtual-dtor -DNOMINMAX -Wnon-virtual-dtor -DNOMINMAX -Wnon-virtual-dtor -DNOMINMAX -Wnon-virtual-dtor -DNOMINMAX -Wnon-virtual-dtor -DNOMINMAX -Wnon-virtual-dtor -DNOMINMAX -Wnon-virtual-dtor -DNOMINMAX -Wnon-virtual-dtor -DNOMINMAX -Wnon-virtual-dtor -DNOMINMAX -Wnon-virtual-dtor -DNOMINMAX -Wnon-virtual-dtor -DNOMINMAX -Wnon-virtual-dtor -DNOMINMAX -Wnon-virtual-dtor -DNOMINMAX -Wnon-virtual-dtor -DNOMINMAX -Wnon-virtual-dtor -DNOMINMAX -Wnon-virtual-dtor -DNOMINMAX -Wnon-virtual-dtor -DNOMINMAX -Wnon-virtual-dtor -DNOMINMAX -Wnon-virtual-dtor -DNOMINMAX -Wnon-virtual-dtor -DNOMINMAX -Wnon-virtual-dtor -DNOMINMAX -Wnon-virtual-dtor -DNOMINMAX -Wnon-virtual-dtor -DNOMINMAX -Wnon-virtual-dtor -DNOMINMAX -Wnon-virtual-dtor -DNOMINMAX -Wnon-virtual-dtor -DNOMINMAX -Wnon-virtual-dtor -DNOMINMAX -Wnon-virtual-dtor -DNOMINMAX -Wnon-virtual-dtor -DNOMINMAX -Wnon-virtual-dtor -DNOMINMAX -Wnon-virtual-dtor -DNOMINMAX -Wnon-virtual-dtor -DNOMINMAX -Wnon-virtual-dtor -DNOMINMAX -Wnon-virtual-dtor -DNOMINMAX -Wnon-virtual-dtor -DNOMINMAX -Wnon-virtual-dtor -DNOMINMAX -Wnon-virtual-dtor -DNOMINMAX -Wnon-virtual-dtor -DNOMINMAX -Wnon-virtual-dtor -DNOMINMAX -Wnon-virtual-dtor -DNOMINMAX -Wnon-virtual-dtor -DNOMINMAX -Wnon-virtual-dtor -DNOMINMAX -Wnon-virtual-dtor -DNOMINMAX -Wnon-virtual-dtor -DNOMINMAX -Wnon-virtual-dtor -DNOMINMAX -Wnon-virtual-dtor -DNOMINMAX -Wnon-virtual-dtor -DNOMINMAX -Wnon-virtual-dtor -DNOMINMAX -Wnon-virtual-dtor -DNOMINMAX -Wnon-virtual-dtor -DNOMINMAX -Wnon-virtual-dtor -DNOMINMAX -Wnon-virtual-dtor -DNOMINMAX -Wnon-virtual-dtor -DNOMINMAX -Wnon-virtual-dtor -DNOMINMAX -Wnon-virtual-dtor -DNOMINMAX -Wnon-virtual-dtor -DNOMINMAX -Wnon-virtual-dtor -DNOMINMAX -Wnon-virtual-dtor -DNOMINMAX -Wnon-virtual-dtor -DNOMINMAX -Wnon-virtual-dtor -DNOMINMAX -Wnon-virtual-dtor -DNOMINMAX -Wnon-virtual-dtor -DNOMINMAX -Wnon-virtual-dtor -DNOMINMAX -Wnon-virtual-dtor -DNOMINMAX -Wnon-virtual-dtor -DNOMINMAX -Wnon-virtual-dtor -DNOMINMAX -Wnon-virtual-dtor -DNOMINMAX -Wnon-virtual-dtor -DNOMINMAX -Wnon-virtual-dtor -DNOMINMAX -Wnon-virtual-dtor -DNOMINMAX -Wnon-virtual-dtor -DNOMINMAX -Wnon-virtual-dtor -DNOMINMAX -Wnon-virtual-dtor -DNOMINMAX -Wnon-virtual-dtor -DNOMINMAX -Wnon-virtual-dtor -DNOMINMAX -Wnon-virtual-dtor -DNOMINMAX -Wnon-virtual-dtor -DNOMINMAX -Wnon-virtual-dtor -DNOMINMAX -Wnon-virtual-dtor -DNOMINMAX -Wnon-virtual-dtor -DNOMINMAX -Wnon-virtual-dtor -DNOMINMAX -Wnon-virtual-dtor -DNOMINMAX -Wnon-virtual-dtor -DNOMINMAX -Wnon-virtual-dtor -DNOMINMAX -Wnon-virtual-dtor -DNOMINMAX -Wnon-virtual-dtor -DNOMINMAX -Wnon-virtual-dtor -DNOMINMAX -Wnon-virtual-dtor -DNOMINMAX -Wnon-virtual-dtor -DNOMINMAX -Wnon-virtual-dtor -DNOMINMAX -Wnon-virtual-dtor -DNOMINMAX -Wnon-virtual-dtor -DNOMINMAX -Wnon-virtual-dtor -DNOMINMAX -Wnon-virtual-dtor -DNOMINMAX -Wnon-virtual-dtor -DNOMINMAX -Wnon-virtual-dtor -DNOMINMAX -Wnon-virtual-dtor -DNOMINMAX -Wnon-virtual-dtor -DNOMINMAX -Wnon-virtual-dtor -DNOMINMAX -Wnon-virtual-dtor -DNOMINMAX -Wnon-virtual-dtor -DNOMINMAX -Wnon-virtual-dtor -DNOMINMAX -Wnon-virtual-dtor -DNOMINMAX -Wnon-virtual-dtor -DNOMINMAX -Wnon-virtual-dtor -DNOMINMAX -Wnon-virtual-dtor -DNOMINMAX -Wnon-virtual-dtor -DNOMINMAX -I/usr/local/include
pkg-config --libs protobuf
-L/usr/local/lib -lprotobuf -labsl_log_internal_check_op -labsl_leak_check -labsl_die_if_null -labsl_log_internal_conditions -labsl_log_internal_message -labsl_examine_stack -labsl_log_internal_format -labsl_log_internal_proto -labsl_log_internal_nullguard -labsl_log_internal_log_sink_set -labsl_log_sink -labsl_log_entry -labsl_flags_internal -labsl_flags_marshalling -labsl_flags_reflection -labsl_flags_private_handle_accessor -labsl_flags_commandlineflag -labsl_flags_commandlineflag_internal -labsl_flags_config -labsl_flags_program_name -labsl_log_initialize -labsl_log_globals -labsl_vlog_config_internal -labsl_log_internal_fnmatch -labsl_log_internal_globals -labsl_raw_hash_set -labsl_hash -labsl_city -labsl_low_level_hash -labsl_hashtablez_sampler -labsl_statusor -labsl_status -labsl_cord -labsl_cordz_info -labsl_cord_internal -labsl_cordz_functions -labsl_exponential_biased -labsl_cordz_handle -labsl_crc_cord_state -labsl_crc32c -labsl_crc_internal -labsl_crc_cpu_detect -labsl_bad_optional_access -labsl_strerror -labsl_str_format_internal -labsl_synchronization -labsl_graphcycles_internal -labsl_kernel_timeout_internal -labsl_stacktrace -labsl_symbolize -labsl_debugging_internal -labsl_demangle_internal -labsl_malloc_internal -labsl_time -labsl_civil_time -labsl_time_zone -labsl_bad_variant_access -lutf8_validity -lutf8_range -labsl_strings -labsl_strings_internal -labsl_string_view -labsl_base -lrt -labsl_spinlock_wait -labsl_int128 -labsl_throw_delegate -labsl_raw_logging_internal -labsl_log_severity

I do not see pthread in it when I install protobuf from source, so this may be an issue.

pkg-config --cflags protobuf
-pthread
eddelbuettel commented 5 months ago

I think we have to call that your local deployment problem in how you configure and build Protocol Buffers (upstream) from source. You can study the sources of the Debian / Ubuntu package and see what it does differently. There is a version 3.25 in experimental too, and I see that 27.0~rc2 is tagged. Might be best to talk to the package maintainer, or upstream. I will updated the package as needed (and folks from upstream / Google have been helpful with PRs for code changes) but I don't expect the basic structure to change.

Let me know if you come up with any needed changes, but I think we can close this for now for the RProtoBuf package.