or-tools v9.7 - Githubissues

regro-cf-autotick-bot commented 11 months ago

It is very likely that the current package version for this feedstock is out of date.

Checklist before merging this PR:

[ ] Dependencies have been updated if changed: see upstream
[ ] Tests have passed
[ ] Updated license if changed and license_file is packaged

Information about this PR:

Feel free to push to the bot's branch to update this PR if needed.
The bot will almost always only open one PR per version.
The bot will stop issuing PRs if more than 3 version bump PRs generated by the bot are open. If you don't want to package a particular version please close the PR.
If you want these PRs to be merged automatically, make an issue with code>@conda-forge-admin,</codeplease add bot automerge in the title and merge the resulting PR. This command will add our bot automerge feature to your feedstock.
If this PR was opened in error or needs to be updated please add the bot-rerun label to this PR. The bot will close this PR and schedule another one. If you do not have permissions to add this label, you can use the phrase code>@<space/conda-forge-admin, please rerun bot in a PR comment to have the conda-forge-admin add it for you.

Pending Dependency Version Updates

Here is a list of all the pending dependency version updates for this repo. Please double check all dependencies before merging.

Name	Upstream Version	Current Version
or-tools	9.7
protobuf	23.4

Dependency Analysis

We couldn't run dependency analysis due to an internal error in the bot, depfinder, or grayskull. :/ Help is very welcome!

_{This PR was created by the regro-cf-autotick-bot. The regro-cf-autotick-bot is a service to automatically track the dependency graph, migrate packages, and propose package version updates for conda-forge. Feel free to drop us a line if there are any issues! This PR was generated by https://github.com/regro/cf-scripts/actions/runs/5799934866, please use this URL for debugging.}

conda-forge-webservices[bot] commented 11 months ago

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.

h-vetinari commented 11 months ago

@Mizux, there's a problem with CpSolverResponse it seems:

[119/504] Building CXX object ortools/constraint_solver/CMakeFiles/ortools_constraint_solver.dir/routing_breaks.cc.o
FAILED: ortools/constraint_solver/CMakeFiles/ortools_constraint_solver.dir/routing_breaks.cc.o 
$BUILD_PREFIX/bin/x86_64-conda-linux-gnu-c++ -DOR_TOOLS_AS_DYNAMIC_LIB -DUSE_BOP -DUSE_CBC -DUSE_CLP -DUSE_GLOP -DUSE_LP_PARSER -DUSE_PDLP -DUSE_SCIP -I$SRC_DIR -I$SRC_DIR/build-cpp -fvisibility-inlines-hidden -fmessage-length=0 -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -ffunction-sections -pipe -isystem $PREFIX/include -fdebug-prefix-map=$SRC_DIR=/usr/local/src/conda/or-tools-package-9.7 -fdebug-prefix-map=$PREFIX=/usr/local/src/conda-prefix -O3 -DNDEBUG -std=c++17 -fPIC -fwrapv -MD -MT ortools/constraint_solver/CMakeFiles/ortools_constraint_solver.dir/routing_breaks.cc.o -MF ortools/constraint_solver/CMakeFiles/ortools_constraint_solver.dir/routing_breaks.cc.o.d -o ortools/constraint_solver/CMakeFiles/ortools_constraint_solver.dir/routing_breaks.cc.o -c $SRC_DIR/ortools/constraint_solver/routing_breaks.cc
In file included from $SRC_DIR/ortools/graph/graph.h:174,
                 from $SRC_DIR/ortools/constraint_solver/routing.h:188,
                 from $SRC_DIR/ortools/constraint_solver/routing_breaks.cc:28:
$SRC_DIR/ortools/graph/iterators.h:106:19: warning: 'template<class _Category, class _Tp, class _Distance, class _Pointer, class _Reference> struct std::iterator' is deprecated [-Wdeprecated-declarations]
  106 |     : public std::iterator<std::input_iterator_tag, IntegerType> {
      |                   ^~~~~~~~
In file included from $BUILD_PREFIX/x86_64-conda-linux-gnu/include/c++/12.3.0/bits/stl_algobase.h:65,
                 from $BUILD_PREFIX/x86_64-conda-linux-gnu/include/c++/12.3.0/algorithm:60,
                 from $SRC_DIR/ortools/constraint_solver/routing_breaks.cc:14:
$BUILD_PREFIX/x86_64-conda-linux-gnu/include/c++/12.3.0/bits/stl_iterator_base_types.h:127:34: note: declared here
  127 |     struct _GLIBCXX17_DEPRECATED iterator
      |                                  ^~~~~~~~
In file included from $PREFIX/include/absl/log/internal/strip.h:24,
                 from $PREFIX/include/absl/log/internal/check_op.h:37,
                 from $PREFIX/include/absl/log/internal/check_impl.h:19,
                 from $PREFIX/include/absl/log/check.h:37,
                 from $SRC_DIR/ortools/base/logging.h:23,
                 from $SRC_DIR/ortools/constraint_solver/routing_breaks.cc:25:
$PREFIX/include/absl/log/internal/log_message.h: In instantiation of 'absl::lts_20230125::log_internal::LogMessage& absl::lts_20230125::log_internal::LogMessage::operator<<(const T&) [with T = operations_research::sat::CpSolverResponse; typename std::enable_if<(! absl::lts_20230125::strings_internal::HasAbslStringify<T>::value), int>::type <anonymous> = 0]':
$SRC_DIR/ortools/constraint_solver/routing_lp_scheduling.h:592:16:   required from here
$PREFIX/include/absl/log/internal/log_message.h:289:17: error: no match for 'operator<<' (operand types are 'std::ostream' {aka 'std::basic_ostream<char>'} and 'const operations_research::sat::CpSolverResponse')
  289 |   view.stream() << log_internal::NullGuard<T>().Guard(v);
      |   ~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

In particular, it seems there are type conversions missing.

In file included from $PREFIX/include/absl/base/log_severity.h:19,
                 from $SRC_DIR/ortools/base/logging.h:21:
$BUILD_PREFIX/x86_64-conda-linux-gnu/include/c++/12.3.0/ostream:108:7: note: candidate: 'std::basic_ostream<_CharT, _Traits>::__ostream_type& std::basic_ostream<_CharT, _Traits>::operator<<(__ostream_type& (*)(__ostream_type&)) [with _CharT = char; _Traits = std::char_traits<char>; __ostream_type = std::basic_ostream<char>]'
  108 |       operator<<(__ostream_type& (*__pf)(__ostream_type&))
      |       ^~~~~~~~
$BUILD_PREFIX/x86_64-conda-linux-gnu/include/c++/12.3.0/ostream:108:36: note:   no known conversion for argument 1 from 'const operations_research::sat::CpSolverResponse' to 'std::basic_ostream<char>::__ostream_type& (*)(std::basic_ostream<char>::__ostream_type&)' {aka 'std::basic_ostream<char>& (*)(std::basic_ostream<char>&)'}
  108 |       operator<<(__ostream_type& (*__pf)(__ostream_type&))
      |                  ~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~
$BUILD_PREFIX/x86_64-conda-linux-gnu/include/c++/12.3.0/ostream:117:7: note: candidate: 'std::basic_ostream<_CharT, _Traits>::__ostream_type& std::basic_ostream<_CharT, _Traits>::operator<<(__ios_type& (*)(__ios_type&)) [with _CharT = char; _Traits = std::char_traits<char>; __ostream_type = std::basic_ostream<char>; __ios_type = std::basic_ios<char>]'
  117 |       operator<<(__ios_type& (*__pf)(__ios_type&))
      |       ^~~~~~~~
$BUILD_PREFIX/x86_64-conda-linux-gnu/include/c++/12.3.0/ostream:117:32: note:   no known conversion for argument 1 from 'const operations_research::sat::CpSolverResponse' to 'std::basic_ostream<char>::__ios_type& (*)(std::basic_ostream<char>::__ios_type&)' {aka 'std::basic_ios<char>& (*)(std::basic_ios<char>&)'}
  117 |       operator<<(__ios_type& (*__pf)(__ios_type&))
      |                  ~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~
$BUILD_PREFIX/x86_64-conda-linux-gnu/include/c++/12.3.0/ostream:127:7: note: candidate: 'std::basic_ostream<_CharT, _Traits>::__ostream_type& std::basic_ostream<_CharT, _Traits>::operator<<(std::ios_base& (*)(std::ios_base&)) [with _CharT = char; _Traits = std::char_traits<char>; __ostream_type = std::basic_ostream<char>]'
  127 |       operator<<(ios_base& (*__pf) (ios_base&))
      |       ^~~~~~~~
$BUILD_PREFIX/x86_64-conda-linux-gnu/include/c++/12.3.0/ostream:127:30: note:   no known conversion for argument 1 from 'const operations_research::sat::CpSolverResponse' to 'std::ios_base& (*)(std::ios_base&)'
  127 |       operator<<(ios_base& (*__pf) (ios_base&))
      |                  ~~~~~~~~~~~~^~~~~~~~~~~~~~~~~
$BUILD_PREFIX/x86_64-conda-linux-gnu/include/c++/12.3.0/ostream:166:7: note: candidate: 'std::basic_ostream<_CharT, _Traits>::__ostream_type& std::basic_ostream<_CharT, _Traits>::operator<<(long int) [with _CharT = char; _Traits = std::char_traits<char>; __ostream_type = std::basic_ostream<char>]'
  166 |       operator<<(long __n)
      |       ^~~~~~~~
$BUILD_PREFIX/x86_64-conda-linux-gnu/include/c++/12.3.0/ostream:166:23: note:   no known conversion for argument 1 from 'const operations_research::sat::CpSolverResponse' to 'long int'
  166 |       operator<<(long __n)
      |                  ~~~~~^~~
[...]

Also, does 9.7 only support protobuf >=4.23.3 or should I still expect it to be buildable (even if non-standard) with 3.21? We still haven't flipped the switch to stop building 3.21, so if possible I'd like to build for both (but not extremely important).

Mizux commented 11 months ago

IIRC this may due to:

the update of re2 which change its type from re2::StringPiece to absl::string_view
- https://github.com/google/or-tools/issues/3201
The bump of protobuf using the last abseil-cpp
- https://github.com/google/or-tools/issues/3826

Seems likely the 2, started from v9.7 we are using Protobuf v23.3 (i.e. abseil-cpp based version)...

h-vetinari commented 11 months ago

the update of re2 which change its type from re2::StringPiece to absl::string_view

Conda-forge's re2 is using absl::string_view since https://github.com/conda-forge/re2-feedstock/pull/62.

The bump of protobuf using the last abseil-cpp

Yeah, I copied the wrong error (was too busy this morning 🙈). It's fine to drop support for 3.21 then. Guess I'll have to dig into the errors on protobuf 4, which indeed look related to re2::StringPiece. Are you using a vendored re2 and/or abseil by any chance?

Mizux commented 11 months ago

for re2 not anymore, for abseil-cpp small patch https://github.com/google/or-tools/blob/stable/patches/abseil-cpp-20230125.3.patch (IIRC mostly add log because since cmake 3.19 ) check_cxx_source_compiles() do not take into account CMAKE_CXX_STANDARD and old debian gcc -> abseil-cpp was fallback to c++14 instead of c++17 (even if set CMAKE_STANDARD_CXX etc is set...) causing the build to fail.

Still need to investigate and send an error to kitware/abseil-cpp to confirm the behaviour and find a clean fix.

h-vetinari commented 10 months ago

@Mizux this still fails with an incompatibility between absl::string_view and re2::StringPiece, which is strange to me, because we have an explicit test for that interoperability and it's passing.

$SRC_DIR/ortools/lp_data/lp_parser.cc: In function 'absl::lts_20230802::StatusOr<operations_research::glop::ParsedConstraint> operations_research::glop::ParseConstraint(absl::lts_20230802::string_view)':
$SRC_DIR/ortools/lp_data/lp_parser.cc:365:20: error: cannot convert 'absl::lts_20230802::string_view*' {aka 'std::basic_string_view<char>*'} to 'operations_research::glop::{anonymous}::StringPiece*' {aka 're2::StringPiece*'}
  365 |       ConsumeToken(&constraint, &consumed_name, &consumed_coeff);
      |                    ^~~~~~~~~~~
      |                    |
      |                    absl::lts_20230802::string_view* {aka std::basic_string_view<char>*}

check_cxx_source_compiles() do not take into account CMAKE_CXX_STANDARD and old debian gcc -> abseil-cpp was fallback to c++14 instead of c++17 (even if set CMAKE_STANDARD_CXX etc is set...) causing the build to fail.

Still need to investigate and send an error to kitware/abseil-cpp to confirm the behaviour and find a clean fix.

I'm not 100% I understand what you mean here - is there still something in or-tools that could force compilation with C++14? That would be a possible explanation for the ABI-mismatch.

Mizux commented 10 months ago

don't know if it help but on main we have:

%git grep StringPiece ortools/lp_data/lp_parser.cc:using StringPiece = ::re2::StringPiece;

and we are using [0]─[~/work/main/build/_deps/re2-src]-[tags/2023-08-01]

cd build/_deps/re2-src
cat re2/stringpiece.h
...
namespace re2 {
// Until RE2 requires C++17 and uses std::string_view, allow users to
// continue to #include "re2/stringpiece.h" and use re2::StringPiece.
using StringPiece = absl::string_view;
}  // namespace re2

so ortools seems to use an re2 version which rely on absl::string_view (when using -DBUILD_DEPS=ON) However, in your case i guess you are compiling ortools against the re2 version provided by conda -> may I have the version ?

note: https://github.com/google/re2/releases seems there is a 2023-09-01, note2: we should release a v9.8 around october so if we can fix the re2 type mismatch before so conda could integrate it and get rid of this v9.7 borken build ^^;

h-vetinari commented 10 months ago

Thanks for the response!

so ortools seems to use an re2 version which rely on absl::string_view

It's my understanding that these should be compatible (when both compiled with C++17), see the comment above. So I'm surprised why this shouldn't work.

However, in your case i guess you are compiling ortools against the re2 version provided by conda -> may I have the version ?

Yes. It's currently 2023.03.01. Normally we're much faster with upgrading, but we have a discussion that got stuck in https://github.com/conda-forge/re2-feedstock/pull/68, and I haven't had time/energy to get it unstuck again.

Mizux commented 10 months ago

Also or-tools need C++17 to compile (actually C++20 for msvc due to usage of "agregate initialization" (which is backported to -std=c++17 on clang/gcc since it comes from c99...)). Which means we are testing with abseil-cpp using c++17/c++20 so absl::string_view is an alias to std::string_view .

ref:

C++20 for (3) https://en.cppreference.com/w/cpp/language/aggregate_initialization
C99 for (2) https://en.cppreference.com/w/c/language/struct_initialization

h-vetinari commented 10 months ago

I'm aware of the ABI-implications of compiling abseil with C++17. ;-)

So since both of those types (re2::StringPiece and absl::string_view) should alias to std::string_view, I don't understand the incompatibility.

Mizux commented 10 months ago

https://dev.azure.com/conda-forge/feedstock-builds/_build/results?buildId=788588&view=logs&j=656edd35-690f-5c53-9ba3-09c10d0bea97&t=986b1512-c876-5f92-0d81-ba851554a0a3&l=1678 correctly see a -std=c++17 so or-tools seems compiled in C++17 which leads me to challenge the version of re2 headers used :thinking:

Mizux commented 10 months ago

@h-vetinari please take a look at: https://github.com/conda-forge/or-tools-feedstock/pull/47/files#diff-3190fed5ebb234b0a002a306df50050c27966b8472a74cff06c84de06fd3b777R46 seems to use re2 2023.03.02?

EDIT: I can see a re2 tag https://github.com/google/re2/tree/2023-03-01 but no 2023.03.02 https://github.com/google/re2/blob/2023-03-01/re2/stringpiece.h don't use absl::string_view so re2::StringPiece do not alias to absl...

Mizux commented 10 months ago

FYI re2 perform the transition to absl::string_view alias in tag 2023-06-01 (which is the next release after re2 2023-03-01 so any release bump should fix this or-tools bug) src: https://github.com/google/re2/blob/2023-06-01/re2/stringpiece.h

h-vetinari commented 10 months ago

FYI re2 perform the transition to absl::string_view alias in tag 2023-06-01 (which is the next release after re2 2023-03-01 so any release bump should fix this or-tools bug)

Thanks. AFAIR the use of abseil types was already supported before that with an option, but it seems the first step should just be to unblock that re2 upgrade. :)

h-vetinari commented 9 months ago

@Mizux, with the newer re2, this gets further, but now fails at

$SRC_DIR/ortools/sat/cuts.cc:79:36: error: no matching function for call to 'StrCat(const char [13], const absl::lts_20230802::int128&, const char [2])'
   79 |   std::string result = absl::StrCat("CutData rhs=", rhs, "\n");
      |                        ~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~

should the rhs variable be (cast to) a string type here, rather than being an integer reference?

h-vetinari commented 9 months ago

Any ideas what's going on here @Mizux? :)

Mizux commented 9 months ago

@h-vetinari seems to comes from https://github.com/google/or-tools/commit/734df45de4cf8c1a59b3fef972182dd0b48ef3e2

https://github.com/google/or-tools/commits/main/ortools/sat/cuts.cc see around August 10th seems we had fought against abseil-cpp/re2 version bump...

Also you are using re2 2023-06-02 while v9.7 has been released with 2023-07-01 ref: https://github.com/google/or-tools/blob/v9.7/Dependencies.txt#L9

ps: For incoming v9.8 we will use re2 2023-09-01 (patch on main comming soon)...

Mizux commented 8 months ago

@h-vetinari Concerning the std::string result = absl::StrCat("CutData rhs=", rhs, "\n"); due to https://github.com/abseil/abseil-cpp/commit/34eb767645347f100bdd66fc1e35eee96e357961 (20230802.1) We had to remove our custom AbslStringify() overload for absl::int128 see https://github.com/google/or-tools/commit/734df45de4cf8c1a59b3fef972182dd0b48ef3e2 (main/v9.8)

So:

to build or-tools v9.7 you need abseil-cpp < 20230802.1,
to build or-tools main/v9.8+ you need abseil-cpp >= 20230802.1 and the commit above which is already in the branch.

note: in your Pending Dependency Version Updates please add the abseil-cpp version and re2 version used...

h-vetinari commented 5 months ago

Sorry for the very long delay here. I finally unblocked newer re2, but by now, abseil has moved on. The lowest we can go is 20230802, though the migration to 20240116 is already in progress. In any case, I'm surprised by the following error.

$PREFIX/include/absl/strings/str_cat.h:468:8: error: invalid 'static_cast' from type 'const absl::lts_20230802::int128' to type 'const absl::lts_20230802::AlphaNum&'
  468 |        static_cast<const AlphaNum&>(args).Piece()...});
      |        ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

AFAICT, this should be explicitly covered by https://github.com/abseil/abseil-cpp/commit/34eb767645347f100bdd66fc1e35eee96e357961, which is part of 20230802, but it seems that either abseil (or the compiler) don't think that the respective template applies, presumably because std::enable_if<strings_internal::HasAbslStringify<T>::value>::type> somehow wrongly evaluates to False?

PS. The errors for abseil 20240116 looked even worse (at first glance), so I made it possible to use newer re2 with the previous abseil as well. That said, if things don't work for 9.7, we can try 9.8 directly as well of course.

Mizux commented 5 months ago

@h-vetinari FYI we are stabilising or-tools for v9.9 release (hopefully this week) so you may try to bump directly to it. As any release, we try to be up to date with last release of our dependencies:

C++

Protobuf=v25.3
abseil-cpp=20240116.1
re2=2024-02-01

Python

pybind11=v2.11.1
pybind11_abseil=52f2739
pybind11_protobuf=3b11990

Conda-forge ref

https://github.com/conda-forge/abseil-cpp-feedstock 20240116.1 up to date https://github.com/conda-forge/protobuf-feedstock (note: current release v21.2 in README but you've merged https://github.com/conda-forge/protobuf-feedstock/pull/212) up to date ? https://github.com/conda-forge/re2-feedstock/pull/75 (pending 2024-02-01 update)

conda-forge / or-tools-feedstock

or-tools v9.7 #47

Pending Dependency Version Updates

Dependency Analysis

C++

Python

Conda-forge ref