Issues Installing tink for python 3.10 on intel chip

[jyothiyzen]

We are having issues installing tink on an intel based MacBook Prod using the commit 9f30c97cb84b10bbba6978bc9c12c86478024050 and master branch. Note: These versions work with Python 3.10 on a MacBook with M1 chip

On an intel based MacBook Pro. We tried the following.

• Install using bazelisk and build tink from python folder
• Install with bazel version 5.1.1 and bazel build ... from the python folder
• Install with latest version of protobuf
• Install with older version of protobuf 3.19.4

What was the expected behavior? Tink library is expected to be built using 9f30c97cb84b10bbba6978bc9c12c86478024050 or later version when using python 3.10

How can we reproduce the bug? Installing Bazel Version 5.1.1

brew install protobuf
export BAZEL_VERSION=5.1.1
curl -fLO "https://github.com/bazelbuild/bazel/releases/download/${BAZEL_VERSION}/bazel-${BAZEL_VERSION}-installer-darwin-x86_64.sh"
chmod +x "bazel-${BAZEL_VERSION}-installer-darwin-x86_64.sh"
./bazel-${BAZEL_VERSION}-installer-darwin-x86_64.sh --user
export PATH="$PATH:$HOME/bin"
bazel --version

Install Tink in virtual environment in python version 3.10.0

pyenv 3.10.0
pipenv shell --python 3.10.0
curl -LO https://github.com/google/tink/archive/9f30c97cb84b10bbba6978bc9c12c86478024050.zip
unzip 9f30c97cb84b10bbba6978bc9c12c86478024050.zip
cd tink-9f30c97cb84b10bbba6978bc9c12c86478024050/python
bazel build ...

Do you have any debugging information?

ERROR: /private/var/tmp/_bazel_vicpar/015aeb58222f5cee6251018ac2ff96ad/external/com_google_protobuf/BUILD:155:11: Compiling src/google/protobuf/arena.cc failed: (Exit 1): cc_wrapper.sh failed: error executing command external/local_config_cc/cc_wrapper.sh -U_FORTIFY_SOURCE -fstack-protector -Wall -Wthread-safety -Wself-assign -Wunused-but-set-parameter -Wno-free-nonheap-object -fcolor-diagnostics ... (remaining 35 arguments skipped)

Use --sandbox_debug to see verbose messages from the sandbox

external/com_google_protobuf/src/google/protobuf/arena.cc:108:14: error: out-of-line definition of 'SerialArena' does not match any declaration in 'google::protobuf::internal::SerialArena'

SerialArena::SerialArena(Block* b, void* owner) : space_allocated_(b->size) {

             ^~~~~~~~~~~

external/com_google_protobuf/src/google/protobuf/arena.cc:115:27: error: out-of-line definition of 'New' does not match any declaration in 'google::protobuf::internal::SerialArena'

SerialArena* SerialArena::New(Memory mem, void* owner) {

                          ^~~

external/com_google_protobuf/src/google/protobuf/arena.cc:119:45: error: no matching constructor for initialization of 'google::protobuf::internal::SerialArena'

  return new (b->Pointer(kBlockHeaderSize)) SerialArena(b, owner);

                                            ^           ~~~~~~~~

/usr/local/include/google/protobuf/arena_impl.h:204:23: note: candidate constructor (the implicit copy constructor) not viable: requires 1 argument, but 2 were provided

class PROTOBUF_EXPORT SerialArena {

                      ^

/usr/local/include/google/protobuf/arena_impl.h:428:10: note: candidate constructor not viable: requires 3 arguments, but 2 were provided

  inline SerialArena(Block* b, void* owner, ThreadSafeArenaStats* stats);

         ^

external/com_google_protobuf/src/google/protobuf/arena.cc:318:70: error: too few arguments to function call, expected 3, have 2

  SerialArena* serial = SerialArena::New({mem, size}, &thread_cache());

                        ~~~~~~~~~~~~~~~~                             ^

/usr/local/include/google/protobuf/arena_impl.h:375:23: note: 'New' declared here

  static SerialArena* New(SerialArena::Memory mem, void* owner,

                      ^

external/com_google_protobuf/src/google/protobuf/arena.cc:477:69: error: too few arguments to function call, expected 3, have 2

        AllocateMemory(alloc_policy_.get(), 0, kSerialArenaSize), me);

                                                                    ^

/usr/local/include/google/protobuf/arena_impl.h:375:23: note: 'New' declared here

  static SerialArena* New(SerialArena::Memory mem, void* owner,

                      ^

/usr/local/include/google/protobuf/arena_impl.h:660:30: warning: private field 'arena_stats_' is not used [-Wunused-private-field]

  ThreadSafeArenaStatsHandle arena_stats_;

                             ^

1 warning and 5 errors generated.

INFO: Elapsed time: 110.389s, Critical Path: 1.76s

INFO: 24 processes: 22 internal, 2 darwin-sandbox.

FAILED: Build did NOT complete successfully

version information:

• Language: Python
• Version: 3.10
• Environment: MacBook (Intel Based)

[chuckx]

Can you try building just the C++ implementation and see if it fails as well?

From the repository root:

$ cd cc
$ bazel build ...

[jyothiyzen]

I get the same error when I try building the C++ implementation on the https://github.com/google/tink/commit/9f30c97cb84b10bbba6978bc9c12c86478024050 commit. When I build it on the master branch I get the following error

ERROR: /private/var/tmp/_bazel_vicpar/3ff93db1c7e2ed0959f0ecf3c53b4b9e/external/com_google_protobuf/BUILD:155:11: Compiling src/google/protobuf/any_lite.cc failed: (Exit 1): cc_wrapper.sh failed: error executing command external/local_config_cc/cc_wrapper.sh -U_FORTIFY_SOURCE -fstack-protector -Wall -Wthread-safety -Wself-assign -Wunused-but-set-parameter -Wno-free-nonheap-object -fcolor-diagnostics ... (remaining 35 arguments skipped)

Use --sandbox_debug to see verbose messages from the sandbox

external/com_google_protobuf/src/google/protobuf/any_lite.cc:59:14: error: no matching member function for call to 'Set'

  type_url_->Set(&::google::protobuf::internal::GetEmptyString(),

  ~~~~~~~~~~~^~~

/usr/local/include/google/protobuf/arenastring.h:412:29: note: candidate function not viable: no known conversion from 'const std::string *' (aka 'const basic_string<char, char_traits<char>, allocator<char>> *') to 'const char *' for 1st argument

inline void ArenaStringPtr::Set(const char* s, size_t n, Arena* arena) {

                            ^

/usr/local/include/google/protobuf/arenastring.h:262:8: note: candidate function not viable: requires 2 arguments, but 3 were provided

  void Set(ConstStringParam value, Arena* arena);

       ^

/usr/local/include/google/protobuf/arenastring.h:263:8: note: candidate function not viable: requires 2 arguments, but 3 were provided

  void Set(std::string&& value, Arena* arena);

       ^

/usr/local/include/google/protobuf/arenastring.h:408:29: note: candidate function not viable: requires 2 arguments, but 3 were provided

inline void ArenaStringPtr::Set(const char* s, Arena* arena) {

                            ^

/usr/local/include/google/protobuf/arenastring.h:273:8: note: candidate function template not viable: requires 2 arguments, but 3 were provided

  void Set(std::reference_wrapper<RefWrappedType> const_string_ref,

       ^

external/com_google_protobuf/src/google/protobuf/any_lite.cc:62:39: error: no member named 'EmptyDefault' in 'google::protobuf::internal::ArenaStringPtr'

      value_->Mutable(ArenaStringPtr::EmptyDefault{}, arena));

[chuckx]

Thanks for testing!

What version of macOS are you running?

Even though we have continuous tests running in a macOS enviroment, it's an older version. I want to make sure I have an accurate environment for reproducing this failure.

[jyothiyzen]

I am using MacOS Monterey on Intel Core I7 chip. Here is the environment I am using

kern.version: Darwin Kernel Version 21.5.0: Tue Apr 26 21:08:22 PDT 2022; root:xnu-8020.121.3~4/RELEASE_X86_64 kern.ostype: Darwin kern.osrelease: 21.5.0 kern.osrevision: 199506 ProductName: macOS ProductVersion: 12.4 BuildVersion: 21F79

Running from the zshell (5.8.1) Bazel: 5.1.1 protoc: 3.21.3

[chuckx]

FYI, I'm still having trouble reproducing the failure you're running into.

One more spot check, does the C++ build fail after performing bazel clean --expunge? I've had a few instances where I've gotten Bazel into an inconsistent state which required flushing build outputs to clear up.

In the meantime, I'm still working on getting an environment where I can reproduce the failure.

[jyothiyzen]

I tried thebazel clean --expunge but the build still fails. I can clean up the rest of my environment and see if any other install is interfering with the build

[chuckx]

After taking another look at the logs you posted and doing a bit more digging, it looks like the issue is with the Bazel execution of the C++ compiler including system headers located in /usr/local/include.

Two potential options for solving the issue:

1. Use the --sandbox_block_path=/usr/local flag in your bazel build invocation
2. Remove the protobuf installation present in /usr/local/include/google/protobuf

For reference, see https://github.com/protocolbuffers/protobuf/issues/5376#issuecomment-1069780157 and https://github.com/bazelbuild/bazel/issues/8053#issuecomment-490838966.

[jyothiyzen]

That worked. Thank you for your help resolving this issue