Build failure when building the docker images

[kasundharmadasa]

Hi All,

I'm trying to build the docker images in https://github.com/IntelAI/he-transformer/tree/master/contrib/docker on an Ubuntu 16.04 machine. When I run the command make check_gcc, it gives the following error.

Building docker image build_he_transformer_ubuntu1604:2020-05-31T12-42-17+05-30 from Dockerfile /home/kasun/Documents/MSc/Research/he-transformer/contrib/docker/.build-49879416a5b659edc1ca5b31493bc659685ae483_2_ubuntu1604/Dockerfile.build_he_transformer_ubuntu1604, context .

invalid argument "build_he_transformer_ubuntu1604:2020-05-31T12-42-17+05-30" for t=build_he_transformer_ubuntu1604:2020-05-31T12-42-17+05-30: Error parsing reference: "build_he_transformer_ubuntu1604:2020-05-31T12-42-17+05-30" is not a valid repository/tag: invalid reference format
See 'docker build --help'.
Makefile:141: recipe for target 'build_docker_image' failed
make: *** [build_docker_image] Error 125

Any idea on a possible cause for this?

[pjattke]

The Docker image generation script (contrib/docker/make-dimage.sh) uses invalid characters in the Docker image tag (e.g., colon).

You can manually edit the following line:

DIMAGE_VERSION=`date -Iseconds | sed -e 's/:/-/g'`

in make-dimage.sh by replacing the command with, for example, date +%s (unix timestamp):

DIMAGE_VERSION=`date +%s`

[pjattke]

I noted that after this change, building he-transformer still fails when calling make build_clang OS_ID=ubuntu1804:

ERROR: /home/dockuser/he-transformer-test/BUILD-CLANG/ext_ngraph_tf/src/ext_ngraph_tf/build_cmake/tensorflow/tensorflow/python/BUILD:336:1: C++ compilation of rule '//tensorflow/python:bfloat16_lib' failed (Exit 1)
tensorflow/python/lib/core/bfloat16.cc: In function 'bool tensorflow::{anonymous}::Initialize()':
tensorflow/python/lib/core/bfloat16.cc:634:36: error: no match for call to '(tensorflow::{anonymous}::Initialize()::<lambda(const char*, PyUFuncGenericFunction, const std::array<int, 3>&)>) (const char [6], <unresolved overloaded function type>, const std::array<int, 3>&)'
                       compare_types)) {
                                    ^
tensorflow/python/lib/core/bfloat16.cc:608:60: note: candidate: tensorflow::{anonymous}::Initialize()::<lambda(const char*, PyUFuncGenericFunction, const std::array<int, 3>&)>
                             const std::array<int, 3>& types) {
                                                            ^
tensorflow/python/lib/core/bfloat16.cc:608:60: note:   no known conversion for argument 2 from '<unresolved overloaded function type>' to 'PyUFuncGenericFunction {aka void (*)(char**, const long int*, const long int*, void*)}'
tensorflow/python/lib/core/bfloat16.cc:638:36: error: no match for call to '(tensorflow::{anonymous}::Initialize()::<lambda(const char*, PyUFuncGenericFunction, const std::array<int, 3>&)>) (const char [10], <unresolved overloaded function type>, const std::array<int, 3>&)'
                       compare_types)) {
                                    ^
tensorflow/python/lib/core/bfloat16.cc:608:60: note: candidate: tensorflow::{anonymous}::Initialize()::<lambda(const char*, PyUFuncGenericFunction, const std::array<int, 3>&)>
                             const std::array<int, 3>& types) {
                                                            ^
tensorflow/python/lib/core/bfloat16.cc:608:60: note:   no known conversion for argument 2 from '<unresolved overloaded function type>' to 'PyUFuncGenericFunction {aka void (*)(char**, const long int*, const long int*, void*)}'
tensorflow/python/lib/core/bfloat16.cc:641:77: error: no match for call to '(tensorflow::{anonymous}::Initialize()::<lambda(const char*, PyUFuncGenericFunction, const std::array<int, 3>&)>) (const char [5], <unresolved overloaded function type>, const std::array<int, 3>&)'
   if (!register_ufunc("less", CompareUFunc<Bfloat16LtFunctor>, compare_types)) {
                                                                             ^
tensorflow/python/lib/core/bfloat16.cc:608:60: note: candidate: tensorflow::{anonymous}::Initialize()::<lambda(const char*, PyUFuncGenericFunction, const std::array<int, 3>&)>
                             const std::array<int, 3>& types) {
                                                            ^
tensorflow/python/lib/core/bfloat16.cc:608:60: note:   no known conversion for argument 2 from '<unresolved overloaded function type>' to 'PyUFuncGenericFunction {aka void (*)(char**, const long int*, const long int*, void*)}'
tensorflow/python/lib/core/bfloat16.cc:645:36: error: no match for call to '(tensorflow::{anonymous}::Initialize()::<lambda(const char*, PyUFuncGenericFunction, const std::array<int, 3>&)>) (const char [8], <unresolved overloaded function type>, const std::array<int, 3>&)'
                       compare_types)) {
                                    ^
tensorflow/python/lib/core/bfloat16.cc:608:60: note: candidate: tensorflow::{anonymous}::Initialize()::<lambda(const char*, PyUFuncGenericFunction, const std::array<int, 3>&)>
                             const std::array<int, 3>& types) {
                                                            ^
tensorflow/python/lib/core/bfloat16.cc:608:60: note:   no known conversion for argument 2 from '<unresolved overloaded function type>' to 'PyUFuncGenericFunction {aka void (*)(char**, const long int*, const long int*, void*)}'
tensorflow/python/lib/core/bfloat16.cc:649:36: error: no match for call to '(tensorflow::{anonymous}::Initialize()::<lambda(const char*, PyUFuncGenericFunction, const std::array<int, 3>&)>) (const char [11], <unresolved overloaded function type>, const std::array<int, 3>&)'
                       compare_types)) {
                                    ^
tensorflow/python/lib/core/bfloat16.cc:608:60: note: candidate: tensorflow::{anonymous}::Initialize()::<lambda(const char*, PyUFuncGenericFunction, const std::array<int, 3>&)>
                             const std::array<int, 3>& types) {
                                                            ^
tensorflow/python/lib/core/bfloat16.cc:608:60: note:   no known conversion for argument 2 from '<unresolved overloaded function type>' to 'PyUFuncGenericFunction {aka void (*)(char**, const long int*, const long int*, void*)}'
tensorflow/python/lib/core/bfloat16.cc:653:36: error: no match for call to '(tensorflow::{anonymous}::Initialize()::<lambda(const char*, PyUFuncGenericFunction, const std::array<int, 3>&)>) (const char [14], <unresolved overloaded function type>, const std::array<int, 3>&)'
                       compare_types)) {
                                    ^
tensorflow/python/lib/core/bfloat16.cc:608:60: note: candidate: tensorflow::{anonymous}::Initialize()::<lambda(const char*, PyUFuncGenericFunction, const std::array<int, 3>&)>
                             const std::array<int, 3>& types) {
                                                            ^
tensorflow/python/lib/core/bfloat16.cc:608:60: note:   no known conversion for argument 2 from '<unresolved overloaded function type>' to 'PyUFuncGenericFunction {aka void (*)(char**, const long int*, const long int*, void*)}'
Target //tensorflow/tools/pip_package:build_pip_package failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 437.714s, Critical Path: 223.13s
INFO: 7974 processes: 7974 local.
FAILED: Build did NOT complete successfully
FAILED: Build did NOT complete successfully
ARTIFACTS location: /home/dockuser/he-transformer-test/BUILD-CLANG/ext_ngraph_tf/src/ext_ngraph_tf/build_cmake/artifacts
Loading virtual environment from: /home/dockuser/he-transformer-test/BUILD-CLANG/ext_ngraph_tf/src/ext_ngraph_tf/build_cmake/venv-tf-py3
Loading virtual environment from: /home/dockuser/he-transformer-test/BUILD-CLANG/ext_ngraph_tf/src/ext_ngraph_tf/build_cmake/venv-tf-py3
PIP location
Target Arch: native
Building TensorFlow from source
PYTHON_BIN_PATH: /home/dockuser/he-transformer-test/BUILD-CLANG/ext_ngraph_tf/src/ext_ngraph_tf/build_cmake/venv-tf-py3/bin/python
SOURCE DIR: /home/dockuser/he-transformer-test/BUILD-CLANG/ext_ngraph_tf/src/ext_ngraph_tf/build_cmake/tensorflow
ARTIFACTS DIR: /home/dockuser/he-transformer-test/BUILD-CLANG/ext_ngraph_tf/src/ext_ngraph_tf/build_cmake/artifacts/tensorflow
Traceback (most recent call last):
  File "/home/dockuser/he-transformer-test/BUILD-CLANG/ext_ngraph_tf/src/ext_ngraph_tf/build_ngtf.py", line 525, in <module>
    main()
  File "/home/dockuser/he-transformer-test/BUILD-CLANG/ext_ngraph_tf/src/ext_ngraph_tf/build_ngtf.py", line 328, in main
    target_arch, verbosity)
  File "/home/dockuser/he-transformer-test/BUILD-CLANG/ext_ngraph_tf/src/ext_ngraph_tf/tools/build_utils.py", line 231, in build_tensorflow
    command_executor(cmd)
  File "/home/dockuser/he-transformer-test/BUILD-CLANG/ext_ngraph_tf/src/ext_ngraph_tf/tools/build_utils.py", line 60, in command_executor
    raise Exception("Error running command: " + cmd)
Exception: Error running command: bazel build --config=opt --config=noaws --config=nohdfs --config=noignite --config=nokafka --config=nonccl //tensorflow/tools/pip_package:build_pip_package
CMakeFiles/ext_ngraph_tf.dir/build.make:132: recipe for target 'ext_ngraph_tf/src/ext_ngraph_tf-stamp/ext_ngraph_tf-build' failed
make[2]: *** [ext_ngraph_tf/src/ext_ngraph_tf-stamp/ext_ngraph_tf-build] Error 1
make[2]: Leaving directory '/home/dockuser/he-transformer-test/BUILD-CLANG'
CMakeFiles/Makefile2:240: recipe for target 'CMakeFiles/ext_ngraph_tf.dir/all' failed
make[1]: *** [CMakeFiles/ext_ngraph_tf.dir/all] Error 2
make[1]: Leaving directory '/home/dockuser/he-transformer-test/BUILD-CLANG'
Makefile:148: recipe for target 'all' failed
make: *** [all] Error 2
make: *** [Makefile:177: build_clang] Error 2

Seems like the error is this one:

ERROR: /home/dockuser/he-transformer-test/BUILD-CLANG/ext_ngraph_tf/src/ext_ngraph_tf/build_cmake/tensorflow/tensorflow/python/BUILD:336:1: C++ compilation of rule '//tensorflow/python:bfloat16_lib' failed (Exit 1)

This is the same as mentioned in #45.