search

huo-ju / photoprism-freebsd-port

The photoprism port for FreeBSD
BSD 2-Clause "Simplified" License
69 stars 15 forks source link

tensorflow compiling fails on FreeBSD 11.4-RELEASE #7

Closed abelbabel closed 2 years ago

abelbabel commented 3 years ago

When compiling photoprism-port in a FreeBSD 11.4-RELEASE-p5 jail (CPU without AV*-features), it fails in tensorflow-compiling-phase:

...
ERROR: /root/photoprism-freebsd-port/work/photoprism-6b0d94ff42fadccfe88c4d68744b9d3afea36e2f/docker/tensorflow/tensorflow-1.15.2/tensorflow/core/BUILD:2285:1: C++ compilation of rule '//tensorflow/c
ore:autotuning_proto_cc_impl' failed (Exit 1)
In file included from bazel-out/host/bin/tensorflow/core/protobuf/autotuning.pb.cc:4:                                                                                                                  
bazel-out/host/bin/tensorflow/core/protobuf/autotuning.pb.h:234:34: error: expected member name or ';' after declaration specifiers
  ::PROTOBUF_NAMESPACE_ID::int32 major() const;                                                    
  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ^                                                                 
/usr/include/sys/types.h:372:20: note: expanded from macro 'major'                                                                                                                                     
#define major(x)        ((int)(((u_int)(x) >> 8)&0xff)) /* major number */                                                                                                                             
                          ^                                                                        
In file included from bazel-out/host/bin/tensorflow/core/protobuf/autotuning.pb.cc:4:
bazel-out/host/bin/tensorflow/core/protobuf/autotuning.pb.h:234:34: error: expected ')'
/usr/include/sys/types.h:372:20: note: expanded from macro 'major'
#define major(x)        ((int)(((u_int)(x) >> 8)&0xff)) /* major number */
                          ^
bazel-out/host/bin/tensorflow/core/protobuf/autotuning.pb.h:234:34: note: to match this '('
/usr/include/sys/types.h:372:19: note: expanded from macro 'major'
#define major(x)        ((int)(((u_int)(x) >> 8)&0xff)) /* major number */
                         ^
In file included from bazel-out/host/bin/tensorflow/core/protobuf/autotuning.pb.cc:4:
bazel-out/host/bin/tensorflow/core/protobuf/autotuning.pb.h:234:34: error: expected ')'
  ::PROTOBUF_NAMESPACE_ID::int32 major() const;
                                 ^
/usr/include/sys/types.h:372:37: note: expanded from macro 'major'
#define major(x)        ((int)(((u_int)(x) >> 8)&0xff)) /* major number */
                                           ^
bazel-out/host/bin/tensorflow/core/protobuf/autotuning.pb.h:234:34: note: to match this '('
/usr/include/sys/types.h:372:25: note: expanded from macro 'major'
#define major(x)        ((int)(((u_int)(x) >> 8)&0xff)) /* major number */
                               ^
In file included from bazel-out/host/bin/tensorflow/core/protobuf/autotuning.pb.cc:4:
bazel-out/host/bin/tensorflow/core/protobuf/autotuning.pb.h:234:34: error: C++ requires a type specifier for all declarations
  ::PROTOBUF_NAMESPACE_ID::int32 major() const;
                                 ^
/usr/include/sys/types.h:372:27: note: expanded from macro 'major'
#define major(x)        ((int)(((u_int)(x) >> 8)&0xff)) /* major number */
                                 ^
In file included from bazel-out/host/bin/tensorflow/core/protobuf/autotuning.pb.cc:4:
bazel-out/host/bin/tensorflow/core/protobuf/autotuning.pb.h:234:34: error: expected ')'
/usr/include/sys/types.h:372:42: note: expanded from macro 'major'
#define major(x)        ((int)(((u_int)(x) >> 8)&0xff)) /* major number */
                                                ^
bazel-out/host/bin/tensorflow/core/protobuf/autotuning.pb.h:234:34: note: to match this '('
/usr/include/sys/types.h:372:24: note: expanded from macro 'major'
#define major(x)        ((int)(((u_int)(x) >> 8)&0xff)) /* major number */
                              ^
...
In file included from bazel-out/host/bin/tensorflow/core/protobuf/autotuning.pb.cc:4:
bazel-out/host/bin/tensorflow/core/protobuf/autotuning.pb.h:380:34: error: non-member function cannot have '&' qualifier
/usr/include/sys/types.h:373:28: note: expanded from macro 'minor'
#define minor(x)        ((int)((x)&0xffff00ff))         /* minor number */
                                  ^
fatal error: too many errors emitted, stopping now [-ferror-limit=]
20 errors generated.
Target //tensorflow:libtensorflow.so failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 4207.232s, Critical Path: 185.05s
INFO: 781 processes: 781 local.
FAILED: Build did NOT complete successfully
*** Error code 1

Stop.
make[1]: stopped in /root/photoprism-freebsd-port
*** Error code 1

Stop.
make: stopped in /root/photoprism-freebsd-port

Seems similar to this PR at first glance ... and there is a patching approach

huo-ju commented 3 years ago

Thanks for the information. I have merged the patch into the port e7e2199d290fab15023c0ccce47894d20586518e and it's only applied to FreeBSD 11.

abelbabel commented 3 years ago

Does not work. Got

2021-01-30 12:40:14 (1.00 MB/s) - 'v1.15.2.tar.gz' saved [46196945]

tar -xzf v1.15.2.tar.gz
cp .tf_configure.bazelrc Makefile *.sh tensorflow-1.15.2
gmake[2]: Leaving directory '/root/photoprism-freebsd-port/work/photoprism-6b0d94ff42fadccfe88c4d68744b9d3afea36e2f/docker/tensorflow'
===>  Patching for photoprism-g20210111
===>  Applying extra patch /root/photoprism-freebsd-port/files/patch-docker_tensorflow_tensorflow-1.15.2_tensorflow_core_protobuf_autotuning.proto
===>  Applying FreeBSD patches for photoprism-g20210111 from /root/photoprism-freebsd-port/files
Ignoring previously applied (or reversed) patch.
1 out of 1 hunks ignored--saving rejects to docker/tensorflow/tensorflow-1.15.2/tensorflow/core/protobuf/autotuning.proto.rej
===>  FAILED Applying FreeBSD patch-docker_tensorflow_tensorflow-1.15.2_tensorflow_core_protobuf_autotuning.proto
===> Cleanly applied FreeBSD patch(es)  patch-docker_tensorflow_Makefile
===> FAILED to apply cleanly FreeBSD patch(es)  patch-docker_tensorflow_tensorflow-1.15.2_tensorflow_core_protobuf_autotuning.proto
*** Error code 1

Stop.
make[1]: stopped in /root/photoprism-freebsd-port
*** Error code 1

Stop.
make: stopped in /root/photoprism-freebsd-port
abelbabel commented 3 years ago

it seems that the patch-file has to have prefix: extra-patch- to prevent multiple patching

this means: extra-patch-docker_tensorflow_tensorflow-1.15.2_tensorflow_core_protobuf_autotuning.proto instead of patch-docker_tensorflow_tensorflow-1.15.2_tensorflow_core_protobuf_autotuning.proto

huo-ju commented 3 years ago

Thanks, I have updated the file and Makefile. It should work now. c22b75f8f726b08ddd4fec9f9a40864ea050218a

abelbabel commented 3 years ago

Thank you!

But there are still problems, other problems ...:

...
ERROR: /root/photoprism-freebsd-port/work/photoprism-6b0d94ff42fadccfe88c4d68744b9d3afea36e2f/docker/tensorflow/tensorflow-1.15.2/tensorflow/BUILD:563:1: Linking of rule '//tensorflow:libtensorflow_framework.so.1.15.2' failed (Exit 1)     
bazel-out/host/bin/external/zlib_archive/libzlib.pic.a(adler32.pic.o): In function `adler32':                                                                                                                                                  
adler32.c:(.text.adler32+0x0): multiple definition of `adler32'                                                                                                                                                                                
bazel-out/host/bin/external/zlib/libzlib.pic.a(adler32.pic.o):adler32.c:(.text.adler32+0x0): first defined here                                                                                                                                
bazel-out/host/bin/external/zlib_archive/libzlib.pic.a(adler32.pic.o): In function `adler32_combine':                                                                                                  
adler32.c:(.text.adler32_combine+0x0): multiple definition of `adler32_combine'                                                                                                                                                                bazel-out/host/bin/external/zlib/libzlib.pic.a(adler32.pic.o):adler32.c:(.text.adler32_combine+0x0): first defined here                                                                                                                        
bazel-out/host/bin/external/zlib_archive/libzlib.pic.a(adler32.pic.o): In function `adler32_combine64':                                                                                                                                        adler32.c:(.text.adler32_combine64+0x0): multiple definition of `adler32_combine64'                                                                            
bazel-out/host/bin/external/zlib/libzlib.pic.a(adler32.pic.o):adler32.c:(.text.adler32_combine64+0x0): first defined here
bazel-out/host/bin/external/zlib_archive/libzlib.pic.a(adler32.pic.o): In function `adler32_z':                                                                                                        
adler32.c:(.text.adler32_z+0x0): multiple definition of `adler32_z'                                                                                                                                                                            bazel-out/host/bin/external/zlib/libzlib.pic.a(adler32.pic.o):adler32.c:(.text.adler32_z+0x0): first defined here                                                                                                                              bazel-out/host/bin/external/zlib_archive/libzlib.pic.a(compress.pic.o): In function `compress':                        
compress.c:(.text.compress+0x0): multiple definition of `compress'                                                     
bazel-out/host/bin/external/zlib/libzlib.pic.a(compress.pic.o):compress.c:(.text.compress+0x0): first defined here      
bazel-out/host/bin/external/zlib_archive/libzlib.pic.a(compress.pic.o): In function `compress2':                                                                                                                                               compress.c:(.text.compress2+0x0): multiple definition of `compress2'
...
bazel-out/host/bin/external/zlib_archive/libzlib.pic.a(zutil.pic.o): In function `zcalloc':                            
zutil.c:(.text.zcalloc+0x0): multiple definition of `zcalloc'                                                          
bazel-out/host/bin/external/zlib/libzlib.pic.a(zutil.pic.o):zutil.c:(.text.zcalloc+0x0): first defined here                                                                                                                                    
bazel-out/host/bin/external/zlib_archive/libzlib.pic.a(zutil.pic.o): In function `zcfree':                             
zutil.c:(.text.zcfree+0x0): multiple definition of `zcfree'                                                            
bazel-out/host/bin/external/zlib/libzlib.pic.a(zutil.pic.o):zutil.c:(.text.zcfree+0x0): first defined here                                                                                                                                     
bazel-out/host/bin/external/zlib_archive/libzlib.pic.a(zutil.pic.o): In function `zlibCompileFlags':                                                                                                                                           
zutil.c:(.text.zlibCompileFlags+0x0): multiple definition of `zlibCompileFlags'                                        
bazel-out/host/bin/external/zlib/libzlib.pic.a(zutil.pic.o):zutil.c:(.text.zlibCompileFlags+0x0): first defined here                                                                                                                           
bazel-out/host/bin/external/zlib_archive/libzlib.pic.a(zutil.pic.o): In function `zlibVersion':                        
zutil.c:(.text.zlibVersion+0x0): multiple definition of `zlibVersion'                                                  
bazel-out/host/bin/external/zlib/libzlib.pic.a(zutil.pic.o):zutil.c:(.text.zlibVersion+0x0): first defined here                                                                                                                                
clang: error: linker command failed with exit code 1 (use -v to see invocation)                                        
Target //tensorflow:libtensorflow.so failed to build                                                                   
Use --verbose_failures to see the command lines of failed build steps.                                                 
INFO: Elapsed time: 12204.121s, Critical Path: 365.29s                                                                 
INFO: 4320 processes: 4320 local.                          
FAILED: Build did NOT complete successfully                
*** Error code 1                                           

Stop.                                                      
make[1]: stopped in /root/photoprism-freebsd-port                                                                      
*** Error code 1                                           

Stop.                                                      
make: stopped in /root/photoprism-freebsd-port

Seems to be the problem that zlib is built multiple times. There was a similar problem ...

huo-ju commented 3 years ago

It's a little wired, and I can't reproduce it on my system (12-RELEASE).

Looks there are some bugs in the tensorflow bazel's build script? I still have no idea on it.

abelbabel commented 3 years ago

so please reopen this issue until it's not solved to keep in mind that there is a problem on 11.4-RELEASE

huo-ju commented 3 years ago

I can reopen this issue, but this error is belongs to tensorflow project.