Getting build error , Host dl.bintray.com:443 was resolved.

[SHUFIL]

Bug Report

Please answer these questions before submitting your issue. Thanks!

1. What did you do? Installing Apache-pegasus in ubuntu 22, using Compile method , doc. Also I tried to install using docker method as well , but got error ERROR: build rdsn failed

   error: downloading 'https://dl.bintray.com/boostorg/release/1.69.0/source/boost_1_69_0.tar.bz2' failed
         status_code: 22
         status_string: "HTTP response code said error"

I have installed below dependence's gcc version 11.4.0 (Ubuntu 11.4.0-1ubuntu1~22.04) , cmake version 3.30.0

also installed all packages in mentioned Dockerfile Dockerfile

2. What did you expect to see? Both method I am getting build issue so need to resolve Build issue
3. What did you see instead? Getting below error while doing build Compile method

CMake Error at /home/ubuntu/pegasus/rdsn/thirdparty/build/Stamp/boost/download-boost.cmake:163 (message):
  Each download failed!

    error: downloading 'http://pegasus-thirdparties.oss-cn-beijing.aliyuncs.com/boost_1_69_0.tar.bz2' failed
          status_code: 22
          status_string: "HTTP response code said error"
          log:
          --- LOG BEGIN ---
          timeout on name lookup is not supported

  Host pegasus-thirdparties.oss-cn-beijing.aliyuncs.com:80 was resolved.

  IPv6: (none)

  IPv4: 59.110.190.37

    Trying 59.110.190.37:80...

  Connected to pegasus-thirdparties.oss-cn-beijing.aliyuncs.com
  (59.110.190.37) port 80

  GET /boost_1_69_0.tar.bz2 HTTP/1.1

  Host: pegasus-thirdparties.oss-cn-beijing.aliyuncs.com

  User-Agent: curl/8.8.0

  Accept: */*

  Request completely sent off

  HTTP/1.1 403 Forbidden

  Server: AliyunOSS

  Date: Tue, 09 Jul 2024 07:48:13 GMT

  Content-Type: application/xml

  Content-Length: 344

  Connection: keep-alive

  x-oss-request-id: 668CEB3D35EB2634329C18EE

  x-oss-ec: 0003-00000905

  The requested URL returned error: 403

  Closing connection

          --- LOG END ---
          error: downloading 'https://dl.bintray.com/boostorg/release/1.69.0/source/boost_1_69_0.tar.bz2' failed
          status_code: 22
          status_string: "HTTP response code said error"
          log:
          --- LOG BEGIN ---
          timeout on name lookup is not supported

  Host dl.bintray.com:443 was resolved.

  IPv6: (none)

  IPv4: 18.214.194.113, 18.232.172.199, 3.95.117.170

    Trying 18.214.194.113:443...

  Connected to dl.bintray.com (18.214.194.113) port 443

  ALPN: curl offers h2,http/1.1

  [5 bytes data]

  TLSv1.3 (OUT), TLS handshake, Client hello (1):

  [512 bytes data]

  [5 bytes data]

  TLSv1.3 (IN), TLS handshake, Server hello (2):

  [112 bytes data]

  [5 bytes data]

  TLSv1.2 (IN), TLS handshake, Certificate (11):

  [3676 bytes data]

  [5 bytes data]

  TLSv1.2 (IN), TLS handshake, Server key exchange (12):

  [300 bytes data]

  [5 bytes data]

  TLSv1.2 (IN), TLS handshake, Server finished (14):

  [4 bytes data]

  [5 bytes data]

  TLSv1.2 (OUT), TLS handshake, Client key exchange (16):

  [37 bytes data]

  [5 bytes data]

  TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):

  [1 bytes data]

  [5 bytes data]

  TLSv1.2 (OUT), TLS handshake, Finished (20):

  [16 bytes data]

  [5 bytes data]

  [5 bytes data]

  TLSv1.2 (IN), TLS handshake, Finished (20):

  [16 bytes data]

  SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384 / X25519 /
  RSASSA-PSS

  ALPN: server accepted http/1.1

  Server certificate:

   subject: CN=*.bintray.com
   start date: Oct 26 00:00:00 2023 GMT
   expire date: Nov 25 23:59:59 2024 GMT
   subjectAltName: host "dl.bintray.com" matched cert's "*.bintray.com"
   issuer: C=US; O=DigiCert Inc; OU=www.digicert.com; CN=GeoTrust TLS RSA CA G1
   SSL certificate verify result: self-signed certificate in certificate chain (19), continuing anyway.
    Certificate level 0: Public key type RSA (2048/112 Bits/secBits), signed using sha256WithRSAEncryption
    Certificate level 1: Public key type RSA (2048/112 Bits/secBits), signed using sha256WithRSAEncryption
    Certificate level 2: Public key type RSA (2048/112 Bits/secBits), signed using sha256WithRSAEncryption

  using HTTP/1.x

  [5 bytes data]

  GET /boostorg/release/1.69.0/source/boost_1_69_0.tar.bz2 HTTP/1.1

  Host: dl.bintray.com

  User-Agent: curl/8.8.0

  Accept: */*

  Request completely sent off

  [5 bytes data]

  HTTP/1.1 404 Not Found

  Date: Tue, 09 Jul 2024 07:48:13 GMT

  Content-Type: text/html

  Content-Length: 146

  Connection: keep-alive

  Strict-Transport-Security: max-age=31536000; includeSubDomains

  The requested URL returned error: 404

  Closing connection

  [5 bytes data]

  TLSv1.2 (OUT), TLS alert, close notify (256):

  [2 bytes data]

          --- LOG END ---

make[2]: *** [CMakeFiles/boost.dir/build.make:99: Stamp/boost/boost-download] Error 1
make[1]: *** [CMakeFiles/Makefile2:119: CMakeFiles/boost.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
-- extracting...
     src='/home/ubuntu/pegasus/rdsn/thirdparty/build/Download/concurrentqueue/concurrentqueue-1.0.1.tar.gz'
     dst='/home/ubuntu/pegasus/rdsn/thirdparty/build/Source/concurrentqueue'
-- extracting... [tar xfz]
-- extracting... [analysis]
-- extracting... [rename]
-- extracting... [clean up]
-- extracting... done
[  3%] No update step for 'concurrentqueue'
[  4%] No patch step for 'concurrentqueue'
[  5%] No configure step for 'concurrentqueue'
[  5%] No build step for 'concurrentqueue'
[  5%] Performing install step for 'concurrentqueue'
[  6%] Completed 'concurrentqueue'
[  6%] Built target concurrentqueue
make: *** [Makefile:91: all] Error 2
ERROR: build rdsn failed

4. What version of Pegasus are you using? apache-pegasus-2.1.0

[acelyc111]

Hi, @SHUFIL

Thanks for the reply!

error: downloading 'https://dl.bintray.com/boostorg/release/1.69.0/source/boost_1_69_0.tar.bz2' failed
          status_code: 22
          status_string: "HTTP response code said error"

or

error: downloading 'http://pegasus-thirdparties.oss-cn-beijing.aliyuncs.com/boost_1_69_0.tar.bz2' failed
          status_code: 22
          status_string: "HTTP response code said error"

Some thirdparty links are broken in old versions. You can Update OSS_URL_PREFIX value in rdsn/thirdparty/CMakeLists.txt from http://pegasus-thirdparties.oss-cn-beijing.aliyuncs.com to http://pegasus-thirdparty-package.oss-cn-beijing.aliyuncs.com if you insist on Pegasus 2.1.

Or

Try to use new Pegasus versions (say https://github.com/apache/incubator-pegasus/releases/tag/v2.5.0) instead.

[SHUFIL]

Hey @acelyc111 , Thanks for reply.

I tried 2.5.0 both way compile and docker , unfortunately I got below error , This is for compiled way.

This time I did not get any reason , just though out a error , that is

-- Installing: /home/ubuntu/apache-pegasus-2.5.0/thirdparty/output/include/thrift/config.h
[ 29%] Completed 'thrift'
[ 29%] Built target thrift
make: *** [Makefile:91: all] Error 2

-- Installing: /root/pegasus/thirdparty/output/bin/arc
-- Set non-toolchain portion of runtime path of "/root/pegasus/thirdparty/output/bin/arc" to ""
[ 94%] Completed 'poco'
[ 94%] Built target poco
make: *** [Makefile:91: all] Error 2

This is docker method , can you please check that error . Thanks

[SHUFIL]

@acelyc111 @ke4qqq @gmcdonald

[acelyc111]

Hey @acelyc111 , Thanks for reply.

I tried 2.5.0 both way compile and docker , unfortunately I got below error , This is for compiled way.

This time I did not get any reason , just though out a error , that is

-- Installing: /home/ubuntu/apache-pegasus-2.5.0/thirdparty/output/include/thrift/config.h
[ 29%] Completed 'thrift'
[ 29%] Built target thrift
make: *** [Makefile:91: all] Error 2

-- Installing: /root/pegasus/thirdparty/output/bin/arc
-- Set non-toolchain portion of runtime path of "/root/pegasus/thirdparty/output/bin/arc" to ""
[ 94%] Completed 'poco'
[ 94%] Built target poco
make: *** [Makefile:91: all] Error 2

This is docker method , can you please check that error . Thanks

@SHUFIL Did you see any other error messages?

I try to reproduce it according to the doc https://github.com/apache/incubator-pegasus-website/blob/master/_docs/en/2.5.0/compile-by-docker.md, but everything goes well. My steps are:

1. git clone git@github.com:apache/incubator-pegasus.git pegasus_2.5
2. cd pegasus_2.5 && git checkout v2.5
3. pwd
4. docker run -v /home/laiyingchun/dev/pegasus_2.5:/root/pegasus apache/pegasus:build-env-ubuntu2004-v2.5 /bin/bash -c "cd /root/pegasus; ./run.sh build --test -c --clear_thirdparty -j 32"
5. Built OK as:

   
   ...
   -- Installing: /root/pegasus/build/release/output/lib/libdsn.replication.tool.a
   -- Installing: /root/pegasus/build/release/output/lib/libdsn_utils.so
   -- Set runtime path of "/root/pegasus/build/release/output/lib/libdsn_utils.so" to ""
   -- Installing: /root/pegasus/build/release/output/bin/long_adder_bench/long_adder_bench
   -- Set runtime path of "/root/pegasus/build/release/output/bin/long_adder_bench/long_adder_bench" to ""
   -- Installing: /root/pegasus/build/release/output/bin/nth_element_bench/nth_element_bench
   -- Set runtime path of "/root/pegasus/build/release/output/bin/nth_element_bench/nth_element_bench" to ""
   -- Installing: /root/pegasus/build/release/output/lib/libdsn.replication.zookeeper_provider.a
   Build finish time: Wed Jul 10 16:28:26 CST 2024
   Build elapsed time: 16m 58s

laiyingchun @ localhost in ~/dev/pegasus_2.5 on git:v2.5 o [16:28:38]

[acelyc111]

I also checked the apache/pegasus:build-env-ubuntu2204-v2.5 image, it's OK too.

[SHUFIL]

@acelyc111 I dont know why it is getting me , is it is any dependency issue ? I have tried the same version docker image but getting below error . I have used below commands for run docker ,

sudo docker run -v /home/ubuntu/apache-pegasus-2.5.0:/root/pegasus apache/pegasus:build-env-ubuntu2004-master /bin/bash -c "cd /root/pegasus; ./run.sh build -c --clear_thirdparty -j $(nproc)"

Error

[ 85%] Building CXX object src/replica/CMakeFiles/dsn_replica_server.dir/duplication/replica_duplicator_manager.cpp.o
make[2]: *** [src/replica/CMakeFiles/dsn_replica_server.dir/build.make:384: src/replica/CMakeFiles/dsn_replica_server.dir/replica_stub.cpp.o] Error 1
make[2]: *** Waiting for unfinished jobs....
make[1]: *** [CMakeFiles/Makefile2:1967: src/replica/CMakeFiles/dsn_replica_server.dir/all] Error 2
make: *** [Makefile:136: all] Error 2

[acelyc111]

@SHUFIL I guess some important error message is missed, could you please try:

1. Use the corresponding docker image, that is to say, use apache/pegasus:build-env-ubuntu2204-v2.5 not apache/pegasus:build-env-ubuntu2004-master, because some software packages are different in different version.
2. Set the build parallelism to 1 instaed of $(nproc).

So run as:

sudo docker run -v /home/ubuntu/apache-pegasus-2.5.0:/root/pegasus apache/pegasus:build-env-ubuntu2004-v2.5 /bin/bash -c "cd /root/pegasus; ./run.sh build -c --clear_thirdparty -j 1"

[SHUFIL]

This is worked for me , and build finished without error. Thanks

[SHUFIL]

Next I need to run test command, for this I run below command ,

docker run -v /home/ubuntu/pegasus:/root/pegasus \
           apache/pegasus:build-env-ubuntu2004-v2.5 \
           /bin/bash -c "cd /root/pegasus; ./run.sh test"

But I got below error ,

Cluster becomes healthy.
sed: can't read /root/pegasus/build/latest/src/server/test/config.ini: No such file or directory

[acelyc111]

Next I need to run test command, for this I run below command ,

docker run -v /home/ubuntu/pegasus:/root/pegasus \
           apache/pegasus:build-env-ubuntu2004-v2.5 \
           /bin/bash -c "cd /root/pegasus; ./run.sh test"

But I got below error ,

Cluster becomes healthy.
sed: can't read /root/pegasus/build/latest/src/server/test/config.ini: No such file or directory

I see, it's needed to add --test when build the project by ./run.sh build ... if you want to run tests.

Please try this at first:

sudo docker run -v /home/ubuntu/apache-pegasus-2.5.0:/root/pegasus apache/pegasus:build-env-ubuntu2004-v2.5 /bin/bash -c "cd /root/pegasus; ./run.sh build -c --clear_thirdparty -j 1 --test"

[acelyc111]

Please keep in mind that some tests maybe flaky, don't be worry about that 😄 .

[SHUFIL]

Ok, that is fine, but I did not get any error this time, I have completed pack_server , pack_client (C/C++) ,and pack_tools,

As per this doc it is explain for pack_client C/C++ development , I need to run for Golang client , for this I need to run client separately or here I can do that ? how it is

[acelyc111]

Congrats!

For Golang client, please see docs here: https://github.com/apache/incubator-pegasus/tree/master/go-client

[SHUFIL]

I have clone again for go-client, but it is already there in main folder , anyway I tried in both folder with make build command , but I getting always below error . For resolving this issue I have done below steps added this values

ulimit -c unlimited
export GOBACTRACE=crash
thrift --version
Thrift version 0.16.0
 go version
go version go1.22.0 linux/386

Error

go mod tidy
go mod verify
all modules verified
go build -o ./bin/echo ./rpc/main/echo.go
# runtime/cgo
In file included from _cgo_export.c:3:
/usr/include/stdlib.h:26:10: fatal error: bits/libc-header-start.h: No such file or directory
   26 | #include <bits/libc-header-start.h>
      |          ^~~~~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
make: *** [Makefile:34: build] Error 1

[acelyc111]

@SHUFIL According to the go-client CI job, it works well on certain thrift and golang versions. Could you try thrift 0.13.0 and golang-1.14?

[SHUFIL]

ok, I think this will check later , because I getting thrift-0.13.0 build error

./configure: line 19838: syntax error near unexpected token `QT5,'
./configure: line 19838: `    PKG_CHECK_MODULES(QT5, Qt5Core >= 5.0, Qt5Network >= 5.0,'

I am trying to start pegasus service after build, but below error blocking me, Can you please.

./run.sh start_onebox ERROR: file /home/ubuntu/pegasus/build/latest/output/bin/pegasus_server/pegasus_server not exist

[acelyc111]

./run.sh start_onebox ERROR: file /home/ubuntu/pegasus/build/latest/output/bin/pegasus_server/pegasus_server not exist

@SHUFIL Has the Pegasus been successfully built before starting the onebox? Are you starting the onebox in the docker?