yugabyte / yugabyte-db

YugabyteDB - the cloud native distributed SQL database for mission-critical applications.
https://www.yugabyte.com
Other
9.01k stars 1.08k forks source link

[Test] Fail: (yb/integration-tests/external_mini_cluster.cc:1791): Key build_username is of invalid type 0 #5344

Open danwt opened 4 years ago

danwt commented 4 years ago

Jira Link: DB-4889 I'm trying to build and run tests for the first time and had a test fail. My environment is inside a Centos7 docker container with a volume forwarded from MacOS Catalina 10.15.5. I'm using Docker with 8GB memory and used the following setup.

docker pull centos:7
docker run -it centos:7

## Inside container
yum install apt-get
sudo yum update
sudo yum groupinstall -y 'Development Tools'
sudo yum install -y ruby perl-Digest epel-release ccache git python2-pip python-devel python3 python3-pip python3-devel
sudo yum install -y cmake3 ctest3
sudo ln -s /usr/bin/cmake3 /usr/local/bin/cmake
sudo ln -s /usr/bin/ctest3 /usr/local/bin/ctest

Then saved the image

docker commit <container-id> centos-yugabyte-dev-env

Create container with volume forwarded

docker run -ti -v $PWD/yugabyte-db:/code/yugabyte-db/ centos-yugabyte-dev-env /bin/bash
./yb_build.sh release --ctest

The build reaches 99% and then

[ 99%] Built target yb-ts-cli-test
Scanning dependencies of target initial_sys_catalog_snapshot
[gen_initial_sys_catalog_snapshot.py:55] 2020-08-08 09:50:36,398 INFO: Starting creating initial system catalog snapshot data
[gen_initial_sys_catalog_snapshot.py:56] 2020-08-08 09:50:36,399 INFO: Logging to: /code/yugabyte-db/build/release-gcc-dynamic/create_initial_sys_catalog_snapshot.err
[ 99%] Built target yb-admin-test_ent
[external_calls.py:337] 2020-08-08 09:50:41,774 ERROR: Non-zero exit code 1 from external program {{ '/code/yugabyte-db/build-support/run-test.sh' /code/yugabyte-db/build/release-gcc-dynamic/tests-pgwrapper/create_initial_sys_catalog_snapshot }} running in '/code/yugabyte-db/build/release-gcc-dynamic', saving stdout to {{ /code/yugabyte-db/build/release-gcc-dynamic/create_initial_sys_catalog_snapshot.out }}, stderr to {{ /code/yugabyte-db/build/release-gcc-dynamic/create_initial_sys_catalog_snapshot.err }}.
Standard output from external program {{ '/code/yugabyte-db/build-support/run-test.sh' /code/yugabyte-db/build/release-gcc-dynamic/tests-pgwrapper/create_initial_sys_catalog_snapshot }} running in '/code/yugabyte-db/build/release-gcc-dynamic', saving stdout to {{ /code/yugabyte-db/build/release-gcc-dynamic/create_initial_sys_catalog_snapshot.out }}, stderr to {{ /code/yugabyte-db/build/release-gcc-dynamic/create_initial_sys_catalog_snapshot.err }}:
Test is running on host 1364d3aaf253, arguments: /code/yugabyte-db/build/release-gcc-dynamic/tests-pgwrapper/create_initial_sys_catalog_snapshot
I0808 09:50:38.647032 92120 run-with-timeout.cc:134] Running on host 1364d3aaf253
I0808 09:50:38.647466 92120 run-with-timeout.cc:141] PATH environment variable: /opt/yb-build/brew/linuxbrew-20181203T161736v9/bin:/code/yugabyte-db/build/venv/bin:/code/yugabyte-db/build/venv/bin:/code/yugabyte-db/build/venv/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
I0808 09:50:38.647825 92120 run-with-timeout.cc:149] Starting child process with a timeout of 1201 seconds
I0808 09:50:38.650043 92120 run-with-timeout.cc:158] Child process pid: 92121
Note: Google Test filter = CreateInitialSysCatalogSnapshotTest.CreateInitialSysCatalogSnapshot
[==========] Running 1 test from 1 test case.
[----------] Global test environment set-up.
[----------] 1 test from CreateInitialSysCatalogSnapshotTest
[ RUN      ] CreateInitialSysCatalogSnapshotTest.CreateInitialSysCatalogSnapshot
I0808 09:50:39.779379 92121 test_util.cc:116] Environment variable ASAN_OPTIONS:
I0808 09:50:39.779747 92121 test_util.cc:116] Environment variable LSAN_OPTIONS:
I0808 09:50:39.779755 92121 test_util.cc:116] Environment variable UBSAN_OPTIONS:
I0808 09:50:39.779759 92121 test_util.cc:116] Environment variable TSAN_OPTIONS:
I0808 09:50:39.779816 92121 create_initial_sys_catalog_snapshot.cc:44] Creating initial system catalog snapshot at: /code/yugabyte-db/build/release-gcc-dynamic/share/initial_sys_catalog_snapshot
I0808 09:50:39.779917 92121 external_mini_cluster.cc:184] Env var YB_EXTRA_MASTER_FLAGS not specified, not setting extra flags from it
I0808 09:50:39.779945 92121 external_mini_cluster.cc:184] Env var YB_EXTRA_TSERVER_FLAGS not specified, not setting extra flags from it
I0808 09:50:39.781913 92121 mem_tracker.cc:249] MemTracker: hard memory limit is 8.000000 GB
I0808 09:50:39.782030 92121 mem_tracker.cc:251] MemTracker: soft memory limit is 6.800000 GB
I0808 09:50:39.793787 92121 thread_pool.cc:171] Starting thread pool { name: minicluster-messenger queue_limit: 10000 max_workers: 1024 }
I0808 09:50:39.800942 92121 external_mini_cluster.cc:337] Starting cluster with option bind_to_unique_loopback_addresses=true
I0808 09:50:39.800971 92121 external_mini_cluster.cc:340] Starting 1 masters
I0808 09:50:39.802480 92121 net_util.cc:556] Selected random free RPC port 28015
I0808 09:50:39.802723 92121 external_mini_cluster.cc:957] Using an auto-assigned port 28015 to start an external mini-cluster master
I0808 09:50:39.806772 92121 net_util.cc:556] Selected random free RPC port 29238
I0808 09:50:39.807071 92121 external_mini_cluster.cc:184] Env var YB_EXTRA_DAEMON_FLAGS not specified, not setting extra flags from it
I0808 09:50:39.807114 92121 external_mini_cluster.cc:1753] Running [m-1]: /code/yugabyte-db/build/release-gcc-dynamic/tests-pgwrapper/../bin/yb-master
yb-master
--fs_data_dirs=/tmp/yb_test.tmp.19923.29004.25197.pid92055/create_initial_sys_catalog_snapshot.CreateInitialSysCatalogSnapshotTest.CreateInitialSysCatalogSnapshot.1596880239639869-92121/minicluster-data/master-0
--rpc_bind_addresses=127.0.0.1:28015
--webserver_interface=localhost
--webserver_port=29238
--load_balancer_initial_delay_secs=0
--master_addresses=127.0.0.1:28015
--callhome_enabled=false
--metrics_log_interval_ms=0
--log_dir=
--enable_tracing
--memory_limit_hard_bytes=1073741824
--logtostderr
--hide_pg_catalog_table_creation_logs
--client_read_write_timeout_ms=120000
--memory_limit_hard_bytes=2147483648
--create_initial_sys_catalog_snapshot
--initial_sys_catalog_snapshot_path=/code/yugabyte-db/build/release-gcc-dynamic/share/initial_sys_catalog_snapshot
--master_auto_run_initdb
--enable_leader_failure_detection=true
--enable_ysql=true
--master_auto_run_initdb
--server_dump_info_path=/tmp/yb_test.tmp.19923.29004.25197.pid92055/create_initial_sys_catalog_snapshot.CreateInitialSysCatalogSnapshotTest.CreateInitialSysCatalogSnapshot.1596880239639869-92121/minicluster-data/master-0/yb-data/master/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--logbuflevel=-1
--metric_node_name=test_invocation_2020-08-08T09_50_36_31890_28416_92055
--fatal_details_path_prefix=/code/yugabyte-db/build/release-gcc-dynamic/yb-test-logs/tests-pgwrapper__create_initial_sys_catalog_snapshot/CreateInitialSysCatalogSnapshotTest_CreateInitialSysCatalogSnapshot.fatal_failure_details.m-1
--minicluster_daemon_id=m-1
[m-1] Illegal state (yb/util/version_info.cc:165): Key build_username is of invalid type 0
/code/yugabyte-db/src/yb/yql/pgwrapper/pg_wrapper_test_base.cc:55: Failure
Failed
Bad status: Runtime error (yb/integration-tests/external_mini_cluster.cc:1791): Failed to start masters.: Unable to start Master at index 0: Process exited with rc=256: /code/yugabyte-db/build/release-gcc-dynamic/tests-pgwrapper/../bin/yb-master
Test failure stack trace:
    @     0x7fa7ff680225  MinimalistPrinter::OnTestPartResult()
    @     0x7fa7eee80ca1  testing::internal::TestEventRepeater::OnTestPartResult()
    @     0x7fa7eee9a581  testing::UnitTest::AddTestPartResult()
    @     0x7fa7eee9aa53  testing::internal::AssertHelper::operator=()
    @     0x7fa7ff896ec4  yb::pgwrapper::PgWrapperTestBase::SetUp()
    @     0x7fa7eeea4fe8  testing::internal::HandleExceptionsInMethodIfSupported<>()
    @     0x7fa7eee9bdb6  testing::Test::Run()
    @     0x7fa7eee9bf68  testing::TestInfo::Run()
    @     0x7fa7eee9c045  testing::TestCase::Run()
    @     0x7fa7eee9c2f8  testing::internal::UnitTestImpl::RunAllTests()
    @     0x7fa7eeea5498  testing::internal::HandleExceptionsInMethodIfSupported<>()
    @     0x7fa7eee9c63e  testing::UnitTest::Run()
    @     0x7fa7ff67fccf  main
    @     0x7fa7edde5825  __libc_start_main
    @           0x406749  _start
    @              (nil)  (unknown)

I0808 09:50:40.574800 92121 test_util.cc:93] -----------------------------------------------
I0808 09:50:40.574851 92121 test_util.cc:94] Had fatal failures, leaving test files at /tmp/yb_test.tmp.19923.29004.25197.pid92055/create_initial_sys_catalog_snapshot.CreateInitialSysCatalogSnapshotTest.CreateInitialSysCatalogSnapshot.1596880239639869-92121
[  FAILED  ] CreateInitialSysCatalogSnapshotTest.CreateInitialSysCatalogSnapshot (803 ms)
[----------] 1 test from CreateInitialSysCatalogSnapshotTest (803 ms total)

[----------] Global test environment tear-down
[==========] 1 test from 1 test case ran. (803 ms total)
[  PASSED  ] 0 tests.
[  FAILED  ] 1 test, listed below:
[  FAILED  ] CreateInitialSysCatalogSnapshotTest.CreateInitialSysCatalogSnapshot

 1 FAILED TEST
I0808 09:50:40.596932 92120 run-with-timeout.cc:196] subprocess.Wait finished, waitpid() returned 256
I0808 09:50:40.597160 92120 run-with-timeout.cc:203] Waiting for reaper thread to join
I0808 09:50:40.597257 92122 run-with-timeout.cc:181] Reaper thread: finished=1, subprocess_is_running=0
I0808 09:50:40.597460 92120 run-with-timeout.cc:207] Child process returned exit code 1
I0808 09:50:40.597491 92120 run-with-timeout.cc:222] Returning exit code 1
I0808 09:50:40.597541 92120 run-with-timeout.cc:224] Total time taken: 1.949 sec
(end of standard output)

Standard error from external program {{ '/code/yugabyte-db/build-support/run-test.sh' /code/yugabyte-db/build/release-gcc-dynamic/tests-pgwrapper/create_initial_sys_catalog_snapshot }} running in '/code/yugabyte-db/build/release-gcc-dynamic', saving stdout to {{ /code/yugabyte-db/build/release-gcc-dynamic/create_initial_sys_catalog_snapshot.out }}, stderr to {{ /code/yugabyte-db/build/release-gcc-dynamic/create_initial_sys_catalog_snapshot.err }}:
Details of third-party dependencies:
    YB_THIRDPARTY_DIR: /opt/yb-build/thirdparty/yugabyte-db-thirdparty-v20200707012818-49ca690c1f-centos (from file '/code/yugabyte-db/build/release-gcc-dynamic/thirdparty_path.txt')
    YB_LINUXBREW_DIR: /opt/yb-build/brew/linuxbrew-20181203T161736v9 (from environment)
    YB_THIRDPARTY_URL: https://github.com/yugabyte/yugabyte-db-thirdparty/releases/download/v20200707012818-49ca690c1f/yugabyte-db-thirdparty-v20200707012818-49ca690c1f-centos.tar.gz (from file '/code/yugabyte-db/build/release-gcc-dynamic/thirdparty_url.txt')
    YB_DOWNLOAD_THIRDPARTY: 1
    NO_REBUILD_THIRDPARTY: 1
+ /code/yugabyte-db/build/release-gcc-dynamic/bin/run-with-timeout 1201 /code/yugabyte-db/build/release-gcc-dynamic/tests-pgwrapper/create_initial_sys_catalog_snapshot --initial_sys_catalog_snapshot_dest_path=/code/yugabyte-db/build/release-gcc-dynamic/share/initial_sys_catalog_snapshot --gtest_output=xml:/code/yugabyte-db/build/release-gcc-dynamic/yb-test-logs/tests-pgwrapper__create_initial_sys_catalog_snapshot/CreateInitialSysCatalogSnapshotTest_CreateInitialSysCatalogSnapshot.xml --gtest_filter=CreateInitialSysCatalogSnapshotTest.CreateInitialSysCatalogSnapshot

TEST FAILURE
Test command: /code/yugabyte-db/build/release-gcc-dynamic/tests-pgwrapper/create_initial_sys_catalog_snapshot --initial_sys_catalog_snapshot_dest_path=/code/yugabyte-db/build/release-gcc-dynamic/share/initial_sys_catalog_snapshot --gtest_output=xml:/code/yugabyte-db/build/release-gcc-dynamic/yb-test-logs/tests-pgwrapper__create_initial_sys_catalog_snapshot/CreateInitialSysCatalogSnapshotTest_CreateInitialSysCatalogSnapshot.xml --gtest_filter=CreateInitialSysCatalogSnapshotTest.CreateInitialSysCatalogSnapshot
Test exit status: 1
Log path: /code/yugabyte-db/build/release-gcc-dynamic/yb-test-logs/tests-pgwrapper__create_initial_sys_catalog_snapshot/CreateInitialSysCatalogSnapshotTest_CreateInitialSysCatalogSnapshot.log
[2020-08-08T09:50:40 common-test-env.sh:702 stop_process_tree_supervisor] Omitting process supervisor log, nothing interesting there
[2020-08-08T09:50:40 common-test-env.sh:768 handle_cxx_test_xml_output] Test failed, updating /code/yugabyte-db/build/release-gcc-dynamic/yb-test-logs/tests-pgwrapper__create_initial_sys_catalog_snapshot/CreateInitialSysCatalogSnapshotTest_CreateInitialSysCatalogSnapshot.xml
+ /code/yugabyte-db/build-support/update_test_result_xml.py --result-xml /code/yugabyte-db/build/release-gcc-dynamic/yb-test-logs/tests-pgwrapper__create_initial_sys_catalog_snapshot/CreateInitialSysCatalogSnapshotTest_CreateInitialSysCatalogSnapshot.xml --mark-as-failed true
[postprocess_test_result.py:145] 2020-08-08 09:50:41,665 INFO: Log path: /code/yugabyte-db/build/release-gcc-dynamic/yb-test-logs/tests-pgwrapper__create_initial_sys_catalog_snapshot/CreateInitialSysCatalogSnapshotTest_CreateInitialSysCatalogSnapshot.log
[postprocess_test_result.py:146] 2020-08-08 09:50:41,666 INFO: JUnit XML path: /code/yugabyte-db/build/release-gcc-dynamic/yb-test-logs/tests-pgwrapper__create_initial_sys_catalog_snapshot/CreateInitialSysCatalogSnapshotTest_CreateInitialSysCatalogSnapshot.xml
[postprocess_test_result.py:246] 2020-08-08 09:50:41,690 INFO: Wrote JSON test report file: /code/yugabyte-db/build/release-gcc-dynamic/yb-test-logs/tests-pgwrapper__create_initial_sys_catalog_snapshot/CreateInitialSysCatalogSnapshotTest_CreateInitialSysCatalogSnapshot_test_report.json
(end of standard error)
Traceback (most recent call last):
  File "/code/yugabyte-db/python/yb/gen_initial_sys_catalog_snapshot.py", line 78, in <module>
    main()
  File "/code/yugabyte-db/python/yb/gen_initial_sys_catalog_snapshot.py", line 68, in main
    shell=True
  File "/code/yugabyte-db/build/venv/lib64/python3.6/site-packages/yugabyte_pycommon/external_calls.py", line 339, in run_program
    result.raise_error_if_failed()
  File "/code/yugabyte-db/build/venv/lib64/python3.6/site-packages/yugabyte_pycommon/external_calls.py", line 146, in raise_error_if_failed
    raise ExternalProgramError(self.error_msg, self)
yugabyte_pycommon.external_calls.ExternalProgramError: ("Non-zero exit code 1 from external program {{ '/code/yugabyte-db/build-support/run-test.sh' /code/yugabyte-db/build/release-gcc-dynamic/tests-pgwrapper/create_initial_sys_catalog_snapshot }} running in '/code/yugabyte-db/build/release-gcc-dynamic', saving stdout to {{ /code/yugabyte-db/build/release-gcc-dynamic/create_initial_sys_catalog_snapshot.out }}, stderr to {{ /code/yugabyte-db/build/release-gcc-dynamic/create_initial_sys_catalog_snapshot.err }}.\nStandard output from external program {{ '/code/yugabyte-db/build-support/run-test.sh' /code/yugabyte-db/build/release-gcc-dynamic/tests-pgwrapper/create_initial_sys_catalog_snapshot }} running in '/code/yugabyte-db/build/release-gcc-dynamic', saving stdout to {{ /code/yugabyte-db/build/release-gcc-dynamic/create_initial_sys_catalog_snapshot.out }}, stderr to {{ /code/yugabyte-db/build/release-gcc-dynamic/create_initial_sys_catalog_snapshot.err }}:\nTest is running on host 1364d3aaf253, arguments: /code/yugabyte-db/build/release-gcc-dynamic/tests-pgwrapper/create_initial_sys_catalog_snapshot\nI0808 09:50:38.647032 92120 run-with-timeout.cc:134] Running on host 1364d3aaf253\nI0808 09:50:38.647466 92120 run-with-timeout.cc:141] PATH environment variable: /opt/yb-build/brew/linuxbrew-20181203T161736v9/bin:/code/yugabyte-db/build/venv/bin:/code/yugabyte-db/build/venv/bin:/code/yugabyte-db/build/venv/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin\nI0808 09:50:38.647825 92120 run-with-timeout.cc:149] Starting child process with a timeout of 1201 seconds\nI0808 09:50:38.650043 92120 run-with-timeout.cc:158] Child process pid: 92121\nNote: Google Test filter = CreateInitialSysCatalogSnapshotTest.CreateInitialSysCatalogSnapshot\n[==========] Running 1 test from 1 test case.\n[----------] Global test environment set-up.\n[----------] 1 test from CreateInitialSysCatalogSnapshotTest\n[ RUN      ] CreateInitialSysCatalogSnapshotTest.CreateInitialSysCatalogSnapshot\nI0808 09:50:39.779379 92121 test_util.cc:116] Environment variable ASAN_OPTIONS:   \nI0808 09:50:39.779747 92121 test_util.cc:116] Environment variable LSAN_OPTIONS:   \nI0808 09:50:39.779755 92121 test_util.cc:116] Environment variable UBSAN_OPTIONS:   \nI0808 09:50:39.779759 92121 test_util.cc:116] Environment variable TSAN_OPTIONS:   \nI0808 09:50:39.779816 92121 create_initial_sys_catalog_snapshot.cc:44] Creating initial system catalog snapshot at: /code/yugabyte-db/build/release-gcc-dynamic/share/initial_sys_catalog_snapshot\nI0808 09:50:39.779917 92121 external_mini_cluster.cc:184] Env var YB_EXTRA_MASTER_FLAGS not specified, not setting extra flags from it\nI0808 09:50:39.779945 92121 external_mini_cluster.cc:184] Env var YB_EXTRA_TSERVER_FLAGS not specified, not setting extra flags from it\nI0808 09:50:39.781913 92121 mem_tracker.cc:249] MemTracker: hard memory limit is 8.000000 GB\nI0808 09:50:39.782030 92121 mem_tracker.cc:251] MemTracker: soft memory limit is 6.800000 GB\nI0808 09:50:39.793787 92121 thread_pool.cc:171] Starting thread pool { name: minicluster-messenger queue_limit: 10000 max_workers: 1024 }\nI0808 09:50:39.800942 92121 external_mini_cluster.cc:337] Starting cluster with option bind_to_unique_loopback_addresses=true\nI0808 09:50:39.800971 92121 external_mini_cluster.cc:340] Starting 1 masters\nI0808 09:50:39.802480 92121 net_util.cc:556] Selected random free RPC port 28015\nI0808 09:50:39.802723 92121 external_mini_cluster.cc:957] Using an auto-assigned port 28015 to start an external mini-cluster master\nI0808 09:50:39.806772 92121 net_util.cc:556] Selected random free RPC port 29238\nI0808 09:50:39.807071 92121 external_mini_cluster.cc:184] Env var YB_EXTRA_DAEMON_FLAGS not specified, not setting extra flags from it\nI0808 09:50:39.807114 92121 external_mini_cluster.cc:1753] Running [m-1]: /code/yugabyte-db/build/release-gcc-dynamic/tests-pgwrapper/../bin/yb-master\nyb-master\n--fs_data_dirs=/tmp/yb_test.tmp.19923.29004.25197.pid92055/create_initial_sys_catalog_snapshot.CreateInitialSysCatalogSnapshotTest.CreateInitialSysCatalogSnapshot.1596880239639869-92121/minicluster-data/master-0\n--rpc_bind_addresses=127.0.0.1:28015\n--webserver_interface=localhost\n--webserver_port=29238\n--load_balancer_initial_delay_secs=0\n--master_addresses=127.0.0.1:28015\n--callhome_enabled=false\n--metrics_log_interval_ms=0\n--log_dir=\n--enable_tracing\n--memory_limit_hard_bytes=1073741824\n--logtostderr\n--hide_pg_catalog_table_creation_logs\n--client_read_write_timeout_ms=120000\n--memory_limit_hard_bytes=2147483648\n--create_initial_sys_catalog_snapshot\n--initial_sys_catalog_snapshot_path=/code/yugabyte-db/build/release-gcc-dynamic/share/initial_sys_catalog_snapshot\n--master_auto_run_initdb\n--enable_leader_failure_detection=true\n--enable_ysql=true\n--master_auto_run_initdb\n--server_dump_info_path=/tmp/yb_test.tmp.19923.29004.25197.pid92055/create_initial_sys_catalog_snapshot.CreateInitialSysCatalogSnapshotTest.CreateInitialSysCatalogSnapshot.1596880239639869-92121/minicluster-data/master-0/yb-data/master/info.pb\n--server_dump_info_format=pb\n--rpc_server_allow_ephemeral_ports\n--logbuflevel=-1\n--metric_node_name=test_invocation_2020-08-08T09_50_36_31890_28416_92055\n--fatal_details_path_prefix=/code/yugabyte-db/build/release-gcc-dynamic/yb-test-logs/tests-pgwrapper__create_initial_sys_catalog_snapshot/CreateInitialSysCatalogSnapshotTest_CreateInitialSysCatalogSnapshot.fatal_failure_details.m-1\n--minicluster_daemon_id=m-1\n[m-1] Illegal state (yb/util/version_info.cc:165): Key build_username is of invalid type 0\n/code/yugabyte-db/src/yb/yql/pgwrapper/pg_wrapper_test_base.cc:55: Failure\nFailed\nBad status: Runtime error (yb/integration-tests/external_mini_cluster.cc:1791): Failed to start masters.: Unable to start Master at index 0: Process exited with rc=256: /code/yugabyte-db/build/release-gcc-dynamic/tests-pgwrapper/../bin/yb-master\nTest failure stack trace:\n    @     0x7fa7ff680225  MinimalistPrinter::OnTestPartResult()\n    @     0x7fa7eee80ca1  testing::internal::TestEventRepeater::OnTestPartResult()\n    @     0x7fa7eee9a581  testing::UnitTest::AddTestPartResult()\n    @     0x7fa7eee9aa53  testing::internal::AssertHelper::operator=()\n    @     0x7fa7ff896ec4  yb::pgwrapper::PgWrapperTestBase::SetUp()\n    @     0x7fa7eeea4fe8  testing::internal::HandleExceptionsInMethodIfSupported<>()\n    @     0x7fa7eee9bdb6  testing::Test::Run()\n    @     0x7fa7eee9bf68  testing::TestInfo::Run()\n    @     0x7fa7eee9c045  testing::TestCase::Run()\n    @     0x7fa7eee9c2f8  testing::internal::UnitTestImpl::RunAllTests()\n    @     0x7fa7eeea5498  testing::internal::HandleExceptionsInMethodIfSupported<>()\n    @     0x7fa7eee9c63e  testing::UnitTest::Run()\n    @     0x7fa7ff67fccf  main\n    @     0x7fa7edde5825  __libc_start_main\n    @           0x406749  _start\n    @              (nil)  (unknown)\n\nI0808 09:50:40.574800 92121 test_util.cc:93] -----------------------------------------------\nI0808 09:50:40.574851 92121 test_util.cc:94] Had fatal failures, leaving test files at /tmp/yb_test.tmp.19923.29004.25197.pid92055/create_initial_sys_catalog_snapshot.CreateInitialSysCatalogSnapshotTest.CreateInitialSysCatalogSnapshot.1596880239639869-92121\n[  FAILED  ] CreateInitialSysCatalogSnapshotTest.CreateInitialSysCatalogSnapshot (803 ms)\n[----------] 1 test from CreateInitialSysCatalogSnapshotTest (803 ms total)\n\n[----------] Global test environment tear-down\n[==========] 1 test from 1 test case ran. (803 ms total)\n[  PASSED  ] 0 tests.\n[  FAILED  ] 1 test, listed below:\n[  FAILED  ] CreateInitialSysCatalogSnapshotTest.CreateInitialSysCatalogSnapshot\n\n 1 FAILED TEST\nI0808 09:50:40.596932 92120 run-with-timeout.cc:196] subprocess.Wait finished, waitpid() returned 256\nI0808 09:50:40.597160 92120 run-with-timeout.cc:203] Waiting for reaper thread to join\nI0808 09:50:40.597257 92122 run-with-timeout.cc:181] Reaper thread: finished=1, subprocess_is_running=0\nI0808 09:50:40.597460 92120 run-with-timeout.cc:207] Child process returned exit code 1\nI0808 09:50:40.597491 92120 run-with-timeout.cc:222] Returning exit code 1\nI0808 09:50:40.597541 92120 run-with-timeout.cc:224] Total time taken: 1.949 sec\n(end of standard output)\n\nStandard error from external program {{ '/code/yugabyte-db/build-support/run-test.sh' /code/yugabyte-db/build/release-gcc-dynamic/tests-pgwrapper/create_initial_sys_catalog_snapshot }} running in '/code/yugabyte-db/build/release-gcc-dynamic', saving stdout to {{ /code/yugabyte-db/build/release-gcc-dynamic/create_initial_sys_catalog_snapshot.out }}, stderr to {{ /code/yugabyte-db/build/release-gcc-dynamic/create_initial_sys_catalog_snapshot.err }}:\nDetails of third-party dependencies:\n    YB_THIRDPARTY_DIR: /opt/yb-build/thirdparty/yugabyte-db-thirdparty-v20200707012818-49ca690c1f-centos (from file '/code/yugabyte-db/build/release-gcc-dynamic/thirdparty_path.txt')\n    YB_LINUXBREW_DIR: /opt/yb-build/brew/linuxbrew-20181203T161736v9 (from environment)\n    YB_THIRDPARTY_URL: https://github.com/yugabyte/yugabyte-db-thirdparty/releases/download/v20200707012818-49ca690c1f/yugabyte-db-thirdparty-v20200707012818-49ca690c1f-centos.tar.gz (from file '/code/yugabyte-db/build/release-gcc-dynamic/thirdparty_url.txt')\n    YB_DOWNLOAD_THIRDPARTY: 1\n    NO_REBUILD_THIRDPARTY: 1\n+ /code/yugabyte-db/build/release-gcc-dynamic/bin/run-with-timeout 1201 /code/yugabyte-db/build/release-gcc-dynamic/tests-pgwrapper/create_initial_sys_catalog_snapshot --initial_sys_catalog_snapshot_dest_path=/code/yugabyte-db/build/release-gcc-dynamic/share/initial_sys_catalog_snapshot --gtest_output=xml:/code/yugabyte-db/build/release-gcc-dynamic/yb-test-logs/tests-pgwrapper__create_initial_sys_catalog_snapshot/CreateInitialSysCatalogSnapshotTest_CreateInitialSysCatalogSnapshot.xml --gtest_filter=CreateInitialSysCatalogSnapshotTest.CreateInitialSysCatalogSnapshot\n\nTEST FAILURE\nTest command: /code/yugabyte-db/build/release-gcc-dynamic/tests-pgwrapper/create_initial_sys_catalog_snapshot --initial_sys_catalog_snapshot_dest_path=/code/yugabyte-db/build/release-gcc-dynamic/share/initial_sys_catalog_snapshot --gtest_output=xml:/code/yugabyte-db/build/release-gcc-dynamic/yb-test-logs/tests-pgwrapper__create_initial_sys_catalog_snapshot/CreateInitialSysCatalogSnapshotTest_CreateInitialSysCatalogSnapshot.xml --gtest_filter=CreateInitialSysCatalogSnapshotTest.CreateInitialSysCatalogSnapshot\nTest exit status: 1\nLog path: /code/yugabyte-db/build/release-gcc-dynamic/yb-test-logs/tests-pgwrapper__create_initial_sys_catalog_snapshot/CreateInitialSysCatalogSnapshotTest_CreateInitialSysCatalogSnapshot.log\n[2020-08-08T09:50:40 common-test-env.sh:702 stop_process_tree_supervisor] Omitting process supervisor log, nothing interesting there\n[2020-08-08T09:50:40 common-test-env.sh:768 handle_cxx_test_xml_output] Test failed, updating /code/yugabyte-db/build/release-gcc-dynamic/yb-test-logs/tests-pgwrapper__create_initial_sys_catalog_snapshot/CreateInitialSysCatalogSnapshotTest_CreateInitialSysCatalogSnapshot.xml\n+ /code/yugabyte-db/build-support/update_test_result_xml.py --result-xml /code/yugabyte-db/build/release-gcc-dynamic/yb-test-logs/tests-pgwrapper__create_initial_sys_catalog_snapshot/CreateInitialSysCatalogSnapshotTest_CreateInitialSysCatalogSnapshot.xml --mark-as-failed true\n[postprocess_test_result.py:145] 2020-08-08 09:50:41,665 INFO: Log path: /code/yugabyte-db/build/release-gcc-dynamic/yb-test-logs/tests-pgwrapper__create_initial_sys_catalog_snapshot/CreateInitialSysCatalogSnapshotTest_CreateInitialSysCatalogSnapshot.log\n[postprocess_test_result.py:146] 2020-08-08 09:50:41,666 INFO: JUnit XML path: /code/yugabyte-db/build/release-gcc-dynamic/yb-test-logs/tests-pgwrapper__create_initial_sys_catalog_snapshot/CreateInitialSysCatalogSnapshotTest_CreateInitialSysCatalogSnapshot.xml\n[postprocess_test_result.py:246] 2020-08-08 09:50:41,690 INFO: Wrote JSON test report file: /code/yugabyte-db/build/release-gcc-dynamic/yb-test-logs/tests-pgwrapper__create_initial_sys_catalog_snapshot/CreateInitialSysCatalogSnapshotTest_CreateInitialSysCatalogSnapshot_test_report.json\n(end of standard error)", <yugabyte_pycommon.external_calls.ProgramResult object at 0x7feccc923358>)
make[2]: *** [src/yb/yql/pgwrapper/CMakeFiles/initial_sys_catalog_snapshot] Error 1
make[1]: *** [src/yb/yql/pgwrapper/CMakeFiles/initial_sys_catalog_snapshot.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
[ 99%] Building CXX object src/yb/tools/CMakeFiles/yb-bulk_load-test.dir/yb-bulk_load-test.cc.o
[ 99%] Linking CXX executable ../../../tests-tools/ysck-test
[ 99%] Built target ysck-test
[100%] Linking CXX executable ../../../tests-tools/ysck_remote-test
[100%] Linking CXX executable ../../../tests-tools/yb-admin-test
[100%] Built target ysck_remote-test
[100%] Built target yb-admin-test
[100%] Linking CXX executable ../../../tests-tools/yb-bulk_load-test
[100%] Built target yb-bulk_load-test
make: *** [all] Error 2

real    93m40.810s
user    250m31.820s
sys 56m31.841s
[2020-08-08T09:51:20 yb_build.sh:416 run_cxx_build] C++ build finished with exit code 2 (build type: release, compiler: gcc). Timing information is available above.

==========================================================================================
YUGABYTE BUILD SUMMARY
==========================================================================================
Build type                       : release
C/C++ compiler                   : gcc
Build directory                  : /code/yugabyte-db/build/release-gcc-dynamic
Third-party dir                  : /opt/yb-build/thirdparty/yugabyte-db-thirdparty-v20200707012818-49ca690c1f-centos
Linuxbrew dir                    : /opt/yb-build/brew/linuxbrew-20181203T161736v9
CMake time                       : 59 seconds
C++ compilation time             : 5620 seconds
Exit code                        : 2
------------------------------------------------------------------------------------------

Thank you.

kmuthukk commented 4 years ago

Looks like we are hitting this check failure:

Key build_username is of invalid type 0

here: https://github.com/yugabyte/yugabyte-db/blob/9fa72cd3048bacf35bc8aab7191a702ed0a6271b/src/yb/util/version_info.cc#L164

That in turn seems to be prepared here: https://github.com/yugabyte/yugabyte-db/blob/9fa72cd3048bacf35bc8aab7191a702ed0a6271b/build-support/gen_version_info.py#L138

from: https://github.com/yugabyte/yugabyte-db/blob/9fa72cd3048bacf35bc8aab7191a702ed0a6271b/build-support/gen_version_info.py#L95

Is it possible that in the docker on Mac environment, the os.getenv("USER") doesn't return a user name string?

danwt commented 4 years ago

That's indeed the case

[root@1364d3aaf253 yugabyte-db]# python
Python 2.7.5 (default, Apr  2 2020, 13:16:51)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-39)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> os.getenv("USER")==None
True
>>>
kmuthukk commented 4 years ago

@danwt --

Possible workarounds in the interim:

1) Perhaps before running the compilation command do a:

export USER=danwt

2) In your local copy of

https://github.com/yugabyte/yugabyte-db/blob/9fa72cd3048bacf35bc8aab7191a702ed0a6271b/build-support/gen_version_info.py#L95

You can add something like this after the above lines:

  if username==None:
     username="none"

@mbautin -- if (2) seems reasonable, we could probably even request @danwt to submit a PR for the same that we can then merge?

mbautin commented 4 years ago

Hi @danwt, thanks a lot for reporting this issue! Perhaps we could use getpass.getuser() if the USER environment variable is not set:

if username is None:
  username = getpass.getuser()

And add import getpass at the top of the script. Please feel free to submit a pull request with this fix.

(Relevant link: https://stackoverflow.com/questions/842059/is-there-a-portable-way-to-get-the-current-username-in-python)

danwt commented 4 years ago

Thanks for your help!