lbakerchef commented 2 years ago

Describe the problem

I just ran across two different knife ec backup failures.

One failure was while running a local terraform external postgresql scenario, after I logged into the chef-server box and tried knife ec backup.

The other issue was with the nightly umbrella test, fresh-install scenario. Although the scenario was green, knife ec backup had failed there also. Which brings up a third issue - we did not know we had a failure because the test was green.

Acceptance criteria:

[x] knife ec backup works, standalone-fresh-install scenario.
[x] The standalone fresh install umbrella scenario fails if knife ec backup fails. https://github.com/chef/umbrella/pull/200
[ ] knife ec backup works, external-pg scenario (when ran locally via terraform, after logging into the box). We might consider adding knife ec backup to this and other scenarios (currently it's only standalone-fresh-install).

Software Version

PLATFORM=ubuntu-18.04 INSTALL_VERSION=14.9.23 UPGRADE_VERSION=14.10.23 SCENARIO=external-postgresql ENABLE_ADDON_PUSH_JOBS=false ENABLE_GATHER_LOGS_TEST=false ENABLE_PEDANT_TEST=false ENABLE_PSQL_TEST=true ENABLE_SMOKE_TEST=false ENABLE_IPV6=false ENABLE_ADDON_CHEF_MANAGE=false make apply
See https://buildkite.com/chef/chef-umbrella-main-chef-server/builds/846#9129b904-f2ea-4507-a6a5-6fd5a67bec28

Replication Case

sudo /opt/opscode/bin/knife ec backup /var/opt/knife-ec-backup/ --with-user-sql --with-key-sql -VVV -c /etc/opscode/pivotal.rb
See https://buildkite.com/chef/chef-umbrella-main-chef-server/builds/846#9129b904-f2ea-4507-a6a5-6fd5a67bec28.

Stacktrace

external postgresql terraform scenario, login to box


root@ip-10-0-4-232:/home/ubuntu# /opt/opscode/bin/knife ec backup /var/opt/knife-ec-backup/ --with-user-sql --with-key-sql -VVV -c /etc/opscode/pivotal.rb

Traceback (most recent call last): 25: from /opt/opscode/bin/knife:146:in `

' 24: from /opt/opscode/bin/knife:146:in `load' 23: from /opt/opscode/embedded/lib/ruby/gems/2.7.0/gems/chef-16.13.16/bin/knife:24:in `' 22: from /opt/opscode/embedded/lib/ruby/gems/2.7.0/gems/chef-16.13.16/lib/chef/application/knife.rb:165:in `run' 21: from /opt/opscode/embedded/lib/ruby/gems/2.7.0/gems/chef-16.13.16/lib/chef/knife.rb:229:in `run' 20: from /opt/opscode/embedded/lib/ruby/gems/2.7.0/gems/chef-16.13.16/lib/chef/knife.rb:489:in `run_with_pretty_exceptions' 19: from /opt/opscode/embedded/lib/ruby/gems/2.7.0/gems/chef-16.13.16/lib/chef/local_mode.rb:42:in `with_server_connectivity' 18: from /opt/opscode/embedded/lib/ruby/gems/2.7.0/gems/chef-16.13.16/lib/chef/knife.rb:490:in `block in run_with_pretty_exceptions' 17: from /opt/opscode/embedded/lib/ruby/gems/2.7.0/gems/knife-ec-backup-2.5.3/lib/chef/knife/ec_backup.rb:45:in `run' 16: from /opt/opscode/embedded/lib/ruby/gems/2.7.0/gems/knife-ec-backup-2.5.3/lib/chef/knife/ec_backup.rb:145:in `export_from_sql' 15: from /opt/opscode/embedded/lib/ruby/gems/2.7.0/gems/knife-ec-backup-2.5.3/lib/chef/knife/ec_key_export.rb:40:in `run' 14: from /opt/opscode/embedded/lib/ruby/gems/2.7.0/gems/knife-ec-backup-2.5.3/lib/chef/knife/ec_key_export.rb:61:in `export' 13: from /opt/opscode/embedded/lib/ruby/gems/2.7.0/gems/knife-ec-backup-2.5.3/lib/chef/knife/ec_key_base.rb:101:in `db' 12: from /opt/opscode/embedded/lib/ruby/gems/2.7.0/gems/sequel-5.49.0/lib/sequel/core.rb:124:in `connect' 11: from /opt/opscode/embedded/lib/ruby/gems/2.7.0/gems/sequel-5.49.0/lib/sequel/database/connecting.rb:57:in `connect' 10: from /opt/opscode/embedded/lib/ruby/gems/2.7.0/gems/sequel-5.49.0/lib/sequel/database/connecting.rb:57:in `new' 9: from /opt/opscode/embedded/lib/ruby/gems/2.7.0/gems/sequel-5.49.0/lib/sequel/database/misc.rb:169:in `initialize' 8: from /opt/opscode/embedded/lib/ruby/gems/2.7.0/gems/sequel-5.49.0/lib/sequel/database/connecting.rb:278:in `test_connection' 7: from /opt/opscode/embedded/lib/ruby/gems/2.7.0/gems/sequel-5.49.0/lib/sequel/database/connecting.rb:269:in `synchronize' 6: from /opt/opscode/embedded/lib/ruby/gems/2.7.0/gems/sequel-5.49.0/lib/sequel/connection_pool/threaded.rb:91:in `hold' 5: from /opt/opscode/embedded/lib/ruby/gems/2.7.0/gems/sequel-5.49.0/lib/sequel/connection_pool/threaded.rb:139:in `acquire' 4: from /opt/opscode/embedded/lib/ruby/gems/2.7.0/gems/sequel-5.49.0/lib/sequel/connection_pool/threaded.rb:209:in `assign_connection' 3: from /opt/opscode/embedded/lib/ruby/gems/2.7.0/gems/sequel-5.49.0/lib/sequel/connection_pool.rb:122:in `make_new' 2: from /opt/opscode/embedded/lib/ruby/gems/2.7.0/gems/sequel-5.49.0/lib/sequel/adapters/postgres.rb:210:in `connect' 1: from /opt/opscode/embedded/lib/ruby/gems/2.7.0/gems/sequel-5.49.0/lib/sequel/adapters/postgres.rb:210:in `new' /opt/opscode/embedded/lib/ruby/gems/2.7.0/gems/sequel-5.49.0/lib/sequel/adapters/postgres.rb:210:in `initialize': could not connect to server: Connection refused (PG::ConnectionBad) Is the server running on host "localhost" (127.0.0.1) and accepting TCP/IP connections on port 5432? 25: from /opt/opscode/bin/knife:146:in `

' 24: from /opt/opscode/bin/knife:146:in `load' 23: from /opt/opscode/embedded/lib/ruby/gems/2.7.0/gems/chef-16.13.16/bin/knife:24:in `' 22: from /opt/opscode/embedded/lib/ruby/gems/2.7.0/gems/chef-16.13.16/lib/chef/application/knife.rb:165:in `run' 21: from /opt/opscode/embedded/lib/ruby/gems/2.7.0/gems/chef-16.13.16/lib/chef/knife.rb:229:in `run' 20: from /opt/opscode/embedded/lib/ruby/gems/2.7.0/gems/chef-16.13.16/lib/chef/knife.rb:489:in `run_with_pretty_exceptions' 19: from /opt/opscode/embedded/lib/ruby/gems/2.7.0/gems/chef-16.13.16/lib/chef/local_mode.rb:42:in `with_server_connectivity' 18: from /opt/opscode/embedded/lib/ruby/gems/2.7.0/gems/chef-16.13.16/lib/chef/knife.rb:490:in `block in run_with_pretty_exceptions' 17: from /opt/opscode/embedded/lib/ruby/gems/2.7.0/gems/knife-ec-backup-2.5.3/lib/chef/knife/ec_backup.rb:45:in `run' 16: from /opt/opscode/embedded/lib/ruby/gems/2.7.0/gems/knife-ec-backup-2.5.3/lib/chef/knife/ec_backup.rb:145:in `export_from_sql' 15: from /opt/opscode/embedded/lib/ruby/gems/2.7.0/gems/knife-ec-backup-2.5.3/lib/chef/knife/ec_key_export.rb:40:in `run' 14: from /opt/opscode/embedded/lib/ruby/gems/2.7.0/gems/knife-ec-backup-2.5.3/lib/chef/knife/ec_key_export.rb:61:in `export' 13: from /opt/opscode/embedded/lib/ruby/gems/2.7.0/gems/knife-ec-backup-2.5.3/lib/chef/knife/ec_key_base.rb:101:in `db' 12: from /opt/opscode/embedded/lib/ruby/gems/2.7.0/gems/sequel-5.49.0/lib/sequel/core.rb:124:in `connect' 11: from /opt/opscode/embedded/lib/ruby/gems/2.7.0/gems/sequel-5.49.0/lib/sequel/database/connecting.rb:57:in `connect' 10: from /opt/opscode/embedded/lib/ruby/gems/2.7.0/gems/sequel-5.49.0/lib/sequel/database/connecting.rb:57:in `new' 9: from /opt/opscode/embedded/lib/ruby/gems/2.7.0/gems/sequel-5.49.0/lib/sequel/database/misc.rb:169:in `initialize' 8: from /opt/opscode/embedded/lib/ruby/gems/2.7.0/gems/sequel-5.49.0/lib/sequel/database/connecting.rb:278:in `test_connection' 7: from /opt/opscode/embedded/lib/ruby/gems/2.7.0/gems/sequel-5.49.0/lib/sequel/database/connecting.rb:269:in `synchronize' 6: from /opt/opscode/embedded/lib/ruby/gems/2.7.0/gems/sequel-5.49.0/lib/sequel/connection_pool/threaded.rb:91:in `hold' 5: from /opt/opscode/embedded/lib/ruby/gems/2.7.0/gems/sequel-5.49.0/lib/sequel/connection_pool/threaded.rb:139:in `acquire' 4: from /opt/opscode/embedded/lib/ruby/gems/2.7.0/gems/sequel-5.49.0/lib/sequel/connection_pool/threaded.rb:209:in `assign_connection' 3: from /opt/opscode/embedded/lib/ruby/gems/2.7.0/gems/sequel-5.49.0/lib/sequel/connection_pool.rb:122:in `make_new' 2: from /opt/opscode/embedded/lib/ruby/gems/2.7.0/gems/sequel-5.49.0/lib/sequel/adapters/postgres.rb:210:in `connect' 1: from /opt/opscode/embedded/lib/ruby/gems/2.7.0/gems/sequel-5.49.0/lib/sequel/adapters/postgres.rb:210:in `new' /opt/opscode/embedded/lib/ruby/gems/2.7.0/gems/sequel-5.49.0/lib/sequel/adapters/postgres.rb:210:in `initialize': PG::ConnectionBad: could not connect to server: Connection refused (Sequel::DatabaseConnectionError) Is the server running on host "localhost" (127.0.0.1) and accepting TCP/IP connections on port 5432? ``` 2. https://buildkite.com/chef/chef-umbrella-main-chef-server/builds/846#9129b904-f2ea-4507-a6a5-6fd5a67bec28 ``` [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mBEGIN KNIFE EC BACKUP [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0m/opt/opscode/embedded/lib/ruby/gems/2.7.0/gems/chef-vault-4.1.4/lib/chef-vault/version.rb:18: warning: already initialized constant ChefVault::VERSION [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0m/opt/opscode/embedded/lib/ruby/gems/2.7.0/gems/chef-vault-4.1.0/lib/chef-vault/version.rb:18: warning: previous definition of VERSION was here [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0m/opt/opscode/embedded/lib/ruby/gems/2.7.0/gems/chef-vault-4.1.4/lib/chef-vault/version.rb:19: warning: already initialized constant ChefVault::MAJOR [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0m/opt/opscode/embedded/lib/ruby/gems/2.7.0/gems/chef-vault-4.1.0/lib/chef-vault/version.rb:19: warning: previous definition of MAJOR was here [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0m/opt/opscode/embedded/lib/ruby/gems/2.7.0/gems/chef-vault-4.1.4/lib/chef-vault/version.rb:19: warning: already initialized constant ChefVault::MINOR [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0m/opt/opscode/embedded/lib/ruby/gems/2.7.0/gems/chef-vault-4.1.0/lib/chef-vault/version.rb:19: warning: previous definition of MINOR was here [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0m/opt/opscode/embedded/lib/ruby/gems/2.7.0/gems/chef-vault-4.1.4/lib/chef-vault/version.rb:19: warning: already initialized constant ChefVault::TINY [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0m/opt/opscode/embedded/lib/ruby/gems/2.7.0/gems/chef-vault-4.1.0/lib/chef-vault/version.rb:19: warning: previous definition of TINY was here [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mINFO: Using configuration from /etc/opscode/pivotal.rb [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: Chef::HTTP calling Chef::HTTP::JSONInput#handle_request [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: Chef::HTTP calling Chef::HTTP::JSONOutput#handle_request [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: Chef::HTTP calling Chef::HTTP::CookieManager#handle_request [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: Chef::HTTP calling Chef::HTTP::Decompressor#handle_request [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: Chef::HTTP calling Chef::HTTP::Authenticator#handle_request [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: Signing the request as pivotal [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: Chef::HTTP calling Chef::HTTP::RemoteRequestID#handle_request [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: Chef::HTTP calling Chef::HTTP::APIVersions#handle_request [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: Chef::HTTP calling Chef::HTTP::ValidateContentLength#handle_request [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: Initiating GET to https://127.0.0.1/version [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: ---- HTTP Request Header Data: ---- [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: Accept: application/json [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: Accept-Encoding: gzip;q=1.0,deflate;q=0.6,identity;q=0.3 [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: X-Ops-Server-API-Version: 2 [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: X-OPS-SIGN: algorithm=sha1;version=1.1; [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: X-OPS-USERID: pivotal [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: X-OPS-TIMESTAMP: 2021-10-26T09:09:54Z [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: X-OPS-CONTENT-HASH: 2jmj7l5rSw0yVb/vlWAYkK/YBwk= [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: X-OPS-AUTHORIZATION-1: H1wondKyabA1askQyGDKq2qBlHZr1/h7TCwuZikSlQmtDnozVUouJJTgysGc [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: X-OPS-AUTHORIZATION-2: BMmslpiugNrKNTZyXVdX7uW55RIP8d66Jle9EuNQd4vXDq/1wLqiC/Ac7pEW [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: X-OPS-AUTHORIZATION-3: Ohc9ACrnJiKRMabbqiXbPYrfb7lQE6RNmRzBa5fAwLePf0Y2fyGiwXlEFV2X [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: X-OPS-AUTHORIZATION-4: pXQB56r00wuzKBPnGt5qkvTPOcuDrv7UNA2gkrE8a7Q/1wew4eQ6UK2dwjpr [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: X-OPS-AUTHORIZATION-5: dpor+QXVsjpN4vTi5eelpO4Ovz01bKIF333dORYiLuyKkgOdz5xIPpnD1vCa [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: X-OPS-AUTHORIZATION-6: I1VwcFt2bEw1JXxQJ4zekBausQRv+hQwbybyVQ6pYQ== [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: HOST: 127.0.0.1:443 [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: X-REMOTE-REQUEST-ID: 80364e30-fd88-4d9a-ab74-88bc4f588fa7 [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: x-ops-request-source: web [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: ---- End HTTP Request Header Data ---- [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: ---- HTTP Status and Header Data: ---- [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: HTTP 1.1 200 OK [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: date: Tue, 26 Oct 2021 09:09:54 GMT [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: content-type: text/plain [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: last-modified: Fri, 22 Oct 2021 06:01:23 GMT [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: transfer-encoding: chunked [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: connection: close [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: etag: W/"617253b3-2374" [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: content-security-policy: default-src 'self'; [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: x-frame-options: DENY [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: content-encoding: gzip [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: ---- End HTTP Status/Header Data ---- [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: Chef::HTTP calling Chef::HTTP::ValidateContentLength#handle_response [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: HTTP server did not include a Content-Length header in response, cannot identify truncated downloads. [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: Chef::HTTP calling Chef::HTTP::APIVersions#handle_response [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: Chef::HTTP calling Chef::HTTP::RemoteRequestID#handle_response [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: Chef::HTTP calling Chef::HTTP::Authenticator#handle_response [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: Chef::HTTP calling Chef::HTTP::Decompressor#handle_response [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: Decompressing gzip response [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: Chef::HTTP calling Chef::HTTP::CookieManager#handle_response [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: Chef::HTTP calling Chef::HTTP::JSONOutput#handle_response [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: Expected JSON response, but got content-type 'text/plain' [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: Response body contains: [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mchef-server 14.10.25 [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mComponent Installed Version Version GUID Overridden From [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0m------------------------------- [...truncated...] [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: Chef::HTTP calling Chef::HTTP::JSONInput#handle_response [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mDownloading Users [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: Chef::HTTP calling Chef::HTTP::JSONInput#handle_request [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: Chef::HTTP calling Chef::HTTP::JSONOutput#handle_request [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: Chef::HTTP calling Chef::HTTP::CookieManager#handle_request [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: Chef::HTTP calling Chef::HTTP::Decompressor#handle_request [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: Chef::HTTP calling Chef::HTTP::Authenticator#handle_request [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: Signing the request as pivotal [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: Chef::HTTP calling Chef::HTTP::RemoteRequestID#handle_request [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: Chef::HTTP calling Chef::HTTP::APIVersions#handle_request [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: Chef::HTTP calling Chef::HTTP::ValidateContentLength#handle_request [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: Initiating GET to https://127.0.0.1/users [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: ---- HTTP Request Header Data: ---- [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: Accept: application/json [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: Accept-Encoding: gzip;q=1.0,deflate;q=0.6,identity;q=0.3 [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: X-Ops-Server-API-Version: 0 [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: X-OPS-SIGN: algorithm=sha1;version=1.1; [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: X-OPS-USERID: pivotal [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: X-OPS-TIMESTAMP: 2021-10-26T09:09:54Z [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: X-OPS-CONTENT-HASH: 2jmj7l5rSw0yVb/vlWAYkK/YBwk= [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: X-OPS-AUTHORIZATION-1: o5HOB2cMiFtkisBd6jUP4TqK1Xd8zzMM/uNQU6FMqOVROC5h8cmD+g4Q6ZF2 [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: X-OPS-AUTHORIZATION-2: AeB7+gBO9uUmqTzKyVu0JuPV1PS2E4qfztzKWNUm1RHEyBh4zkYa1cgy2YuR [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: X-OPS-AUTHORIZATION-3: Jqx1ufE9hUKcgN5CIVMDI5UcFgV0SNN/lhheewQJ+eq+doH8ruKKLrtTrVmL [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: X-OPS-AUTHORIZATION-4: g6PuLdZpnXL9Z4kLTARuh/QdtowyoM72QwyDSMJxz/wxMpaCv3FztPFYB3Zr [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: X-OPS-AUTHORIZATION-5: oqjVVg49FTWJlbhTpD6E1IAJiBIb0jMGOIMBNsZ+cNlbT0yam7h63Ch78M3V [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: X-OPS-AUTHORIZATION-6: QTo1MF4hsxNRktNCvJFZ8Bu29v6/hHIgUAtIVUVJ7Q== [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: HOST: 127.0.0.1:443 [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: X-REMOTE-REQUEST-ID: 80364e30-fd88-4d9a-ab74-88bc4f588fa7 [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: x-ops-request-source: web [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: ---- End HTTP Request Header Data ---- [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: ---- HTTP Status and Header Data: ---- [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: HTTP 1.1 404 Not Found [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: date: Tue, 26 Oct 2021 09:09:54 GMT [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: content-type: text/html [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: transfer-encoding: chunked [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: connection: close [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: content-encoding: gzip [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: ---- End HTTP Status/Header Data ---- [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: ---- HTTP Response Body ---- [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: ³ɨɍ±㥲ɈML±³)ɬɉµ310Qퟮ�p˯͋±чڨ &委褔¼Ԣ;Ct@}¨4Ȭ "b$a}AúPG þN [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: ---- End HTTP Response Body ----- [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: Chef::HTTP calling Chef::HTTP::ValidateContentLength#handle_response [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: HTTP server did not include a Content-Length header in response, cannot identify truncated downloads. [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: Chef::HTTP calling Chef::HTTP::APIVersions#handle_response [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: Chef::HTTP calling Chef::HTTP::RemoteRequestID#handle_response [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: Chef::HTTP calling Chef::HTTP::Authenticator#handle_response [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: Chef::HTTP calling Chef::HTTP::Decompressor#handle_response [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: Decompressing gzip response [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: Chef::HTTP calling Chef::HTTP::CookieManager#handle_response [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: Chef::HTTP calling Chef::HTTP::JSONOutput#handle_response [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: Expected JSON response, but got content-type 'text/html' [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: Response body contains: [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0m [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0m404 Not Found [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0m [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0m

404 Not Found

[0m[1mnull_resource.chef_server_test (remote-exec):[0m [0m

[0m[1mnull_resource.chef_server_test (remote-exec):[0m [0m [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0m [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: Chef::HTTP calling Chef::HTTP::JSONInput#handle_response [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: Chef::HTTP calling Chef::HTTP::JSONInput#handle_request [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: Chef::HTTP calling Chef::HTTP::JSONOutput#handle_request [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: Chef::HTTP calling Chef::HTTP::CookieManager#handle_request [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: Chef::HTTP calling Chef::HTTP::Decompressor#handle_request [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: Chef::HTTP calling Chef::HTTP::Authenticator#handle_request [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: Signing the request as pivotal [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: Chef::HTTP calling Chef::HTTP::RemoteRequestID#handle_request [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: Chef::HTTP calling Chef::HTTP::APIVersions#handle_request [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: Chef::HTTP calling Chef::HTTP::ValidateContentLength#handle_request [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: Initiating GET to https://127.0.0.1/organizations [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: ---- HTTP Request Header Data: ---- [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: Accept: application/json [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: Accept-Encoding: gzip;q=1.0,deflate;q=0.6,identity;q=0.3 [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: X-Ops-Server-API-Version: 0 [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: X-OPS-SIGN: algorithm=sha1;version=1.1; [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: X-OPS-USERID: pivotal [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: X-OPS-TIMESTAMP: 2021-10-26T09:09:54Z [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: X-OPS-CONTENT-HASH: 2jmj7l5rSw0yVb/vlWAYkK/YBwk= [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: X-OPS-AUTHORIZATION-1: qKPNagMaVFd5376d5FpqJRdeCwcCPJDBDZMJJArcG2ZoSLMwj8huTWwmBVDn [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: X-OPS-AUTHORIZATION-2: S5/hgyGZkcAYnT9YYSKluEDZKmTHVN8W6szGYSYjLOyBtukNpTw9XT1Yj6OA [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: X-OPS-AUTHORIZATION-3: 4C+SaeT60l/tRgO3Q138CDC4kwYN7EqRNCKdBoIhns4BWUtxi+MSb3ocY6jJ [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: X-OPS-AUTHORIZATION-4: 2kUlOLaRPluhfhoHXOK98qxQC/f2P4JZt9dVs2jlOzJvzwDhNaa2/YRXfPgn [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: X-OPS-AUTHORIZATION-5: L/t6P+3VPXtH+Sslrdzx5Aesl7byWTR36rxkutR4WekjjOouA+524TIT9hmh [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: X-OPS-AUTHORIZATION-6: t42wePSmmysn0S0z8BvWRy73PlOlKd7GfZ7Kqi0vEw== [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: HOST: 127.0.0.1:443 [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: X-REMOTE-REQUEST-ID: 80364e30-fd88-4d9a-ab74-88bc4f588fa7 [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: x-ops-request-source: web [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: ---- End HTTP Request Header Data ---- [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: ---- HTTP Status and Header Data: ---- [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: HTTP 1.1 404 Not Found [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: date: Tue, 26 Oct 2021 09:09:54 GMT [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: content-type: text/html [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: transfer-encoding: chunked [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: connection: close [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: content-encoding: gzip [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: ---- End HTTP Status/Header Data ---- [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: ---- HTTP Response Body ---- [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: ³ɨɍ±㥲ɈML±³)ɬɉµ310Qퟮ�p˯͋±чڨ &委褔¼Ԣ;Ct@}¨4Ȭ "b$a}AúPG þN [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: ---- End HTTP Response Body ----- [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: Chef::HTTP calling Chef::HTTP::ValidateContentLength#handle_response [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: HTTP server did not include a Content-Length header in response, cannot identify truncated downloads. [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: Chef::HTTP calling Chef::HTTP::APIVersions#handle_response [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: Chef::HTTP calling Chef::HTTP::RemoteRequestID#handle_response [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: Chef::HTTP calling Chef::HTTP::Authenticator#handle_response [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: Chef::HTTP calling Chef::HTTP::Decompressor#handle_response [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: Decompressing gzip response [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: Chef::HTTP calling Chef::HTTP::CookieManager#handle_response [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: Chef::HTTP calling Chef::HTTP::JSONOutput#handle_response [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: Expected JSON response, but got content-type 'text/html' [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: Response body contains: [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0m [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0m404 Not Found [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0m [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0m

404 Not Found

[0m[1mnull_resource.chef_server_test (remote-exec):[0m [0m

[0m[1mnull_resource.chef_server_test (remote-exec):[0m [0m [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0m [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mTRACE: Chef::HTTP calling Chef::HTTP::JSONInput#handle_response [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mError Summary Report [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0m{ [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0m "timestamp": "2021-10-26 09:09:54 +0000", [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0m "message": "404 \"Not Found\"", [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0m "backtrace": [ [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0m "/opt/opscode/embedded/lib/ruby/2.7.0/net/http/response.rb:124:in `error!'", [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0m "/opt/opscode/embedded/lib/ruby/gems/2.7.0/gems/chef-16.13.16/lib/chef/http.rb:154:in `request'", [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0m "/opt/opscode/embedded/lib/ruby/gems/2.7.0/gems/chef-16.13.16/lib/chef/http.rb:117:in `get'", [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0m "/opt/opscode/embedded/lib/ruby/gems/2.7.0/gems/knife-ec-backup-2.5.3/lib/chef/knife/ec_base.rb:166:in `remote_users'", [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0m "/opt/opscode/embedded/lib/ruby/gems/2.7.0/gems/knife-ec-backup-2.5.3/lib/chef/knife/ec_backup.rb:85:in `for_each_user'", [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0m "/opt/opscode/embedded/lib/ruby/gems/2.7.0/gems/knife-ec-backup-2.5.3/lib/chef/knife/ec_backup.rb:34:in `run'", [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0m "/opt/opscode/embedded/lib/ruby/gems/2.7.0/gems/chef-16.13.16/lib/chef/knife.rb:490:in `block in run_with_pretty_exceptions'", [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0m "/opt/opscode/embedded/lib/ruby/gems/2.7.0/gems/chef-16.13.16/lib/chef/local_mode.rb:42:in `with_server_connectivity'", [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0m "/opt/opscode/embedded/lib/ruby/gems/2.7.0/gems/chef-16.13.16/lib/chef/knife.rb:489:in `run_with_pretty_exceptions'", [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0m "/opt/opscode/embedded/lib/ruby/gems/2.7.0/gems/chef-16.13.16/lib/chef/knife.rb:229:in `run'", [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0m "/opt/opscode/embedded/lib/ruby/gems/2.7.0/gems/chef-16.13.16/lib/chef/application/knife.rb:165:in `run'", [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0m "/opt/opscode/embedded/lib/ruby/gems/2.7.0/gems/chef-16.13.16/bin/knife:24:in `'", [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0m "/opt/opscode/bin/knife:146:in `load'", [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0m "/opt/opscode/bin/knife:146:in `

'" [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0m ], [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0m "exception": "Net::HTTPServerException" [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0m} [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mError(s) Summary file located at: '/var/opt/knife-ec-backup//errors/backup-2021-10-26T09:09:54+00:00.json' [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0m[1mTraceback[m (most recent call last): [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0m 12: from /opt/opscode/bin/knife:146:in `

' [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0m 11: from /opt/opscode/bin/knife:146:in `load' [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0m 10: from /opt/opscode/embedded/lib/ruby/gems/2.7.0/gems/chef-16.13.16/bin/knife:24:in `' [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0m 9: from /opt/opscode/embedded/lib/ruby/gems/2.7.0/gems/chef-16.13.16/lib/chef/application/knife.rb:165:in `run' [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0m 8: from /opt/opscode/embedded/lib/ruby/gems/2.7.0/gems/chef-16.13.16/lib/chef/knife.rb:229:in `run' [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0m 7: from /opt/opscode/embedded/lib/ruby/gems/2.7.0/gems/chef-16.13.16/lib/chef/knife.rb:489:in `run_with_pretty_exceptions' [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0m 6: from /opt/opscode/embedded/lib/ruby/gems/2.7.0/gems/chef-16.13.16/lib/chef/local_mode.rb:42:in `with_server_connectivity' [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0m 5: from /opt/opscode/embedded/lib/ruby/gems/2.7.0/gems/chef-16.13.16/lib/chef/knife.rb:490:in `block in run_with_pretty_exceptions' [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0m 4: from /opt/opscode/embedded/lib/ruby/gems/2.7.0/gems/knife-ec-backup-2.5.3/lib/chef/knife/ec_backup.rb:49:in `run' [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0m 3: from /opt/opscode/embedded/lib/ruby/gems/2.7.0/gems/knife-ec-backup-2.5.3/lib/chef/knife/ec_backup.rb:93:in `for_each_organization' [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0m 2: from /opt/opscode/embedded/lib/ruby/gems/2.7.0/gems/chef-16.13.16/lib/chef/http.rb:117:in `get' [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0m 1: from /opt/opscode/embedded/lib/ruby/gems/2.7.0/gems/chef-16.13.16/lib/chef/http.rb:154:in `request' [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0m/opt/opscode/embedded/lib/ruby/2.7.0/net/http/response.rb:124:in `error!': [1m404 "Not Found" ([1;4mNet::HTTPServerException[m[1m)[m [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0m-e [0m[1mnull_resource.chef_server_test (remote-exec):[0m [0mEND KNIFE EC BACKUP ```

lbakerchef commented 2 years ago

Output of chef-server-ctl tail while running sudo /opt/opscode/bin/knife ec backup /var/opt/knife-ec-backup/ --with-user-sql --with-key-sql -c /etc/opscode/pivotal.rb, external postgresql terraform scenario:

==> /var/log/opscode/nginx/access.log <==
127.0.0.1 - - [2021-10-26T19:26:45+00:00]  "GET /version HTTP/1.1" 200 "0.000" 2821 "-" "Chef Infra Client Knife/16.13.16 (ruby-2.7.4-p191; ohai-16.13.0; x86_64-linux; +https://chef.io)" "-" "-" "-" "16.13.16" "algorithm=sha1;version=1.1;" "pivotal" "2021-10-26T19:26:45Z" "2jmj7l5rSw0yVb/vlWAYkK/YBwk=" 1047 "e7bd6d00-9a2a-4946-81b7-73be89ffb821"
127.0.0.1 - - [2021-10-26T19:26:45+00:00]  "GET /users HTTP/1.1" 200 "0.010" 93 "-" "Chef Infra Client Knife/16.13.16 (ruby-2.7.4-p191; ohai-16.13.0; x86_64-linux; +https://chef.io)" "127.0.0.1:8000" "200" "0.012" "16.13.16" "algorithm=sha1;version=1.1;" "pivotal" "2021-10-26T19:26:45Z" "2jmj7l5rSw0yVb/vlWAYkK/YBwk=" 1045 "e7bd6d00-9a2a-4946-81b7-73be89ffb821"
127.0.0.1 - - [2021-10-26T19:26:45+00:00]  "GET /users/pivotal HTTP/1.1" 200 "0.010" 502 "-" "Chef Infra Client Knife/16.13.16 (ruby-2.7.4-p191; ohai-16.13.0; x86_64-linux; +https://chef.io)" "127.0.0.1:8000" "200" "0.008" "16.13.16" "algorithm=sha1;version=1.1;" "pivotal" "2021-10-26T19:26:45Z" "2jmj7l5rSw0yVb/vlWAYkK/YBwk=" 1053 "e7bd6d00-9a2a-4946-81b7-73be89ffb821"
127.0.0.1 - - [2021-10-26T19:26:45+00:00]  "GET /users/pivotal/_acl HTTP/1.1" 200 "0.021" 101 "-" "Chef Infra Client Knife/16.13.16 (ruby-2.7.4-p191; ohai-16.13.0; x86_64-linux; +https://chef.io)" "127.0.0.1:8000" "200" "0.024" "16.13.16" "algorithm=sha1;version=1.1;" "pivotal" "2021-10-26T19:26:45Z" "2jmj7l5rSw0yVb/vlWAYkK/YBwk=" 1058 "e7bd6d00-9a2a-4946-81b7-73be89ffb821"
127.0.0.1 - - [2021-10-26T19:26:45+00:00]  "GET /users/janedoe HTTP/1.1" 200 "0.010" 496 "-" "Chef Infra Client Knife/16.13.16 (ruby-2.7.4-p191; ohai-16.13.0; x86_64-linux; +https://chef.io)" "127.0.0.1:8000" "200" "0.012" "16.13.16" "algorithm=sha1;version=1.1;" "pivotal" "2021-10-26T19:26:45Z" "2jmj7l5rSw0yVb/vlWAYkK/YBwk=" 1053 "e7bd6d00-9a2a-4946-81b7-73be89ffb821"
127.0.0.1 - - [2021-10-26T19:26:45+00:00]  "GET /users/janedoe/_acl HTTP/1.1" 200 "0.027" 146 "-" "Chef Infra Client Knife/16.13.16 (ruby-2.7.4-p191; ohai-16.13.0; x86_64-linux; +https://chef.io)" "127.0.0.1:8000" "200" "0.028" "16.13.16" "algorithm=sha1;version=1.1;" "pivotal" "2021-10-26T19:26:45Z" "2jmj7l5rSw0yVb/vlWAYkK/YBwk=" 1058 "e7bd6d00-9a2a-4946-81b7-73be89ffb821"
127.0.0.1 - - [2021-10-26T19:26:45+00:00]  "GET /users/janedoeafter HTTP/1.1" 200 "0.009" 501 "-" "Chef Infra Client Knife/16.13.16 (ruby-2.7.4-p191; ohai-16.13.0; x86_64-linux; +https://chef.io)" "127.0.0.1:8000" "200" "0.008" "16.13.16" "algorithm=sha1;version=1.1;" "pivotal" "2021-10-26T19:26:45Z" "2jmj7l5rSw0yVb/vlWAYkK/YBwk=" 1058 "e7bd6d00-9a2a-4946-81b7-73be89ffb821"
127.0.0.1 - - [2021-10-26T19:26:45+00:00]  "GET /users/janedoeafter/_acl HTTP/1.1" 200 "0.026" 159 "-" "Chef Infra Client Knife/16.13.16 (ruby-2.7.4-p191; ohai-16.13.0; x86_64-linux; +https://chef.io)" "127.0.0.1:8000" "200" "0.024" "16.13.16" "algorithm=sha1;version=1.1;" "pivotal" "2021-10-26T19:26:45Z" "2jmj7l5rSw0yVb/vlWAYkK/YBwk=" 1063 "e7bd6d00-9a2a-4946-81b7-73be89ffb821"

==> /var/log/opscode/oc_bifrost/requests.log.1 <==
2021-10-26T19:26:45Z oc_bifrost@127.0.0.1 method=GET; path=/containers/239084da39b2dd7a8ac90b3423d59ed0/acl/read/actors/deb29c2977bcdd0dd6630adea9ea1127; status=200; requestor_id=deb29c2977bcdd0dd6630adea9ea1127; req_time=2; rdbms.bifrost_db.has_permission_time=0; rdbms.bifrost_db.has_permission_count=1; rdbms.bifrost_db.exists_time=0; rdbms.bifrost_db.exists_count=1;
2021-10-26T19:26:45Z oc_bifrost@127.0.0.1 method=GET; path=/actors/deb29c2977bcdd0dd6630adea9ea1127/acl/read/actors/deb29c2977bcdd0dd6630adea9ea1127; status=200; requestor_id=deb29c2977bcdd0dd6630adea9ea1127; req_time=2; rdbms.bifrost_db.has_permission_time=0; rdbms.bifrost_db.has_permission_count=1; rdbms.bifrost_db.exists_time=0; rdbms.bifrost_db.exists_count=1;
2021-10-26T19:26:45Z oc_bifrost@127.0.0.1 method=GET; path=/actors/deb29c2977bcdd0dd6630adea9ea1127/acl/grant/actors/deb29c2977bcdd0dd6630adea9ea1127; status=200; requestor_id=superuser; req_time=1; rdbms.bifrost_db.has_permission_time=1; rdbms.bifrost_db.has_permission_count=1;
2021-10-26T19:26:45Z oc_bifrost@127.0.0.1 method=GET; path=/actors/deb29c2977bcdd0dd6630adea9ea1127/acl; status=200; requestor_id=superuser; req_time=5; rdbms.bifrost_db.exists_time=0; rdbms.bifrost_db.exists_count=1; rdbms.bifrost_db.acl_membership_time=0; rdbms.bifrost_db.acl_membership_count=10;
2021-10-26T19:26:45Z oc_bifrost@127.0.0.1 method=GET; path=/actors/87b10aea22beadfa8cb969988cc04399/acl/read/actors/deb29c2977bcdd0dd6630adea9ea1127; status=200; requestor_id=deb29c2977bcdd0dd6630adea9ea1127; req_time=2; rdbms.bifrost_db.has_permission_time=0; rdbms.bifrost_db.has_permission_count=1; rdbms.bifrost_db.exists_time=0; rdbms.bifrost_db.exists_count=1;
2021-10-26T19:26:45Z oc_bifrost@127.0.0.1 method=GET; path=/actors/87b10aea22beadfa8cb969988cc04399/acl/grant/actors/deb29c2977bcdd0dd6630adea9ea1127; status=200; requestor_id=superuser; req_time=2; rdbms.bifrost_db.has_permission_time=0; rdbms.bifrost_db.has_permission_count=1;
2021-10-26T19:26:45Z oc_bifrost@127.0.0.1 method=GET; path=/actors/87b10aea22beadfa8cb969988cc04399/acl; status=200; requestor_id=superuser; req_time=5; rdbms.bifrost_db.exists_time=0; rdbms.bifrost_db.exists_count=1; rdbms.bifrost_db.acl_membership_time=0; rdbms.bifrost_db.acl_membership_count=10;
2021-10-26T19:26:45Z oc_bifrost@127.0.0.1 method=GET; path=/actors/4c20099aeebc751d7fa60718d3c2bece/acl/read/actors/deb29c2977bcdd0dd6630adea9ea1127; status=200; requestor_id=deb29c2977bcdd0dd6630adea9ea1127; req_time=2; rdbms.bifrost_db.has_permission_time=0; rdbms.bifrost_db.has_permission_count=1; rdbms.bifrost_db.exists_time=0; rdbms.bifrost_db.exists_count=1;
2021-10-26T19:26:45Z oc_bifrost@127.0.0.1 method=GET; path=/actors/4c20099aeebc751d7fa60718d3c2bece/acl/grant/actors/deb29c2977bcdd0dd6630adea9ea1127; status=200; requestor_id=superuser; req_time=1; rdbms.bifrost_db.has_permission_time=0; rdbms.bifrost_db.has_permission_count=1;
2021-10-26T19:26:45Z oc_bifrost@127.0.0.1 method=GET; path=/actors/4c20099aeebc751d7fa60718d3c2bece/acl; status=200; requestor_id=superuser; req_time=6; rdbms.bifrost_db.exists_time=0; rdbms.bifrost_db.exists_count=1; rdbms.bifrost_db.acl_membership_time=0; rdbms.bifrost_db.acl_membership_count=10;

==> /var/log/opscode/opscode-erchef/requests.log.1 <==
2021-10-26T19:26:45Z erchef@127.0.0.1 method=GET; path=/users; status=200; req_id=g3IAA2QAEGVyY2hlZkAxMjcuMC4wLjECAALazBNgAAG6ZN6E; couchdb_groups=false; couchdb_organizations=false; couchdb_containers=false; couchdb_acls=false; 503_mode=false; couchdb_association_requests=false; couchdb_associations=false; req_time=8; rdbms_time=0; rdbms_count=3; authz_time=4; authz_count=1; user=pivotal; req_api_version=0;
2021-10-26T19:26:45Z erchef@127.0.0.1 method=GET; path=/users/pivotal; status=200; req_id=g3IAA2QAEGVyY2hlZkAxMjcuMC4wLjECAALbBRNgAAG6ZN6E; couchdb_groups=false; couchdb_organizations=false; couchdb_containers=false; couchdb_acls=false; 503_mode=false; couchdb_association_requests=false; couchdb_associations=false; req_time=7; rdbms_time=0; rdbms_count=2; authz_time=4; authz_count=1; user=pivotal; req_api_version=0;
2021-10-26T19:26:45Z erchef@127.0.0.1 method=GET; path=/users/pivotal/_acl; status=200; req_id=g3IAA2QAEGVyY2hlZkAxMjcuMC4wLjECAALbNRNgAAG6ZN6E; couchdb_groups=false; couchdb_organizations=false; couchdb_containers=false; couchdb_acls=false; 503_mode=false; couchdb_association_requests=false; couchdb_associations=false; req_time=18; rdbms_time=0; rdbms_count=2; user=pivotal; req_api_version=0;
2021-10-26T19:26:45Z erchef@127.0.0.1 method=GET; path=/users/janedoe; status=200; req_id=g3IAA2QAEGVyY2hlZkAxMjcuMC4wLjECAALbrRNgAAG6ZN6E; couchdb_groups=false; couchdb_organizations=false; couchdb_containers=false; couchdb_acls=false; 503_mode=false; couchdb_association_requests=false; couchdb_associations=false; req_time=7; rdbms_time=1; rdbms_count=2; authz_time=3; authz_count=1; user=pivotal; req_api_version=0;
2021-10-26T19:26:45Z erchef@127.0.0.1 method=GET; path=/users/janedoe/_acl; status=200; req_id=g3IAA2QAEGVyY2hlZkAxMjcuMC4wLjECAALb3RNgAAG6ZN6E; couchdb_groups=false; couchdb_organizations=false; couchdb_containers=false; couchdb_acls=false; 503_mode=false; couchdb_association_requests=false; couchdb_associations=false; req_time=24; rdbms_time=0; rdbms_count=2; user=pivotal; req_api_version=0;
2021-10-26T19:26:45Z erchef@127.0.0.1 method=GET; path=/users/janedoeafter; status=200; req_id=g3IAA2QAEGVyY2hlZkAxMjcuMC4wLjECAALcbRNgAAG6ZN6E; couchdb_groups=false; couchdb_organizations=false; couchdb_containers=false; couchdb_acls=false; 503_mode=false; couchdb_association_requests=false; couchdb_associations=false; req_time=6; rdbms_time=0; rdbms_count=2; authz_time=3; authz_count=1; user=pivotal; req_api_version=0;
2021-10-26T19:26:45Z erchef@127.0.0.1 method=GET; path=/users/janedoeafter/_acl; status=200; req_id=g3IAA2QAEGVyY2hlZkAxMjcuMC4wLjECAALcnxNgAAG6ZN6E; couchdb_groups=false; couchdb_organizations=false; couchdb_containers=false; couchdb_acls=false; 503_mode=false; couchdb_association_requests=false; couchdb_associations=false; req_time=23; rdbms_time=0; rdbms_count=2; user=pivotal; req_api_version=0;

lbakerchef commented 2 years ago

Created new standalone fresh install scenario. The initial knife-ec-backup test within the running scenario failed. After logging into the box and re-running it, it passed.

Setup:

mac:aws lbaker$ PLATFORM=ubuntu-18.04 INSTALL_VERSION=14.10.23 UPGRADE_VERSION=14.10.23 SCENARIO=standalone-fresh-install ENABLE_ADDON_PUSH_JOBS=false ENABLE_GATHER_LOGS_TEST=false ENABLE_PEDANT_TEST=false ENABLE_PSQL_TEST=true ENABLE_SMOKE_TEST=false ENABLE_IPV6=false ENABLE_ADDON_CHEF_MANAGE=false make apply

Initial test:

null_resource.chef_server_test (remote-exec): BEGIN KNIFE EC BACKUP

null_resource.chef_server_test (remote-exec): /opt/opscode/embedded/lib/ruby/gems/2.7.0/gems/chef-vault-4.1.4/lib/chef-vault/version.rb:18: warning: already initialized constant ChefVault::VERSION
null_resource.chef_server_test (remote-exec): /opt/opscode/embedded/lib/ruby/gems/2.7.0/gems/chef-vault-4.1.0/lib/chef-vault/version.rb:18: warning: previous definition of VERSION was here
null_resource.chef_server_test (remote-exec): /opt/opscode/embedded/lib/ruby/gems/2.7.0/gems/chef-vault-4.1.4/lib/chef-vault/version.rb:19: warning: already initialized constant ChefVault::MAJOR
null_resource.chef_server_test (remote-exec): /opt/opscode/embedded/lib/ruby/gems/2.7.0/gems/chef-vault-4.1.0/lib/chef-vault/version.rb:19: warning: previous definition of MAJOR was here
null_resource.chef_server_test (remote-exec): /opt/opscode/embedded/lib/ruby/gems/2.7.0/gems/chef-vault-4.1.4/lib/chef-vault/version.rb:19: warning: already initialized constant ChefVault::MINOR
null_resource.chef_server_test (remote-exec): /opt/opscode/embedded/lib/ruby/gems/2.7.0/gems/chef-vault-4.1.0/lib/chef-vault/version.rb:19: warning: previous definition of MINOR was here
null_resource.chef_server_test (remote-exec): /opt/opscode/embedded/lib/ruby/gems/2.7.0/gems/chef-vault-4.1.4/lib/chef-vault/version.rb:19: warning: already initialized constant ChefVault::TINY
null_resource.chef_server_test (remote-exec): /opt/opscode/embedded/lib/ruby/gems/2.7.0/gems/chef-vault-4.1.0/lib/chef-vault/version.rb:19: warning: previous definition of TINY was here
null_resource.chef_server_test (remote-exec): INFO: Using configuration from /etc/opscode/pivotal.rb
null_resource.chef_server_test (remote-exec): TRACE: Chef::HTTP calling Chef::HTTP::JSONInput#handle_request
null_resource.chef_server_test (remote-exec): TRACE: Chef::HTTP calling Chef::HTTP::JSONOutput#handle_request
null_resource.chef_server_test (remote-exec): TRACE: Chef::HTTP calling Chef::HTTP::CookieManager#handle_request
null_resource.chef_server_test (remote-exec): TRACE: Chef::HTTP calling Chef::HTTP::Decompressor#handle_request
null_resource.chef_server_test (remote-exec): TRACE: Chef::HTTP calling Chef::HTTP::Authenticator#handle_request
null_resource.chef_server_test (remote-exec): TRACE: Signing the request as pivotal
null_resource.chef_server_test (remote-exec): TRACE: Chef::HTTP calling Chef::HTTP::RemoteRequestID#handle_request
null_resource.chef_server_test (remote-exec): TRACE: Chef::HTTP calling Chef::HTTP::APIVersions#handle_request
null_resource.chef_server_test (remote-exec): TRACE: Chef::HTTP calling Chef::HTTP::ValidateContentLength#handle_request
null_resource.chef_server_test (remote-exec): TRACE: Initiating GET to https://127.0.0.1/version
null_resource.chef_server_test (remote-exec): TRACE: ---- HTTP Request Header Data: ----
null_resource.chef_server_test (remote-exec): TRACE: Accept: application/json
null_resource.chef_server_test (remote-exec): TRACE: Accept-Encoding: gzip;q=1.0,deflate;q=0.6,identity;q=0.3
null_resource.chef_server_test (remote-exec): TRACE: X-Ops-Server-API-Version: 2
null_resource.chef_server_test (remote-exec): TRACE: X-OPS-SIGN: algorithm=sha1;version=1.1;
null_resource.chef_server_test (remote-exec): TRACE: X-OPS-USERID: pivotal
null_resource.chef_server_test (remote-exec): TRACE: X-OPS-TIMESTAMP: 2021-10-26T21:14:16Z
null_resource.chef_server_test (remote-exec): TRACE: X-OPS-CONTENT-HASH: 2jmj7l5rSw0yVb/vlWAYkK/YBwk=
null_resource.chef_server_test (remote-exec): TRACE: X-OPS-AUTHORIZATION-1: rq6Fb1fZ6XuZqUGNfTQwcLkAVbzXvPqwl9YWSGpcgxMSV42xQ1tRrKc3/NvX
null_resource.chef_server_test (remote-exec): TRACE: X-OPS-AUTHORIZATION-2: Uobi74R5LtX62XkHHc4WS+TJ4NKyCQ1kMyaL6KGtVapmNhloVTBWLyenKJUA
null_resource.chef_server_test (remote-exec): TRACE: X-OPS-AUTHORIZATION-3: gm7gh1vd5yB4ngOQ6WrX/owlasxn9aYjW455FLWAOkPM7K7qKo8BWLsS1yjW
null_resource.chef_server_test (remote-exec): TRACE: X-OPS-AUTHORIZATION-4: TXwlO/4E0BNITZaKNbZ/fjCZVZ6t1zwO7P8xt6xDpdRK66sWocWJ9+OxMDhc
null_resource.chef_server_test (remote-exec): TRACE: X-OPS-AUTHORIZATION-5: vstufpsAIcxk/cb45H8RyEm8V+a/qlHxMEcvr7RHxZ4E653MNn7WSQZzwGW4
null_resource.chef_server_test (remote-exec): TRACE: X-OPS-AUTHORIZATION-6: 03Uxnnw0+iRaEj89kw6SQr3xTi3WlcpZVht1W4Ot/A==
null_resource.chef_server_test (remote-exec): TRACE: HOST: 127.0.0.1:443
null_resource.chef_server_test (remote-exec): TRACE: X-REMOTE-REQUEST-ID: 4ce2732b-62d8-43dd-a827-3ab3a31f5cbb
null_resource.chef_server_test (remote-exec): TRACE: x-ops-request-source: web
null_resource.chef_server_test (remote-exec): TRACE: ---- End HTTP Request Header Data ----
null_resource.chef_server_test (remote-exec): TRACE: ---- HTTP Status and Header Data: ----
null_resource.chef_server_test (remote-exec): TRACE: HTTP 1.1 200 OK
null_resource.chef_server_test (remote-exec): TRACE: date: Tue, 26 Oct 2021 21:14:16 GMT
null_resource.chef_server_test (remote-exec): TRACE: content-type: text/plain
null_resource.chef_server_test (remote-exec): TRACE: last-modified: Thu, 21 Oct 2021 18:54:23 GMT
null_resource.chef_server_test (remote-exec): TRACE: transfer-encoding: chunked
null_resource.chef_server_test (remote-exec): TRACE: connection: close
null_resource.chef_server_test (remote-exec): TRACE: etag: W/"6171b75f-2374"
null_resource.chef_server_test (remote-exec): TRACE: content-security-policy: default-src 'self';
null_resource.chef_server_test (remote-exec): TRACE: x-frame-options: DENY
null_resource.chef_server_test (remote-exec): TRACE: content-encoding: gzip
null_resource.chef_server_test (remote-exec): TRACE: ---- End HTTP Status/Header Data ----
null_resource.chef_server_test (remote-exec): TRACE: Chef::HTTP calling Chef::HTTP::ValidateContentLength#handle_response
null_resource.chef_server_test (remote-exec): TRACE: HTTP server did not include a Content-Length header in response, cannot identify truncated downloads.
null_resource.chef_server_test (remote-exec): TRACE: Chef::HTTP calling Chef::HTTP::APIVersions#handle_response
null_resource.chef_server_test (remote-exec): TRACE: Chef::HTTP calling Chef::HTTP::RemoteRequestID#handle_response
null_resource.chef_server_test (remote-exec): TRACE: Chef::HTTP calling Chef::HTTP::Authenticator#handle_response
null_resource.chef_server_test (remote-exec): TRACE: Chef::HTTP calling Chef::HTTP::Decompressor#handle_response
null_resource.chef_server_test (remote-exec): TRACE: Decompressing gzip response
null_resource.chef_server_test (remote-exec): TRACE: Chef::HTTP calling Chef::HTTP::CookieManager#handle_response
null_resource.chef_server_test (remote-exec): TRACE: Chef::HTTP calling Chef::HTTP::JSONOutput#handle_response
null_resource.chef_server_test (remote-exec): TRACE: Expected JSON response, but got content-type 'text/plain'
null_resource.chef_server_test (remote-exec): TRACE: Response body contains:
null_resource.chef_server_test (remote-exec): chef-server 14.10.23

null_resource.chef_server_test (remote-exec): Component                  Installed Version   Version GUID                                                                                                                              Overridden From
null_resource.chef_server_test (remote-exec): ------------------------------- [...truncated...]
null_resource.chef_server_test (remote-exec): TRACE: Chef::HTTP calling Chef::HTTP::JSONInput#handle_response
null_resource.chef_server_test (remote-exec): Downloading Users
null_resource.chef_server_test (remote-exec): TRACE: Chef::HTTP calling Chef::HTTP::JSONInput#handle_request
null_resource.chef_server_test (remote-exec): TRACE: Chef::HTTP calling Chef::HTTP::JSONOutput#handle_request
null_resource.chef_server_test (remote-exec): TRACE: Chef::HTTP calling Chef::HTTP::CookieManager#handle_request
null_resource.chef_server_test (remote-exec): TRACE: Chef::HTTP calling Chef::HTTP::Decompressor#handle_request
null_resource.chef_server_test (remote-exec): TRACE: Chef::HTTP calling Chef::HTTP::Authenticator#handle_request
null_resource.chef_server_test (remote-exec): TRACE: Signing the request as pivotal
null_resource.chef_server_test (remote-exec): TRACE: Chef::HTTP calling Chef::HTTP::RemoteRequestID#handle_request
null_resource.chef_server_test (remote-exec): TRACE: Chef::HTTP calling Chef::HTTP::APIVersions#handle_request
null_resource.chef_server_test (remote-exec): TRACE: Chef::HTTP calling Chef::HTTP::ValidateContentLength#handle_request
null_resource.chef_server_test (remote-exec): TRACE: Initiating GET to https://127.0.0.1/users
null_resource.chef_server_test (remote-exec): TRACE: ---- HTTP Request Header Data: ----
null_resource.chef_server_test (remote-exec): TRACE: Accept: application/json
null_resource.chef_server_test (remote-exec): TRACE: Accept-Encoding: gzip;q=1.0,deflate;q=0.6,identity;q=0.3
null_resource.chef_server_test (remote-exec): TRACE: X-Ops-Server-API-Version: 0
null_resource.chef_server_test (remote-exec): TRACE: X-OPS-SIGN: algorithm=sha1;version=1.1;
null_resource.chef_server_test (remote-exec): TRACE: X-OPS-USERID: pivotal
null_resource.chef_server_test (remote-exec): TRACE: X-OPS-TIMESTAMP: 2021-10-26T21:14:16Z
null_resource.chef_server_test (remote-exec): TRACE: X-OPS-CONTENT-HASH: 2jmj7l5rSw0yVb/vlWAYkK/YBwk=
null_resource.chef_server_test (remote-exec): TRACE: X-OPS-AUTHORIZATION-1: g2U+oVFtt1iU/2JN33gVt430aKsr/x1LjOPT4UxaQ4od5G7Sj7eipth8X8Nv
null_resource.chef_server_test (remote-exec): TRACE: X-OPS-AUTHORIZATION-2: StgZg+sVmaPLAClehs5iAcSTBsB7jYQNlui5pUGnS7ebqRdA3sQ1V7V/8+/s
null_resource.chef_server_test (remote-exec): TRACE: X-OPS-AUTHORIZATION-3: 1iWQJ7V+hC9rSb6NmgujsBPjJS1+0Nnlcor7+XEnaSaYeb7MejHwESYZaow6
null_resource.chef_server_test (remote-exec): TRACE: X-OPS-AUTHORIZATION-4: +WBYjiWPr+QKwcAGPhlfvDxF3k+sUz7bBRr04CUNnCQgPgjzsUy3chxc9Pi1
null_resource.chef_server_test (remote-exec): TRACE: X-OPS-AUTHORIZATION-5: 8syr8FjLbfg3hd6Hk3IfXrfkp7TngbWFDtjoTpFNLBFmNCiunFxJ5eRC6Pjs
null_resource.chef_server_test (remote-exec): TRACE: X-OPS-AUTHORIZATION-6: +Vd3GKrPz/Oqe+QyAHuYMV0iXpntTQAXVHlVxhWGGw==
null_resource.chef_server_test (remote-exec): TRACE: HOST: 127.0.0.1:443
null_resource.chef_server_test (remote-exec): TRACE: X-REMOTE-REQUEST-ID: 4ce2732b-62d8-43dd-a827-3ab3a31f5cbb
null_resource.chef_server_test (remote-exec): TRACE: x-ops-request-source: web
null_resource.chef_server_test (remote-exec): TRACE: ---- End HTTP Request Header Data ----
null_resource.chef_server_test (remote-exec): TRACE: ---- HTTP Status and Header Data: ----
null_resource.chef_server_test (remote-exec): TRACE: HTTP 1.1 404 Not Found
null_resource.chef_server_test (remote-exec): TRACE: date: Tue, 26 Oct 2021 21:14:16 GMT
null_resource.chef_server_test (remote-exec): TRACE: content-type: text/html
null_resource.chef_server_test (remote-exec): TRACE: transfer-encoding: chunked
null_resource.chef_server_test (remote-exec): TRACE: connection: close
null_resource.chef_server_test (remote-exec): TRACE: content-encoding: gzip
null_resource.chef_server_test (remote-exec): TRACE: ---- End HTTP Status/Header Data ----
null_resource.chef_server_test (remote-exec): TRACE: ---- HTTP Response Body ----
null_resource.chef_server_test (remote-exec): TRACE: ▒▒(▒ͱ▒▒▒HML▒▒)▒,▒I▒310Q▒▒/Qp▒/▒K▒▒胕▒&▒T▒▒▒Լ▒▒";▒
                                                                                                      Ct@▒4▒l▒"▒b$a}▒A▒PG▒▒N▒
null_resource.chef_server_test (remote-exec): TRACE: ---- End HTTP Response Body -----
null_resource.chef_server_test (remote-exec): TRACE: Chef::HTTP calling Chef::HTTP::ValidateContentLength#handle_response
null_resource.chef_server_test (remote-exec): TRACE: HTTP server did not include a Content-Length header in response, cannot identify truncated downloads.
null_resource.chef_server_test (remote-exec): TRACE: Chef::HTTP calling Chef::HTTP::APIVersions#handle_response
null_resource.chef_server_test (remote-exec): TRACE: Chef::HTTP calling Chef::HTTP::RemoteRequestID#handle_response
null_resource.chef_server_test (remote-exec): TRACE: Chef::HTTP calling Chef::HTTP::Authenticator#handle_response
null_resource.chef_server_test (remote-exec): TRACE: Chef::HTTP calling Chef::HTTP::Decompressor#handle_response
null_resource.chef_server_test (remote-exec): TRACE: Decompressing gzip response
null_resource.chef_server_test (remote-exec): TRACE: Chef::HTTP calling Chef::HTTP::CookieManager#handle_response
null_resource.chef_server_test (remote-exec): TRACE: Chef::HTTP calling Chef::HTTP::JSONOutput#handle_response
null_resource.chef_server_test (remote-exec): TRACE: Expected JSON response, but got content-type 'text/html'
null_resource.chef_server_test (remote-exec): TRACE: Response body contains:
null_resource.chef_server_test (remote-exec): <html>
null_resource.chef_server_test (remote-exec): <head><title>404 Not Found</title></head>
null_resource.chef_server_test (remote-exec): <body>
null_resource.chef_server_test (remote-exec): <center><h1>404 Not Found</h1></center>
null_resource.chef_server_test (remote-exec): <hr><center></center>
null_resource.chef_server_test (remote-exec): </body>
null_resource.chef_server_test (remote-exec): </html>

null_resource.chef_server_test (remote-exec): TRACE: Chef::HTTP calling Chef::HTTP::JSONInput#handle_response
null_resource.chef_server_test (remote-exec): TRACE: Chef::HTTP calling Chef::HTTP::JSONInput#handle_request
null_resource.chef_server_test (remote-exec): TRACE: Chef::HTTP calling Chef::HTTP::JSONOutput#handle_request
null_resource.chef_server_test (remote-exec): TRACE: Chef::HTTP calling Chef::HTTP::CookieManager#handle_request
null_resource.chef_server_test (remote-exec): TRACE: Chef::HTTP calling Chef::HTTP::Decompressor#handle_request
null_resource.chef_server_test (remote-exec): TRACE: Chef::HTTP calling Chef::HTTP::Authenticator#handle_request
null_resource.chef_server_test (remote-exec): TRACE: Signing the request as pivotal
null_resource.chef_server_test (remote-exec): TRACE: Chef::HTTP calling Chef::HTTP::RemoteRequestID#handle_request
null_resource.chef_server_test (remote-exec): TRACE: Chef::HTTP calling Chef::HTTP::APIVersions#handle_request
null_resource.chef_server_test (remote-exec): TRACE: Chef::HTTP calling Chef::HTTP::ValidateContentLength#handle_request
null_resource.chef_server_test (remote-exec): TRACE: Initiating GET to https://127.0.0.1/organizations
null_resource.chef_server_test (remote-exec): TRACE: ---- HTTP Request Header Data: ----
null_resource.chef_server_test (remote-exec): TRACE: Accept: application/json
null_resource.chef_server_test (remote-exec): TRACE: Accept-Encoding: gzip;q=1.0,deflate;q=0.6,identity;q=0.3
null_resource.chef_server_test (remote-exec): TRACE: X-Ops-Server-API-Version: 0
null_resource.chef_server_test (remote-exec): TRACE: X-OPS-SIGN: algorithm=sha1;version=1.1;
null_resource.chef_server_test (remote-exec): TRACE: X-OPS-USERID: pivotal
null_resource.chef_server_test (remote-exec): TRACE: X-OPS-TIMESTAMP: 2021-10-26T21:14:16Z
null_resource.chef_server_test (remote-exec): TRACE: X-OPS-CONTENT-HASH: 2jmj7l5rSw0yVb/vlWAYkK/YBwk=
null_resource.chef_server_test (remote-exec): TRACE: X-OPS-AUTHORIZATION-1: eaJI1jZImTfxbfsHS8gv4x+zVl6xs+nr7FuPDEqoZIoHfHQTnjHGHyjRIuUU
null_resource.chef_server_test (remote-exec): TRACE: X-OPS-AUTHORIZATION-2: z8ZU5uE6rAYETHNwRTVJg2rb5SP+Ni1LKX1UFFhhIIWemv1AQIrSSa0NfkB4
null_resource.chef_server_test (remote-exec): TRACE: X-OPS-AUTHORIZATION-3: ze0h120tKDnbmCOXq/Bq/2d8YoEzJ4MxGlte28mhypPIFtAVQeTb4IXfyshb
null_resource.chef_server_test (remote-exec): TRACE: X-OPS-AUTHORIZATION-4: o8g30fXP/DpCciJr2/pJ0tmDO8Nhg6PWKvgbblI8/ECfsEv3wu+i/o8Wxzp3
null_resource.chef_server_test (remote-exec): TRACE: X-OPS-AUTHORIZATION-5: 8pzocx2M7Hv0c/gvv4oouknRZ4GR+a2OeOLVFd2ci8A983ExooqnCl423uEG
null_resource.chef_server_test (remote-exec): TRACE: X-OPS-AUTHORIZATION-6: aiJF19g10RRldCy9LCgSxqtChs45tQ8st2GIyuGQsg==
null_resource.chef_server_test (remote-exec): TRACE: HOST: 127.0.0.1:443
null_resource.chef_server_test (remote-exec): TRACE: X-REMOTE-REQUEST-ID: 4ce2732b-62d8-43dd-a827-3ab3a31f5cbb
null_resource.chef_server_test (remote-exec): TRACE: x-ops-request-source: web
null_resource.chef_server_test (remote-exec): TRACE: ---- End HTTP Request Header Data ----
null_resource.chef_server_test (remote-exec): TRACE: ---- HTTP Status and Header Data: ----
null_resource.chef_server_test (remote-exec): TRACE: HTTP 1.1 404 Not Found
null_resource.chef_server_test (remote-exec): TRACE: date: Tue, 26 Oct 2021 21:14:16 GMT
null_resource.chef_server_test (remote-exec): TRACE: content-type: text/html
null_resource.chef_server_test (remote-exec): TRACE: transfer-encoding: chunked
null_resource.chef_server_test (remote-exec): TRACE: connection: close
null_resource.chef_server_test (remote-exec): TRACE: content-encoding: gzip
null_resource.chef_server_test (remote-exec): TRACE: ---- End HTTP Status/Header Data ----
null_resource.chef_server_test (remote-exec): TRACE: ---- HTTP Response Body ----
null_resource.chef_server_test (remote-exec): TRACE: ▒▒(▒ͱ▒▒▒HML▒▒)▒,▒I▒310Q▒▒/Qp▒/▒K▒▒胕▒&▒T▒▒▒Լ▒▒";▒
                                                                                                      Ct@▒4▒l▒"▒b$a}▒A▒PG▒▒N▒
null_resource.chef_server_test (remote-exec): TRACE: ---- End HTTP Response Body -----
null_resource.chef_server_test (remote-exec): TRACE: Chef::HTTP calling Chef::HTTP::ValidateContentLength#handle_response
null_resource.chef_server_test (remote-exec): TRACE: HTTP server did not include a Content-Length header in response, cannot identify truncated downloads.
null_resource.chef_server_test (remote-exec): TRACE: Chef::HTTP calling Chef::HTTP::APIVersions#handle_response
null_resource.chef_server_test (remote-exec): TRACE: Chef::HTTP calling Chef::HTTP::RemoteRequestID#handle_response
null_resource.chef_server_test (remote-exec): TRACE: Chef::HTTP calling Chef::HTTP::Authenticator#handle_response
null_resource.chef_server_test (remote-exec): TRACE: Chef::HTTP calling Chef::HTTP::Decompressor#handle_response
null_resource.chef_server_test (remote-exec): TRACE: Decompressing gzip response
null_resource.chef_server_test (remote-exec): TRACE: Chef::HTTP calling Chef::HTTP::CookieManager#handle_response
null_resource.chef_server_test (remote-exec): TRACE: Chef::HTTP calling Chef::HTTP::JSONOutput#handle_response
null_resource.chef_server_test (remote-exec): TRACE: Expected JSON response, but got content-type 'text/html'
null_resource.chef_server_test (remote-exec): TRACE: Response body contains:
null_resource.chef_server_test (remote-exec): <html>
null_resource.chef_server_test (remote-exec): <head><title>404 Not Found</title></head>
null_resource.chef_server_test (remote-exec): <body>
null_resource.chef_server_test (remote-exec): <center><h1>404 Not Found</h1></center>
null_resource.chef_server_test (remote-exec): <hr><center></center>
null_resource.chef_server_test (remote-exec): </body>
null_resource.chef_server_test (remote-exec): </html>

null_resource.chef_server_test (remote-exec): TRACE: Chef::HTTP calling Chef::HTTP::JSONInput#handle_response

null_resource.chef_server_test (remote-exec): Error Summary Report
null_resource.chef_server_test (remote-exec): {
null_resource.chef_server_test (remote-exec):   "timestamp": "2021-10-26 21:14:16 +0000",
null_resource.chef_server_test (remote-exec):   "message": "404 \"Not Found\"",
null_resource.chef_server_test (remote-exec):   "backtrace": [
null_resource.chef_server_test (remote-exec):     "/opt/opscode/embedded/lib/ruby/2.7.0/net/http/response.rb:124:in `error!'",
null_resource.chef_server_test (remote-exec):     "/opt/opscode/embedded/lib/ruby/gems/2.7.0/gems/chef-16.13.16/lib/chef/http.rb:154:in `request'",
null_resource.chef_server_test (remote-exec):     "/opt/opscode/embedded/lib/ruby/gems/2.7.0/gems/chef-16.13.16/lib/chef/http.rb:117:in `get'",
null_resource.chef_server_test (remote-exec):     "/opt/opscode/embedded/lib/ruby/gems/2.7.0/gems/knife-ec-backup-2.5.3/lib/chef/knife/ec_base.rb:166:in `remote_users'",
null_resource.chef_server_test (remote-exec):     "/opt/opscode/embedded/lib/ruby/gems/2.7.0/gems/knife-ec-backup-2.5.3/lib/chef/knife/ec_backup.rb:85:in `for_each_user'",
null_resource.chef_server_test (remote-exec):     "/opt/opscode/embedded/lib/ruby/gems/2.7.0/gems/knife-ec-backup-2.5.3/lib/chef/knife/ec_backup.rb:34:in `run'",
null_resource.chef_server_test (remote-exec):     "/opt/opscode/embedded/lib/ruby/gems/2.7.0/gems/chef-16.13.16/lib/chef/knife.rb:490:in `block in run_with_pretty_exceptions'",
null_resource.chef_server_test (remote-exec):     "/opt/opscode/embedded/lib/ruby/gems/2.7.0/gems/chef-16.13.16/lib/chef/local_mode.rb:42:in `with_server_connectivity'",
null_resource.chef_server_test (remote-exec):     "/opt/opscode/embedded/lib/ruby/gems/2.7.0/gems/chef-16.13.16/lib/chef/knife.rb:489:in `run_with_pretty_exceptions'",
null_resource.chef_server_test (remote-exec):     "/opt/opscode/embedded/lib/ruby/gems/2.7.0/gems/chef-16.13.16/lib/chef/knife.rb:229:in `run'",
null_resource.chef_server_test (remote-exec):     "/opt/opscode/embedded/lib/ruby/gems/2.7.0/gems/chef-16.13.16/lib/chef/application/knife.rb:165:in `run'",
null_resource.chef_server_test (remote-exec):     "/opt/opscode/embedded/lib/ruby/gems/2.7.0/gems/chef-16.13.16/bin/knife:24:in `<top (required)>'",
null_resource.chef_server_test (remote-exec):     "/opt/opscode/bin/knife:146:in `load'",
null_resource.chef_server_test (remote-exec):     "/opt/opscode/bin/knife:146:in `<main>'"
null_resource.chef_server_test (remote-exec):   ],
null_resource.chef_server_test (remote-exec):   "exception": "Net::HTTPServerException"
null_resource.chef_server_test (remote-exec): }

null_resource.chef_server_test (remote-exec): Error(s) Summary file located at: '/var/opt/knife-ec-backup//errors/backup-2021-10-26T21:14:16+00:00.json'
null_resource.chef_server_test (remote-exec): Traceback (most recent call last):
null_resource.chef_server_test (remote-exec):   12: from /opt/opscode/bin/knife:146:in `<main>'
null_resource.chef_server_test (remote-exec):   11: from /opt/opscode/bin/knife:146:in `load'
null_resource.chef_server_test (remote-exec):   10: from /opt/opscode/embedded/lib/ruby/gems/2.7.0/gems/chef-16.13.16/bin/knife:24:in `<top (required)>'
null_resource.chef_server_test (remote-exec):    9: from /opt/opscode/embedded/lib/ruby/gems/2.7.0/gems/chef-16.13.16/lib/chef/application/knife.rb:165:in `run'
null_resource.chef_server_test (remote-exec):    8: from /opt/opscode/embedded/lib/ruby/gems/2.7.0/gems/chef-16.13.16/lib/chef/knife.rb:229:in `run'
null_resource.chef_server_test (remote-exec):    7: from /opt/opscode/embedded/lib/ruby/gems/2.7.0/gems/chef-16.13.16/lib/chef/knife.rb:489:in `run_with_pretty_exceptions'
null_resource.chef_server_test (remote-exec):    6: from /opt/opscode/embedded/lib/ruby/gems/2.7.0/gems/chef-16.13.16/lib/chef/local_mode.rb:42:in `with_server_connectivity'
null_resource.chef_server_test (remote-exec):    5: from /opt/opscode/embedded/lib/ruby/gems/2.7.0/gems/chef-16.13.16/lib/chef/knife.rb:490:in `block in run_with_pretty_exceptions'
null_resource.chef_server_test (remote-exec):    4: from /opt/opscode/embedded/lib/ruby/gems/2.7.0/gems/knife-ec-backup-2.5.3/lib/chef/knife/ec_backup.rb:49:in `run'
null_resource.chef_server_test (remote-exec):    3: from /opt/opscode/embedded/lib/ruby/gems/2.7.0/gems/knife-ec-backup-2.5.3/lib/chef/knife/ec_backup.rb:93:in `for_each_organization'
null_resource.chef_server_test (remote-exec):    2: from /opt/opscode/embedded/lib/ruby/gems/2.7.0/gems/chef-16.13.16/lib/chef/http.rb:117:in `get'
null_resource.chef_server_test (remote-exec):    1: from /opt/opscode/embedded/lib/ruby/gems/2.7.0/gems/chef-16.13.16/lib/chef/http.rb:154:in `request'
null_resource.chef_server_test (remote-exec): /opt/opscode/embedded/lib/ruby/2.7.0/net/http/response.rb:124:in `error!': 404 "Not Found" (Net::HTTPServerException)
null_resource.chef_server_test (remote-exec): -e
null_resource.chef_server_test (remote-exec): END KNIFE EC BACKUP

Login to box:

ubuntu@ip-10-0-12-28:~$ sudo /opt/opscode/bin/knife ec backup /var/opt/knife-ec-backup/ --with-user-sql --with-key-sql -c /etc/opscode/pivotal.rb
/opt/opscode/embedded/lib/ruby/gems/2.7.0/gems/chef-vault-4.1.4/lib/chef-vault/version.rb:18: warning: already initialized constant ChefVault::VERSION
/opt/opscode/embedded/lib/ruby/gems/2.7.0/gems/chef-vault-4.1.0/lib/chef-vault/version.rb:18: warning: previous definition of VERSION was here
/opt/opscode/embedded/lib/ruby/gems/2.7.0/gems/chef-vault-4.1.4/lib/chef-vault/version.rb:19: warning: already initialized constant ChefVault::MAJOR
/opt/opscode/embedded/lib/ruby/gems/2.7.0/gems/chef-vault-4.1.0/lib/chef-vault/version.rb:19: warning: previous definition of MAJOR was here
/opt/opscode/embedded/lib/ruby/gems/2.7.0/gems/chef-vault-4.1.4/lib/chef-vault/version.rb:19: warning: already initialized constant ChefVault::MINOR
/opt/opscode/embedded/lib/ruby/gems/2.7.0/gems/chef-vault-4.1.0/lib/chef-vault/version.rb:19: warning: previous definition of MINOR was here
/opt/opscode/embedded/lib/ruby/gems/2.7.0/gems/chef-vault-4.1.4/lib/chef-vault/version.rb:19: warning: already initialized constant ChefVault::TINY
/opt/opscode/embedded/lib/ruby/gems/2.7.0/gems/chef-vault-4.1.0/lib/chef-vault/version.rb:19: warning: previous definition of TINY was here
Downloading Users
Downloading organization object for 4thcoffee from https://127.0.0.1/organizations/4thcoffee
Copying /acls/groups/billing-admins.json
Copying /groups/billing-admins.json
Copying /groups/admins.json
Copying /acls/groups/public_key_read_access.json
Copying /groups/public_key_read_access.json
Copying /clients
Copying /containers
Copying /cookbook_artifacts
Copying /cookbooks
Copying /data_bags
Copying /environments
Copying /invitations.json
Copying /members.json
Copying /nodes
Copying /org.json
Copying /policies
Copying /policy_groups
Copying /roles
Copying /acls/organization.json
Copying /acls/clients/4thcoffee-validator.json
Copying /acls/containers/clients.json
Copying /acls/containers/containers.json
Copying /acls/containers/cookbook_artifacts.json
Copying /acls/containers/cookbooks.json
Copying /acls/containers/data.json
Copying /acls/containers/environments.json
Copying /acls/containers/groups.json
Copying /acls/containers/nodes.json
Copying /acls/containers/policies.json
Copying /acls/containers/policy_groups.json
Copying /acls/containers/roles.json
Copying /acls/containers/sandboxes.json
Copying /acls/environments/_default.json
Copying /acls/groups/0000000000005ed51582337f8da81361.json
Copying /acls/groups/admins.json
Copying /acls/groups/clients.json
Copying /acls/groups/users.json
Copying /groups/0000000000005ed51582337f8da81361.json
Copying /groups/admins.json
Copying /groups/clients.json
Copying /groups/users.json
** Finished **

PrajaktaPurohit commented 2 years ago

can be repoened as needed.

chef / chef-server

knife ec backup failures #2907

Describe the problem

Software Version

Replication Case

Stacktrace

404 Not Found

404 Not Found