search

inspec / inspec

InSpec: Auditing and Testing Framework
http://inspec.io
Other
2.85k stars 682 forks source link

inspec compliance upload fails on automate server #1360

Closed poshpaws closed 7 years ago

poshpaws commented 7 years ago

Description

inspec compliance upload should work

InSpec and Platform Version

1.7.1 mac s x Tell us which version of InSpec (inspec version or SHA of a commit) and Operating System distribution you are using

Replication Case

build a compliance profile and upload it.

Possible Solutions

none

Stacktrace

from workstation:

tooticky:compliance gavreid$ inspec compliance upload sus.cis.l1.cent7
Profile is already vendored. Use --overwrite.
I, [2016-12-12T18:54:00.810355 #41596]  INFO -- : Checking profile in sus.cis.l1.cent7
I, [2016-12-12T18:54:00.810527 #41596]  INFO -- : Metadata OK.
I, [2016-12-12T18:54:03.210800 #41596]  INFO -- : Found 163 controls.
W, [2016-12-12T18:54:03.210903 #41596]  WARN -- : Control xccdf_org.cisecurity.benchmarks_rule_1.1.11_Add_nodev_Option_to_Removable_Media_Partitions has no tests defined
W, [2016-12-12T18:54:03.210945 #41596]  WARN -- : Control xccdf_org.cisecurity.benchmarks_rule_1.1.12_Add_noexec_Option_to_Removable_Media_Partitions has no tests defined
W, [2016-12-12T18:54:03.210973 #41596]  WARN -- : Control xccdf_org.cisecurity.benchmarks_rule_1.1.13_Add_nosuid_Option_to_Removable_Media_Partitions has no tests defined
W, [2016-12-12T18:54:03.211010 #41596]  WARN -- : Control xccdf_org.cisecurity.benchmarks_rule_1.2.3_Obtain_Software_Package_Updates_with_yum has no tests defined
W, [2016-12-12T18:54:03.211030 #41596]  WARN -- : Control xccdf_org.cisecurity.benchmarks_rule_1.2.4_Verify_Package_Integrity_Using_RPM has no tests defined
W, [2016-12-12T18:54:03.211052 #41596]  WARN -- : Control xccdf_org.cisecurity.benchmarks_rule_1.7_Use_the_Latest_OS_Release has no tests defined
W, [2016-12-12T18:54:03.211097 #41596]  WARN -- : Control xccdf_org.cisecurity.benchmarks_rule_3.16_Configure_Mail_Transfer_Agent_for_Local-Only_Mode has no tests defined
W, [2016-12-12T18:54:03.211127 #41596]  WARN -- : Control xccdf_org.cisecurity.benchmarks_rule_4.3.1_Deactivate_Wireless_Interfaces has no tests defined
W, [2016-12-12T18:54:03.211151 #41596]  WARN -- : Control xccdf_org.cisecurity.benchmarks_rule_4.4.1.1_Disable_IPv6_Router_Advertisements has no tests defined
W, [2016-12-12T18:54:03.211169 #41596]  WARN -- : Control xccdf_org.cisecurity.benchmarks_rule_4.4.1.2_Disable_IPv6_Redirect_Acceptance has no tests defined
W, [2016-12-12T18:54:03.211186 #41596]  WARN -- : Control xccdf_org.cisecurity.benchmarks_rule_4.4.2_Disable_IPv6 has no tests defined
W, [2016-12-12T18:54:03.211203 #41596]  WARN -- : Control xccdf_org.cisecurity.benchmarks_rule_4.5.1_Install_TCP_Wrappers has no tests defined
W, [2016-12-12T18:54:03.211219 #41596]  WARN -- : Control xccdf_org.cisecurity.benchmarks_rule_4.5.2_Create_etchosts.allow has no tests defined
W, [2016-12-12T18:54:03.211236 #41596]  WARN -- : Control xccdf_org.cisecurity.benchmarks_rule_4.5.4_Create_etchosts.deny has no tests defined
W, [2016-12-12T18:54:03.211254 #41596]  WARN -- : Control xccdf_org.cisecurity.benchmarks_rule_4.6.1_Disable_DCCP has no tests defined
W, [2016-12-12T18:54:03.211270 #41596]  WARN -- : Control xccdf_org.cisecurity.benchmarks_rule_4.6.2_Disable_SCTP has no tests defined
W, [2016-12-12T18:54:03.211286 #41596]  WARN -- : Control xccdf_org.cisecurity.benchmarks_rule_4.6.3_Disable_RDS has no tests defined
W, [2016-12-12T18:54:03.211302 #41596]  WARN -- : Control xccdf_org.cisecurity.benchmarks_rule_4.6.4_Disable_TIPC has no tests defined
W, [2016-12-12T18:54:03.211322 #41596]  WARN -- : Control xccdf_org.cisecurity.benchmarks_rule_5.1.3_Configure_etcrsyslog.conf has no tests defined
W, [2016-12-12T18:54:03.211402 #41596]  WARN -- : Control xccdf_org.cisecurity.benchmarks_rule_5.1.4_Create_and_Set_Permissions_on_rsyslog_Log_Files has no tests defined
W, [2016-12-12T18:54:03.211421 #41596]  WARN -- : Control xccdf_org.cisecurity.benchmarks_rule_5.1.6_Accept_Remote_rsyslog_Messages_Only_on_Designated_Log_Hosts has no tests defined
W, [2016-12-12T18:54:03.211447 #41596]  WARN -- : Control xccdf_org.cisecurity.benchmarks_rule_5.3_Configure_logrotate has no tests defined
W, [2016-12-12T18:54:03.211488 #41596]  WARN -- : Control xccdf_org.cisecurity.benchmarks_rule_6.3.3_Set_Lockout_for_Failed_Password_Attempts has no tests defined
W, [2016-12-12T18:54:03.211509 #41596]  WARN -- : Control xccdf_org.cisecurity.benchmarks_rule_6.4_Restrict_root_Login_to_System_Console has no tests defined
W, [2016-12-12T18:54:03.211536 #41596]  WARN -- : Control xccdf_org.cisecurity.benchmarks_rule_8.3_Set_GNOME_Warning_Banner has no tests defined
W, [2016-12-12T18:54:03.211561 #41596]  WARN -- : Control xccdf_org.cisecurity.benchmarks_rule_9.1.10_Find_World_Writable_Files has no tests defined
W, [2016-12-12T18:54:03.211580 #41596]  WARN -- : Control xccdf_org.cisecurity.benchmarks_rule_9.1.13_Find_SUID_System_Executables has no tests defined
W, [2016-12-12T18:54:03.211597 #41596]  WARN -- : Control xccdf_org.cisecurity.benchmarks_rule_9.1.14_Find_SGID_System_Executables has no tests defined
W, [2016-12-12T18:54:03.211614 #41596]  WARN -- : Control xccdf_org.cisecurity.benchmarks_rule_9.2.1_Ensure_Password_Fields_are_Not_Empty has no tests defined
W, [2016-12-12T18:54:03.211637 #41596]  WARN -- : Control xccdf_org.cisecurity.benchmarks_rule_9.2.8_Check_User_Dot_File_Permissions has no tests defined
W, [2016-12-12T18:54:03.211658 #41596]  WARN -- : Control xccdf_org.cisecurity.benchmarks_rule_9.2.13_Check_User_Home_Directory_Ownership has no tests defined
Profile is valid
Generate temporary profile archive at /var/folders/n3/vjry65_55k188lspycsg7ylc0000gn/T/sus.cis.l1.cent720161212-41596-1t1t3l8.tar.gz
I, [2016-12-12T18:54:03.246669 #41596]  INFO -- : Generate archive /var/folders/n3/vjry65_55k188lspycsg7ylc0000gn/T/sus.cis.l1.cent720161212-41596-1t1t3l8.tar.gz.
I, [2016-12-12T18:54:03.252643 #41596]  INFO -- : Finished archive generation.
Start upload to admin/sus.cis.l1.cent7
Uploading to Chef Automate
Error during profile upload:
<html>
<head><title>404 Not Found</title></head>
<body bgcolor="white">
<center><h1>404 Not Found</h1></center>
<hr><center>nginx</center>
</body>
</html>

server error log:

2016/12/12 16:58:32 [error] 2266#0: *842642 open() "/var/opt/delivery/nginx/console.json" failed (2: No such file or directory), client: 10.8.0.3, server: automate.lab.sus.private, request: "GET /status/console HTTP/1.1", host: "automate.lab.sus.private", referrer: "https://automate.lab.sus.private/viz/"
2016/12/12 17:00:36 [error] 15059#0: *26 open() "/opt/delivery/embedded/nginx/html/favicon.ico" failed (2: No such file or directory), client: 10.8.0.3, server: automate.lab.sus.private, request: "GET /favicon.ico HTTP/1.1", host: "automate.lab.sus.private", referrer: "https://automate.lab.sus.private/status/version"
2016/12/12 17:01:28 [error] 15059#0: *197 open() "/var/opt/delivery/nginx/console.json" failed (2: No such file or directory), client: 10.8.0.3, server: automate.lab.sus.private, request: "GET /status/console HTTP/1.1", host: "automate.lab.sus.private", referrer: "https://automate.lab.sus.private/viz/"
2016/12/12 17:01:30 [error] 15059#0: *198 open() "/var/opt/delivery/nginx/console.json" failed (2: No such file or directory), client: 10.8.0.3, server: automate.lab.sus.private, request: "GET /status/console HTTP/1.1", host: "automate.lab.sus.private", referrer: "https://automate.lab.sus.private/viz/"
2016/12/12 18:53:40 [error] 15059#0: *21696 open() "/opt/delivery/embedded/nginx/html/compliance/profiles/admin" failed (2: No such file or directory), client: 192.168.1.4, server: automate.lab.sus.private, request: "GET /compliance/profiles/admin HTTP/1.1", host: "automate.lab.sus.private"
2016/12/12 18:53:40 [error] 15059#0: *21697 open() "/opt/delivery/embedded/nginx/html/compliance/profiles/admin" failed (2: No such file or directory), client: 192.168.1.4, server: automate.lab.sus.private, request: "POST /compliance/profiles/admin HTTP/1.1", host: "automate.lab.sus.private"
2016/12/12 18:54:05 [error] 15059#0: *21778 open() "/opt/delivery/embedded/nginx/html/compliance/profiles/admin" failed (2: No such file or directory), client: 192.168.1.4, server: automate.lab.sus.private, request: "GET /compliance/profiles/admin HTTP/1.1", host: "automate.lab.sus.private"
2016/12/12 18:54:05 [error] 15059#0: *21779 open() "/opt/delivery/embedded/nginx/html/compliance/profiles/admin" failed (2: No such file or directory), client: 192.168.1.4, server: automate.lab.sus.private, request: "POST /compliance/profiles/admin HTTP/1.1", host: "automate.lab.sus.private"
poshpaws commented 7 years ago

inspec compliance profiles An unexpected error occurred (HTTP 404): Not Found Could not find any profiles

so profile is already vendored is bogus.

chris-rock commented 7 years ago

@poshpaws Right now, Compliance in Automate is shipped with a feature flag. Have you activated it? https://docs.chef.io/install_chef_automate.html#compliance

chris-rock commented 7 years ago

so profile is already vendored is bogus.

Not sure what you mean by that?

poshpaws commented 7 years ago

yes its enabled

/opt/delivery/embedded/nginx/html/compliance is missing

Now the bogus statement - the inspec compliance upload , complains that this profile already exists yet the inspec compliance profiles shows nothing , hence bogus warning on upload

chris-rock commented 7 years ago

Ah I see. InSpec vendors all dependencies of a profile. We should change the terminology to Profile dependencies are already vendored More information about that is available here: http://lollyrock.com/articles/chef-compliance-meta-profiles

chris-rock commented 7 years ago

Which version of Automate are you using? This feature is available since 0.6.6 https://discourse.chef.io/t/chef-automate-release-0-6-6/9899

poshpaws commented 7 years ago

Sorry for delay , I wanted to rebuild my env from scratch - but same results

delivery 0.6.64

Component Installed Version Version GUID Overridden From

appbundler master git:6582b68884453b5ca5c962458a0c08046c719558
bundler 0.6.64
bzip2 1.0.6 md5:00b516f4704d4a7cb50a1d97e6e8e15b
cacerts 2016-04-20 md5:782dcde8f5d53b1b9e888fdf113c42b9
chef v12.15.19 git:a8278f840bea3c669b052a8fe4a3496141fae5e3 master
cmake 3.4.3 md5:4cb3ff35b2472aae70f542116d616e63
common 0.0.1
compliance-profiles ap-sr/asset-store git:e977169c0a2243446a786f451043824d3807f325
config_guess master git:084a40766085aaa54c1c188ff586815726007522
cpanminus 1.7004 md5:02fe90392f33a12979e188ea110dae67
curator 4.1.0 md5:69cff1eb1206cf9da5c99759106da3f9
curl 7.51.0 sha256:65b5216a6fbfa72f547eb7706ca5902d7400db9868269017a8888aa91d87977c
delivery 0.0.1
delivery-cookbooks 0.6.64
delivery-ctl 0.6.64
delivery-git-hooks 0.6.64
delivery-schema 0.6.64
delivery-scripts 0.6.64
delivery-server 0.6.64
delivery-vendor-cookbooks 0.6.64
delivery-web 0.6.64
elasticsearch 2.4.1 sha1:6a6acfc7bf7b4354dc6136daea54db1c844d632f
erlang 18.2 md5:b336d2a8ccfbe60266f71d102e99f7ed 18.3
expat 2.1.0 md5:dd7dab7a5fea97d2a6a43f511449b7cd
git 2.10.2 md5:45e8b30a9e7c1b734128cc0fc6663619
go 1.7.3 sha256:508028aac0654e993564b6e2014bf2d4a9751e3b286661b0b0040046cf18028e
godep master
insights 0.0.1
insights-batch-jobs 0.6.64
insights-elasticsearch 0.6.64
insights-logstash 0.6.64
insights-scripts 0.6.64
insights-web 0.6.64
inspec master git:7990ad655a7f9aa511035098f4c4a81d85cbc547
inspec-msccm master git:49e60b9f1203ddd6c98c86ede2bf0e613b6b633a
inspec-scap master git:8fe11b75a8c8c84d6faf9ea8307661c076814ccf
kibana 4.6.1 sha1:b0191e833c83a348bbf6b09628756e898cceb73b
libarchive 3.2.1 md5:afa257047d1941a565216edbf0171e72
libedit 20130712-3.1 md5:0891336c697362727a1fa7e60c5cb96c 20120601-3.0
libffi 3.2.1 md5:83b89587607e3eb65c70d361f13bab43
libiconv 1.14 md5:e34509b1623cec449dfeb73d7ce9c6c6
libidn 1.32 md5:4dd8356ba577287ea7076bfa1554b534
libintl-perl 1.23 md5:2e79dc842af1c9efc14fbe6664dc89bf
liblzma 5.2.2 md5:7cf6a8544a7dae8e8106fdf7addfa28c
libossp-uuid 1.6.2 md5:5db0d43a9022a6ebbbc25337ae28942f
libtool 2.4.2 md5:d2f3b7d4627e69e13514a40e72a24d50 2.4
libxml2 2.9.4 md5:ae249165c173b1ff386ee8ad676815f5
libxslt 1.1.29 md5:a129d3c44c022de3b9dcf6d6f288d72e
libyaml 0.1.6 md5:5fe00cda18ca5daeb43762b80c38e06e
logrotate 3.9.2 md5:584bca013dcceeb23b06b27d6d0342fb
logstash 2.3.3 sha1:bce7c753dd19848e29253f706f834d43a74152f8
lsyncd 2.1.6 md5:e0618ac837a25aa027fa1cbb67815c23
lua 5.2.4 sha256:b9e2e4aad6789b3b63a056d442f7b39f0ecfca3ae0f1fc0ae4e9614401b69f4b 5.3.3
makedepend 1.0.5 md5:efb2d7c7e22840947863efaedc175747
mlsa 1.0.1 path:/home/jenkins/workspace/delivery-build/architecture/x86_64/platform/el-7/project/delivery/role/builder/omnibus/files
ncurses 5.9 md5:8cb9c412e5f2d96bc6f459aa8c6282a1
nodejs-binary 5.6.0 md5:17cb2fc57a0cf46bcc021060b25edf48
nokogiri 0.6.64
ohai master git:b8427163ad659179af16171e329d77c5344d537a
omnibus-ctl 0.3.6 git:c514d1d4ecb24e30fdbd310b2dd038b2192b4fa7
openresty 1.11.2.1 md5:f26d152f40c5263b383a5b7c826a6c7e
openssl 1.0.2j sha256:e7aff292be21c259c6af26469c7a9b3ba26e9abaaffd325e3dccc9785256c431
pcre 8.38 md5:8a353fe1450216b6655dfcf3561716d9
perl 5.18.1 md5:304cb5bd18e48c44edd6053337d3386d
perl_pg_driver 3.3.0 md5:547de1382a47d66872912fe64282ff55
pkg-config-lite 0.28-1 md5:61f05feb6bab0a6bbfab4b6e3b2f44b6
popt 1.16 md5:3743beefa3dd6247a73f8f7a32c14c33
postgresql 9.3.14 sha256:5c4322f1c42ba1ff4b28383069c56663b46160bb08e85d41fa2ab9a5009d039d 9.2.10
preparation 1.0.0
python 2.7.9 md5:5eebcaa0030dc4061156d3429657fb83
rabbitmq 3.6.3 md5:37c79fe55d7cc93baef3d9ccbda6b15b
rb-readline master git:60cd64a7d2a44daaacf2f82a111e9ba29f632ec0
reaper 0.6.64
rebar 2.5.1 git:e9f62c45807ce2db39e0606c4d97cd071416bd64 93621d0d0c98035f79790ffd24beac94581b0758
remove-build-deps 0.0.1
remove-cmake 3.4
remove-libidn 1.32
remove-nodejs 5.6.0
rsync 2.6.9 md5:996d8d8831dbca17910094e56dcb5942 3.1.1
ruby 2.2.5 md5:bd8e349d4fb2c75d90817649674f94be 2.1.8
rubygems 0.6.64
runit 2.1.2 md5:6c985fbfe3a34608eb3c53dc719172c4
server-jre 8u91 md5:c8aefc3b97328d6f5197315fa6507bbf
setuptools 20.0 md5:fb22b2474ca037e0b08f3c3b293e02e6 0.7.7
sqitch 0.973 md5:0994e9f906a7a4a2e97049c8dbaef584
strip-objects 0.0.1
tools 0.0.1
util-macros 1.18.0 md5:fd0ba21b3179703c071bbb4c3e5fb0f4
xproto 7.0.25 md5:a47db46cb117805bd6947aa5928a7436
zlib 1.2.8 md5:44d667c142d7cda120332623eab69f40
zstd 1.0.0 md5:ca9a01cd81265ac235acdf611a25122e

samsonkolge commented 7 years ago

Team,

I am struggling with uploading a profile on Chef Automate.

$ curl -XV GET "https://XXXX.net/compliance/profiles/admin" -H "chef-delivery-enterprise: hpe" -H "chef-delivery-user: admin" -H "chef-delivery-token: w30d...." -k
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   613  100   613    0     0    769      0 --:--:-- --:--:-- --:--:--  1187<HTML><HEAD>
<TITLE>Network Error</TITLE>
</HEAD>
<BODY>
<FONT face="Helvetica">
<big><strong></strong></big><BR>
</FONT>
<blockquote>
<TABLE border=0 cellPadding=1 width="80%">
<TR><TD>
<FONT face="Helvetica">
<big>Network Error (dns_unresolved_hostname)</big>
<BR>
<BR>
</FONT>
</TD></TR>
<TR><TD>
<FONT face="Helvetica">
Your requested host "get" could not be resolved by DNS.
</FONT>
</TD></TR>
<TR><TD>
<FONT face="Helvetica">

</FONT>
</TD></TR>
<TR><TD>
<FONT face="Helvetica" SIZE=2>
<BR>
For assistance, contact your network support team.
</FONT>
</TD></TR>
</TABLE>
</blockquote>
</FONT>
</BODY></HTML>

100    18  100    18    0     0     11      0  0:00:01  0:00:01 --:--:--    17404 page not found

I am able to ping the server from my workstation. Further below is the automate status as well

[root@xxxx ~]# automate-ctl status
run: compliance_profiles: (pid 102811) 79191s; run: log: (pid 102860) 79189s
run: delivery: (pid 56594) 334733s; run: log: (pid 56635) 334732s
run: elasticsearch: (pid 56140) 334765s; run: log: (pid 56233) 334754s
run: kibana: (pid 57287) 334714s; run: log: (pid 57301) 334714s
run: logstash: (pid 26167) 1519s; run: log: (pid 57202) 334715s
run: nginx: (pid 24515) 1561s; run: log: (pid 56566) 334733s
run: postgresql: (pid 50305) 335057s; run: log: (pid 50315) 335057s
run: rabbitmq: (pid 57087) 334715s; run: log: (pid 56639) 334732s
[root@xxxx ~]#

Am I missing something? Why is the compliance link not accessible?

adamleff commented 7 years ago

Hi folks, including recent commenter @samsonkolge!

Since this is a feature/issue dealing with one of Chef's commercial products, even if the issue stems from a problem in the open-source tool, I'd like to ask you to please submit a support request to Chef Support. We want to assist you in resolving your issue, but to ensure the issue gets the appropriate attention and nothing slips through the cracks, working with our support team for issues interacting with our commercial products is the right way to go.

I will be closing this issue now, but I'm happy to reopen it or have additional discussion as needed. Thank you for using InSpec!