search

theforeman / foreman_scap_client

GNU General Public License v3.0
2 stars 21 forks source link

Feature request: Downloading content and tailoring files from other sources [incl. basic authentication] #36

Open tberreis opened 3 years ago

tberreis commented 3 years ago

It seems that usually the scap client uses the same settings for uploading reports and for downloading content which is not suitable in some circumstances. After installing Foreman with default settings, the proxy for uploading reports listens on :9090, but downloading content and tailoring files is available via API on :443.

Example Configuration

# Foreman proxy to which reports should be uploaded
:server: 'foreman.local'
:port: 9090

# HTTP proxy server for downloading remote resources
:http_proxy_server:
:http_proxy_port:

# policies
1:
  :profile: 'xccdf_org.ssgproject.content_profile_pci-dss'
  :content_path: '/usr/share/xml/scap/ssg/content/ssg-centos7-ds-1.2.xml'
  :download_path: '/api/compliance/policies/1/content/'

may lead to

[root@local]# foreman_scap_client 1
File /usr/share/xml/scap/ssg/content/ssg-centos7-ds-1.2.xml is missing. Downloading it from proxy.
Download SCAP content xml from: https://foreman.local:9090/api/compliance/policies/1/content/
SCAP content is missing and download failed with error: 404 "Not Found"

It should be possible to define the source via separate settings, e.g.

:content_server: 'public-source.for.scap-content'
:content_port: 443

And if so - the possibility to use basic authentication would be great too.

# Foreman proxy to which reports should be uploaded
:server: 'foreman.local'
:port: 9090

# HTTP content server for downloading remote resources (no proxy here - see below)
:content_server: 'public-source.for.scap-content'
:content_port: 443
:content_user: 'scap-user'
:content_pass: 'MyVerySecretPassword'

# HTTP proxy server for downloading remote resources
:http_proxy_server:
:http_proxy_port:

In the end it should look like

[root@local]# foreman_scap_client 1
File /usr/share/xml/scap/ssg/content/ssg-centos7-ds-1.2.xml is missing. Downloading it from proxy.
Download SCAP content xml from: https://public-source.for.scap-content/api/compliance/policies/1/content/
Basic authentication enabled.
DEBUG: running: oscap xccdf eval --fetch-remote-resources --profile xccdf_org.ssgproject.content_profile_pci-dss  --results-arf /tmp/d20210813-29128-1ocljac/results.xml /usr/share/xml/scap/ssg/content/ssg-centos7-ds-1.2.xml
Downloading: https://www.redhat.com/security/data/oval/com.redhat.rhsa-RHEL7.xml ... ok
DEBUG: running: /usr/bin/env bzip2 /tmp/d20210813-29128-1ocljac/results.xml
Uploading results to https://foreman.local:9090/compliance/arf/1
Report uploaded, report id: 156
xprazak2 commented 3 years ago

Hi, thanks you for opening this. Do you have a reproducer steps for the 404 error? I can think of steps that may lead to what you are describing, though I haven't tested yet:

  1. create a policy and add it to a host
  2. run Puppet/Ansible to propagate configuration to host which will add policy entry to config
  3. delete policy in Foreman
  4. run foreman_scap_client $id where $id belongs to deleted policy

This will cause a configuration discrepancy - the policy no longer exists in Foreman, but host still has an entry in config. Because the policy has not yet been executed, host does not yet have scap content file and tries to download it - which of course fails. Getting the config into a consistent state is the fix in this case.

Is defining a separate source meant to fix the error or is it a separate feature?

tberreis commented 3 years ago

Hi,

I performed a clean install of Foreman 2.5 with Katello as seen below:

foreman-installer --scenario katello \
                  --foreman-initial-organization my_orga \
                  --foreman-initial-location my_location \
                  --foreman-initial-admin-username admin \
                  --foreman-initial-admin-password MySecurePassword \
                  --enable-foreman-plugin-openscap \
                  --enable-foreman-proxy-plugin-openscap \
                  --enable-foreman-plugin-ansible \
                  --enable-foreman-proxy-plugin-ansible \
                  --enable-foreman-plugin-remote-execution \
                  --enable-foreman-proxy-plugin-remote-execution-ssh \
                  --enable-foreman-plugin-azure \
                  --enable-foreman-plugin-statistics \
                  --enable-foreman-plugin-tasks \
                  --enable-foreman-cli-ansible \
                  --enable-foreman-cli-azure \
                  --enable-foreman-cli-katello \
                  --enable-foreman-cli-openscap \
                  --enable-foreman-cli-remote-execution \
                  --enable-foreman-cli-tasks \
                  --foreman-plugin-tasks-automatic-cleanup true \
                  --foreman-plugin-tasks-backup true

The smart proxy is listening on :9090 and uploading arf reports is working as expected. But the scap content is not available on :9090. The smart proxy is not delivering any SCAP content and only the Foreman API itself which is available on :443 can be used to download the policy content and tailoring files. Therefore upload and download target differ.

Not sure, if it's a corner case or some misconfiguration on my side. Because I used the default settings, I'm wondering if I'm the first, who's struggling ...

grafik

grafik

xprazak2 commented 3 years ago

Thank you for the details. Taking a closer look, I noticed the client config has a couple of unusual values.

oscap-config

:download_path: in your config starts with /api/compliance, while mine has just /compliance, which is probably the cause of 404.

Was your config file created manually?

tberreis commented 3 years ago

Unfortunately, that doesn't change anything:

[root ~]# foreman_scap_client 2
File /usr/share/xml/scap/ssg/content/ssg-centos7-ds-1.2.xml is missing. Downloading it from proxy.
Download SCAP content xml from: https://*****.com:9090/compliance/policies/2/content/
SCAP content is missing and download failed with error: 404 "Not Found"
[root ~]# vi /etc/foreman_scap_client/config.yaml
[root ~]# foreman_scap_client 2
File /usr/share/xml/scap/ssg/content/ssg-centos7-ds-1.2.xml is missing. Downloading it from proxy.
Download SCAP content xml from: https://*****.com:9090/api/compliance/policies/2/content/
SCAP content is missing and download failed with error: 404 "Not Found"
[root ~]# curl https://*****.com:9090/compliance/policies/2/content/
No client SSL certificate supplied
[root ~]# curl --key /etc/pki/consumer/key.pem --cert /etc/pki/consumer/cert.pem https://*****.com:9090/compliance/policies/2/content/
Requested url was not found
[root ~]# curl https://*****.com:/api/compliance/policies/2/content/
{
  "error": {"message":"Access denied","details":"Missing one of the required permissions: view_policies","missing_permissions":["view_policies"]}
}
[root ~]# curl -s -u ***:****** https://*****.com:/api/compliance/policies/2/content/
<ds:data-stream-collection xmlns:cat="urn:oasis:names:tc:entity:xmlns:xml:catalog" xmlns:cpe-dict="http://cpe.mitre.org/dictionary/2.0" xmlns:dc="http://purl.org/dc/elements/1.1/" [...]

API is used as a workaround more or less:

grafik

xprazak2 commented 3 years ago

Hi, this is really strange. I just installed 2.5, but could not reproduce. I would check /var/log/foreman/production.log and /var/log/foreman-proxy/proxy.log as you run the client, maybe something useful will show up.

Guigouu commented 1 year ago

Any update ? I have exactly the same behaviour