HDF and inspec 4.x data standards and support for extended properties for the HDF, Tools and Heimdall Apps

[aaronlippold]

I know I am repeating myself in various issues on this but I hope this collects a lot of these ideas.

We should also write up a 'HDF Profile and Control' Stype guide that would help lay out a lot of these 'standards' that we have established.

I am sure many of these will break out into smaller sub-issues on the vaious projects.

I am also very sure that all this stuff will have to go into the guide referenced above.

• [ ] That we update the parsing to ensure we always use sub-descriptions on text block types rather than tags:
• fix
• check
• desc

The below are used in many overlays and tailoring profiles

• /vulnerability/

• vulnerability_discussion ( in xccdf, csv and xls for example )

• /justification/

• caveat

• etc.

• [ ] That we allow for allow for three new types: desc, justification, caveat, discussion a. That caveat and or justification are appended to the 'Finding Details' in all the converters and Heidmall Applications b. We also allow for /*caveat*/ and /*justification*/ - such that myorg-/_caveat is discovered. c. that discussion or /*discussion*/ be appended to the bottom of the general description - such that vulnerability_discussion would be discovered.

• [ ] That we support both text based impacts and numeric based impacts in our parsing and conversations to allow for better user interface ( in xccdf, csv, xccdf etc. )

• [x] That we update inspec_tools and heimdall_tools to use the new sub-sections

• [ ] That the sub-descriptions are grouped and created together in the control logically:

• desc [ req ]

• title [ req ]

• impact [ req ]

• desc vuln (opt)

• vuln (opt)

• desc caveat (opt)

• desc caveat (opt)

{ tags, nist: [reg || UM-1] => if STIG || CIS cci: [req || N/A] => if STIG || CIS cis_cdc [ req || N/A ] => IF CIS }

• desc check [ req ]

• desc fix [ req ]

• ref * (opt)

describe blocks [ req ]

...
- [x] That `CAT I / CAT II / CAT III` can be processed and their derivities and be replaced by CVSS 3.0 standards in all our tools forward and backward - such that if someone creats a CSV or XLS they can ask to use the CVSS, CAT[I-IV] or CAT[1-4] severity style.

- [ ] That our tools do not create code that uses `"` where `'` are the correct style 

- [ ] Ensure the output of our converters formats with a standard of 2-space utf-8 conmpliant chars. Automate and other windows and ruby unzip some time have issues with the char encoding or odd things. This is an old issue but I think they are just plian using an old library or something.

- [ ] Remove the 'Rev_*' from the `nist tag` array as the actual mapping for the control and its nist revision is in the CCI database, and we have the CCI. This also confuses the end users as they think when the 800-53 mapping changes we have rework the profile and silly things like that. Less is more.

- [ ] Convert all our XLS mapping systems into proper ruby data structures that the tools and apps. Allow for us to be able to use the XLS as a source but convert that into our stable ruby defined structures so that in the binary builds we are alows useing pure ruby objects and data structures which can can loaded as static constancts so we only have to load them one etc.

As a note, once we have Compliance Mapper working - we can just interact and gen these data there.

- [ ] Ensure the output of our converts creates well formatted aligned starting point - I think rubocop can be fanagled to support this. See the Windows 10 profile for an example - given I had to review every control I just did this by hand on the 243 controls ... I would prefer never to do that again.

```ruby
# encoding: utf-8

control 'V-63319' do
  title "Domain-joined systems must use Windows 10 Enterprise Edition 64-bit
        version."
  desc  "Features such as Credential Guard use virtualization based security to
        protect information that could be used in credential theft attacks if
        compromised. There are a number of system requirements that must be met in
        order for Credential Guard to be configured and enabled properly.
        Virtualization based security and Credential Guard are only available with
        Windows 10 Enterprise 64-bit version."
  impact 0.5
  tag severity: 'medium'
  tag gtitle: 'WN10-00-000005'
  tag gid: 'V-63319'
  tag rid: 'SV-77809r3_rule'
  tag stig_id: 'WN10-00-000005'
  tag fix_id: 'F-69237r2_fix'
  tag cci: ['CCI-000366']
  tag nist: ['CM-6 b']
  tag false_negatives: nil
  tag false_positives: nil
  tag documentable: false
  tag mitigations: nil
  tag severity_override_guidance: false
  tag potential_impacts: nil
  tag third_party_tools: nil
  tag mitigation_controls: nil
  tag responsibility: nil
  tag ia_controls: nil

  desc "check", "Verify domain-joined systems are using Windows 10 Enterprise
        Edition 64-bit version.

        For standalone systems, this is NA.

        Open \"Settings\".

        Select \"System\", then \"About\".

        If \"Edition\" is not \"Windows 10 Enterprise\", this is a finding.

        If \"System type\" is not \"64-bit operating system…\", this is a finding."

  desc "fix", 'Use Windows 10 Enterprise 64-bit version for domain-joined systems.'

  describe os.arch do
    it { should eq 'x86_64' }
  end

  describe os.name do
    it { should eq 'windows_10_enterprise' }
  end
end

• [ ] Given the above, can we create a .rubocop standard file that would help enfore a lot of this

[aaronlippold]

This issue has evolved as we have evolved the HDF standard. Please consolidate this ticket generally:

[aaronlippold]

many of these items are either done or in progress ... if we want ... we should break these into issues and close this one.

[aaronlippold]

https://github.com/mitre/inspec_tools/issues/171