search

oscal-compass / compliance-trestle

An opinionated tooling platform for managing compliance as code, using continuous integration and NIST's OSCAL standard.
https://oscal-compass.github.io/compliance-trestle
Apache License 2.0
165 stars 61 forks source link

Trestle version 2.0.0 released #1315

Closed fsuits closed 5 months ago

fsuits commented 1 year ago

The new version of trestle has been released and is bumped to 2.0.0 because of breaking changes caused by handling SSP authoring in a very different way, based on component-definitions and rules.

We welcome feedback on the new version, and will be updating associated demos shortly.

degenaro commented 1 year ago

degenaro:~$ python -m venv venv.trestle-test degenaro:~$ source venv.trestle-test/bin/activate (venv.trestle-test) degenaro:~$ pip install compliance-trestle ... (venv.trestle-test) degenaro:~$ trestle version Trestle version v2.0.0 based on OSCAL version 1.0.2

On Mon, Mar 6, 2023 at 10:42 AM ride808 @.***> wrote:

Just tried installing 2.0.0 and it appears it installed 1.1.0 of the trestle CLI. What am I missing?

@.*** /]$ which trestle /usr/bin/which: no trestle in (/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/home/ec2-user/.local/bin:/home/ec2-user/bin)

@. /]$ sudo pip3 install compliance-trestle Looking in indexes: @./repository/pypi-proxy/simple, https://gitlab.global.lmco.com/api/v4/projects/2633/packages/pypi/simple Collecting compliance-trestle Using cached compliance_trestle-2.0.0-py2.py3-none-any.whl (392 kB) Requirement already satisfied: orjson in /usr/local/lib/python3.9/site-packages (from compliance-trestle) (3.8.7) Requirement already satisfied: python-frontmatter in /usr/local/lib/python3.9/site-packages (from compliance-trestle) (1.0.0) Requirement already satisfied: cmarkgfm==0.6.* in /usr/local/lib/python3.9/site-packages (from compliance-trestle) (0.6.0) Requirement already satisfied: attrs in /usr/local/lib/python3.9/site-packages (from compliance-trestle) (21.4.0) Requirement already satisfied: ilcli in /usr/local/lib/python3.9/site-packages (from compliance-trestle) (0.3.2) Requirement already satisfied: cryptography in /usr/local/lib/python3.9/site-packages (from compliance-trestle) (36.0.2) Requirement already satisfied: pydantic[email]>=1.8.2 in /usr/local/lib/python3.9/site-packages (from compliance-trestle) (1.10.5) . . . Requirement already satisfied: charset-normalizer~=2.0.0 in /usr/local/lib/python3.9/site-packages (from requests>=2.25->prance<1.0,>=0.18.2->datamodel-code-generator[http]>=0.11.14->compliance-trestle) (2.0.12) Installing collected packages: compliance-trestle Successfully installed compliance-trestle-2.0.0

@.*** /]$ which trestle /usr/local/bin/trestle

@.*** /]$ trestle version Trestle version v1.1.0 based on OSCAL version 1.0.2

— Reply to this email directly, view it on GitHub https://github.com/IBM/compliance-trestle/issues/1315#issuecomment-1456372118, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAD66XN3KZX7S4KGMBBSKBLW2YAV3ANCNFSM6AAAAAAVMTROSE . You are receiving this because you are subscribed to this thread.Message ID: @.***>

ride808 commented 1 year ago

Hey all. I've been having some issues attempting to assemble an SSP with my components. I can generate ssp markdown docs with no issue and they appear to be correct with rules, implementation prose being populated, etc. But I'm not sure why I'm getting this error when I attempt to re-assemble the markdown to my existing SSP. Seeing this comment in the code I'm wondering what must exist in the SSP prior to be reassembling? I already got past an error where a component titled "This System" had to exists. Do I actually have to have my components in the SSP in more complete form (hoping not as that would seem to defeat the purpose of assembling an SSP from components)?

[ec2-user@ip-166-28-30-97 trestle]$ trestle author ssp-generate --profile NIST_SP-800-53_rev5 --compdefs "Terraform,VirtualMachine" --output /tmp/myssp
....

[ec2-user@ip-166-28-30-97 trestle]$ trestle author ssp-assemble --markdown /tmp/myssp  --output SSP --verbose
trestle.core.profile_resolver:146 DEBUG: get resolved profile catalog for trestle://profiles/NIST_SP-800-53_rev5/profile.json via generated Import.
trestle.core.profile_resolver:96 DEBUG: get resolved profile catalog and inherited props for trestle://profiles/NIST_SP-800-53_rev5/profile.json via generated Import.
trestle.core.resolver._import:99 DEBUG: import href is trestle://profiles/NIST_SP-800-53_rev5/profile.json
trestle.core.profile_resolver:112 DEBUG: launch pipeline
trestle.core.resolver._import:103 DEBUG: import entering process with href trestle://profiles/NIST_SP-800-53_rev5/profile.json
trestle.core.remote.cache:58 DEBUG: Initializing FetcherBase
trestle.core.parser:57 DEBUG: Loading class "Profile" from "trestle.oscal.profile"
trestle.core.resolver._import:124 DEBUG: import pipelines for sub_imports of profile trestle://profiles/NIST_SP-800-53_rev5/profile.json with title NIST Special Publication 800-53 Revision 5 MODERATE IMPACT BASELINE
trestle.core.resolver._import:99 DEBUG: import href is ../models/catalogs/NIST_SP-800-53_rev5_catalog.json
trestle.core.resolver._import:138 DEBUG: sub_import add pipeline for sub href ../models/catalogs/NIST_SP-800-53_rev5_catalog.json of main href trestle://profiles/NIST_SP-800-53_rev5/profile.json
trestle.core.resolver.merge:62 DEBUG: merge filter initialize
trestle.core.resolver.modify:59 DEBUG: modify initialize filter with profile NIST Special Publication 800-53 Revision 5 MODERATE IMPACT BASELINE
trestle.core.resolver.merge:210 DEBUG: merge entering process with 1 pipelines
trestle.core.resolver._import:103 DEBUG: import entering process with href ../models/catalogs/NIST_SP-800-53_rev5_catalog.json
trestle.core.remote.cache:58 DEBUG: Initializing FetcherBase
trestle.core.parser:57 DEBUG: Loading class "Catalog" from "trestle.oscal.catalog"
trestle.core.resolver._import:110 DEBUG: DIRECT YIELD in import of catalog Electronic Version of NIST SP 800-53 Rev 5 Controls and SP 800-53A Rev 5 Assessment Procedures
trestle.core.resolver.prune:198 DEBUG: prune yielding catalog Electronic Version of NIST SP 800-53 Rev 5 Controls and SP 800-53A Rev 5 Assessment Procedures with import ../models/catalogs/NIST_SP-800-53_rev5_catalog.json
trestle.core.resolver.merge:186 DEBUG: Profile has merge but no combine so defaulting to combine/merge.
trestle.core.resolver.modify:351 DEBUG: modify process with catalog Electronic Version of NIST SP 800-53 Rev 5 Controls and SP 800-53A Rev 5 Assessment Procedures using profile NIST Special Publication 800-53 Revision 5 MODERATE IMPACT BASELINE
trestle.core.resolver.modify:269 DEBUG: modify specify catalog Electronic Version of NIST SP 800-53 Rev 5 Controls and SP 800-53A Rev 5 Assessment Procedures for profile NIST Special Publication 800-53 Revision 5 MODERATE IMPACT BASELINE
trestle.common.model_utils:266 DEBUG: singular model type <class 'trestle.oscal.ssp.SystemSecurityPlan'> model alias system-security-plan
trestle.common.model_utils:280 DEBUG: not collection field type, malias: system-security-plan
trestle.common.model_utils:293 DEBUG: aliases to be stripped: set()
trestle.core.base_model:297 DEBUG: oscal_read content type FileContentType.JSON and alias system-security-plan from /framework/oscal/trestle/system-security-plans/SSP/system-security-plan.json
trestle.common.model_utils:266 DEBUG: singular model type <class 'trestle.oscal.ssp.SystemSecurityPlan'> model alias system-security-plan
trestle.common.model_utils:280 DEBUG: not collection field type, malias: system-security-plan
trestle.common.model_utils:293 DEBUG: aliases to be stripped: set()
trestle.core.base_model:297 DEBUG: oscal_read content type FileContentType.JSON and alias system-security-plan from /framework/oscal/trestle/system-security-plans/SSP/system-security-plan.json
trestle.core.commands.author.ssp:96 ERROR: Error while assembling SSP: 'VirtualMachine'
Traceback (most recent call last):
  File "/home/ec2-user/.local/lib/python3.9/site-packages/trestle/core/commands/author/ssp.py", line 489, in _run
    CatalogReader.read_ssp_md_content(md_path, ssp, comp_dict, part_id_map_by_label, context)
  File "/home/ec2-user/.local/lib/python3.9/site-packages/trestle/core/catalog/catalog_reader.py", line 364, in read_ssp_md_content
    ssp, control_id, comp_dict[comp_name], comp_info_dict, part_id_map_by_label
KeyError: 'VirtualMachine'

My Test data for reference:

Here are my two components:

{
  "component-definition": {
    "uuid": "573ddad0-af9c-4eab-b7e9-3a8ee96d456c",
    "metadata": {
      "title": "Terraform",
      "last-modified": "2023-02-21T21:41:23+00:00",
      "version": "1.0.0",
      "oscal-version": "1.0.2"
    },
    "components": [
      {
        "uuid": "cf90a0e2-b946-4d78-9754-6c7f26bc5f4f",
        "type": "software",
        "title": "Terraform",
        "description": "Terraform Component",
        "props": [
          {
            "name": "Rule_Id",
            "ns": "http://terraform",
            "value": "xccdf_terraform_rule_1000001",
            "class": "Rule_Id",
            "remarks": "rule_set_0"
          },
          {
            "name": "Rule_Description",
            "ns": "http://terraform",
            "value": "Terraform produces and inventory of deployed system infrastructure",
            "remarks": "rule_set_0"
          }
        ],
        "control-implementations": [
          {
            "uuid": "d9e40a35-142a-4581-8ad9-ad2073756321",
            "source": "trestle://profiles/NIST_SP-800-53_rev5/profile.json",
            "description": "Profile from catalog NIST Special Publication 800-53 Revision 5",
            "props": [
              {
                "name": "Rule_Id",
                "ns": "http://terraform",
                "value": "xccdf_terraform_rule_1000002",
                "class": "Rule_Id",
                "remarks": "rule_set_1"
              },
              {
                "name": "Rule_Description",
                "ns": "http://terraform",
                "value": "A Terraform inventory.ini exists as an artifact within deployment pipelines",
                "remarks": "rule_set_1"
              },
              {
                "name": "Rule_Id",
                "ns": "http://terraform",
                "value": "xccdf_terraform_rule_1000003",
                "class": "Rule_Id",
                "remarks": "rule_set_2"
              },
              {
                "name": "Rule_Description",
                "ns": "http://terraform",
                "value": "Inventory is updated within deployment pipeline",
                "remarks": "rule_set_2"
              }
            ],
            "implemented-requirements": [
              {
                "uuid": "c2acff33-ba60-4cba-9dec-38b9ca0354ba",
                "control-id": "cm-8",
                "description": "Implementation Prose: Terraform produces an inventory of infrastructure components",
                "props": [
                  {
                    "name": "Rule_Id",
                    "value": "xccdf_terraform_rule_1000001"
                  },
                  {
                    "name": "implementation-status",
                    "value": "implemented"
                  }
                ],
                "statements": [
                  {
                    "statement-id": "cm-8_smt.a",
                    "uuid": "2652b814-2a6b-4b6d-a0ae-8bc7a0072201",
                    "description": "Implementation Prose: A Terraform inventory.ini exists as an artifact within deployment pipelines",
                    "props": [
                      {
                        "name": "Rule_Id",
                        "value": "xccdf_terraform_rule_1000002"
                      },
                      {
                        "name": "implementation-status",
                        "value": "implemented"
                      }
                    ]
                  },
                  {
                    "statement-id": "cm-8_smt.b",
                    "uuid": "ae829273-3eab-4c2b-854e-ccb9f546a91a",
                    "description": "Implementation Prose: Inventory is updated within deployment pipeline",
                    "props": [
                      {
                        "name": "Rule_Id",
                        "value": "xccdf_terraform_rule_1000003"
                      },
                      {
                        "name": "implementation-status",
                        "value": "implemented"
                      }
                    ]
                  }
                ]
              }
            ]
          }
        ]
      }
    ]
  }
}
{
  "component-definition": {
    "uuid": "96ebfd55-758c-40f1-9939-6b4911ce3bc9",
    "metadata": {
      "title": "VirtualMachine",
      "last-modified": "2023-02-21T21:42:02+00:00",
      "version": "1.0.0",
      "oscal-version": "1.0.2"
    },
    "components": [
      {
        "uuid": "a2487289-de3a-4bcc-a45f-7d98e96fdfc3",
        "type": "software",
        "title": "VirtualMachine",
        "description": "",
        "props": [
              {
                "name": "Rule_Id",
                "value": "xccdf_vm_rule_1000001",
                "remarks": "rule_set_0"
              },
              {
                "name": "Rule_Description",
                "value": "The environment contains a deployed Virtual Machine instance that serves as a Bastion Server.  It is configured to allow remote connections from trusted hosts via a Cloud Security Group/Firewall",
                "remarks": "rule_set_0"
              },
              {
                "name": "Rule_Id",
                "value": "xccdf_vm_rule_1000002",
                "remarks": "rule_set_1"
              },
              {
                "name": "Rule_Description",
                "value": "All Virtual Machines have an associated Security Group/Firewall resource that can be used to disable remote access to the system.",
                "remarks": "rule_set_1"
              },
              {
                "name": "Rule_Id",
                "value": "xccdf_vm_rule_1000003",
                "remarks": "rule_set_2"
              },
              {
                "name": "Rule_Description",
                "value": "All Virtual Machines have an associated Security Group/Firewall resource that limits external network traffic to the system and the security group does not allow ingress on the CIDR range 0.0.0.0/32",
                "remarks": "rule_set_2"
              }
        ],
        "control-implementations": [
          {
            "uuid": "b0402bde-9b8c-4afd-a8ac-d17290128c70",
            "source": "trestle://profiles/NIST_SP-800-53_rev5/profile.json",
            "description": "Profile from catalog NIST Special Publication 800-53 Revision 5",
            "implemented-requirements": [
              {
                "uuid": "5edb26b5-c81b-4995-b8fe-b855831d37dc",
                "control-id": "ac-17.3",
                "description": "Implementation Prose: The environment contains a deployed Virtual Machine instance that serves as a Bastion Server.  It is configured to allow remote connections from trusted hosts via a Cloud Security Group/Firewall ",
                "props": [
                  {
                    "name": "Rule_Id",
                    "value": "xccdf_vm_rule_1000001"
                  },
                  {
                    "name": "implementation-status",
                    "value": "implemented"
                  }
                ]
              },
              {
                "uuid": "33ef6b41-4210-45d6-bc12-0d8c3fa1eebe",
                "control-id": "ac-17.9",
                "description": "Implementation Prose: All Virtual Machines have an associated Security Group/Firewall resource that can be used to disable remote access to the system. ",
                "props": [
                  {
                    "name": "Rule_Id",
                    "value": "xccdf_vm_rule_1000002"
                  },
                  {
                    "name": "implementation-status",
                    "value": "implemented"
                  }
                ]
              },
              {
                "uuid": "57c7ea15-a684-4970-b6d7-a1d58315193f",
                "control-id": "sc-7.3",
                "description": "All Virtual Machines have an associated Security Group/Firewall resource that limits external network traffic to the system and the security group does not allow ingress on the CIDR range 0.0.0.0/32",
                "props": [
                  {
                    "name": "Rule_Id",
                    "value": "xccdf_vm_rule_1000003"
                  },
                  {
                    "name": "implementation-status",
                    "value": "implemented"
                  }
                ]
              }
            ]
          }
        ]
      }
    ]
  }
}

Here is my SSP that I'm assembling from markdown against:

{
  "system-security-plan": {
    "uuid": "df053186-52e3-48cd-bdda-ed52216aa064",
    "metadata": {
      "title": "System Security Plan",
      "last-modified": "2023-02-28T17:48:26.476060+11:00",
      "version": "0.0.1",
      "oscal-version": "1.0.0",
      "parties": [
        {
          "uuid": "3b2a5599-cc37-403f-ae36-5708fa804b27",
          "type": "organization",
          "name": "My Company"
        }
      ]
    },
    "import-profile": {
      "href": "trestle://profiles/NIST_SP-800-53_rev5/profile.json"
    },
    "system-characteristics": {
      "system-ids": [
        {
          "id": "paas_system"
        }
      ],
      "system-name": "PaaS System",
      "description": "This system provides Platform as a Service capabilities to be used in conjunction with Cloud Service Providers",
      "security-sensitivity-level": "moderate",
      "system-information": {
        "information-types": [
          {
            "uuid": "7d28ac6e-5970-4f4c-a508-5a3715f0f02b",
            "title": "IT Infrastructure Maintenance",
            "categorizations": [
              {
                "system": "https://doi.org/10.6028/NIST.SP.800-60v2r1",
                "information-type-ids": [
                  "C.3.5.4"
                ]
              }
            ],
            "description": "This sytem involes the planning,design, implementation, and maintenance of IT infrastructure to effectively support automated needs.",
            "confidentiality-impact": {
              "base": "fips-199-moderate"
            },
            "integrity-impact": {
              "base": "fips-199-moderate"
            },
            "availability-impact": {
              "base": "fips-199-moderate"
            }
          }
        ]
      },
      "security-impact-level": {
        "security-objective-confidentiality": "fips-199-moderate",
        "security-objective-integrity": "fips-199-moderate",
        "security-objective-availability": "fips-199-moderate"
      },
      "status": {
        "state": "operational"
      },
      "authorization-boundary": {
        "description": "The virtualized components captured in the system inventory and components implemented within the PaaS"
      }
    },
    "system-implementation": {
      "users": [
         {
          "uuid": "22222222-0000-4000-9000-200000000001",
          "role-ids": [
            "admin"
          ],
          "authorized-privileges": [
            {
              "title": "Administrator",
              "functions-performed": [
                "Manages the components within the PaaS."
              ]
            }
          ]
        }
      ],
      "components": [
        {
          "uuid": "80511208-2643-4d2a-bef4-d593ba86b73f",
          "type": "this-system",
          "title": "This System",
          "description": "The system is described by this SSP.",
          "status": {
            "state": "operational"
          }
        },
        {
          "uuid": "cf90a0e2-b946-4d78-9754-6c7f26bc5f4f",
          "type": "Software",
          "title": "Terraform",
          "description": "Terraform Component",
          "status": {
            "state": "operational"
          }
        },
        {
          "uuid": "a2487289-de3a-4bcc-a45f-7d98e96fdfc3",
          "type": "Software",
          "title": "VirtualMachine",
          "description": "",
          "status": {
            "state": "operational"
          }
        }
      ]
    },
    "control-implementation": {
      "description": "Control implementations for the system.",
      "implemented-requirements": []
    },
    "back-matter": {
      "resources": [
        {
          "uuid": "22222222-0000-4000-9999-000000000001",
          "title": "DoD Enterprise DevSecOps Reference Design",
          "description": "Reference design to provide clear guidance on how to build a secure and effective Software Factory based on Kubernetes",
          "rlinks": [
            {
              "href": "https://dodcio.defense.gov/Portals/0/Documents/Library/DevSecOpsReferenceDesign.pdf"
            }
          ]
        }
      ]
    }
  }
}

As another data point - If I do the ssp-assemble but with an --output to a new non-existent SSP I get the same KeyError. 

[ec2-user@ip-166-28-30-97 trestle]$ trestle author ssp-assemble --markdown /tmp/myssp  --output newSSP --verbose
trestle.core.profile_resolver:146 DEBUG: get resolved profile catalog for trestle://profiles/NIST_SP-800-53_rev5/profile.json via generated Import.
trestle.core.profile_resolver:96 DEBUG: get resolved profile catalog and inherited props for trestle://profiles/NIST_SP-800-53_rev5/profile.json via generated Import.
trestle.core.resolver._import:99 DEBUG: import href is trestle://profiles/NIST_SP-800-53_rev5/profile.json
trestle.core.profile_resolver:112 DEBUG: launch pipeline
trestle.core.resolver._import:103 DEBUG: import entering process with href trestle://profiles/NIST_SP-800-53_rev5/profile.json
trestle.core.remote.cache:58 DEBUG: Initializing FetcherBase
trestle.core.parser:57 DEBUG: Loading class "Profile" from "trestle.oscal.profile"
trestle.core.resolver._import:124 DEBUG: import pipelines for sub_imports of profile trestle://profiles/NIST_SP-800-53_rev5/profile.json with title NIST Special Publication 800-53 Revision 5 MODERATE IMPACT BASELINE
trestle.core.resolver._import:99 DEBUG: import href is ../models/catalogs/NIST_SP-800-53_rev5_catalog.json
trestle.core.resolver._import:138 DEBUG: sub_import add pipeline for sub href ../models/catalogs/NIST_SP-800-53_rev5_catalog.json of main href trestle://profiles/NIST_SP-800-53_rev5/profile.json
trestle.core.resolver.merge:62 DEBUG: merge filter initialize
trestle.core.resolver.modify:59 DEBUG: modify initialize filter with profile NIST Special Publication 800-53 Revision 5 MODERATE IMPACT BASELINE
trestle.core.resolver.merge:210 DEBUG: merge entering process with 1 pipelines
trestle.core.resolver._import:103 DEBUG: import entering process with href ../models/catalogs/NIST_SP-800-53_rev5_catalog.json
trestle.core.remote.cache:58 DEBUG: Initializing FetcherBase
trestle.core.parser:57 DEBUG: Loading class "Catalog" from "trestle.oscal.catalog"
trestle.core.resolver._import:110 DEBUG: DIRECT YIELD in import of catalog Electronic Version of NIST SP 800-53 Rev 5 Controls and SP 800-53A Rev 5 Assessment Procedures
trestle.core.resolver.prune:198 DEBUG: prune yielding catalog Electronic Version of NIST SP 800-53 Rev 5 Controls and SP 800-53A Rev 5 Assessment Procedures with import ../models/catalogs/NIST_SP-800-53_rev5_catalog.json
trestle.core.resolver.merge:186 DEBUG: Profile has merge but no combine so defaulting to combine/merge.
trestle.core.resolver.modify:351 DEBUG: modify process with catalog Electronic Version of NIST SP 800-53 Rev 5 Controls and SP 800-53A Rev 5 Assessment Procedures using profile NIST Special Publication 800-53 Revision 5 MODERATE IMPACT BASELINE
trestle.core.resolver.modify:269 DEBUG: modify specify catalog Electronic Version of NIST SP 800-53 Rev 5 Controls and SP 800-53A Rev 5 Assessment Procedures for profile NIST Special Publication 800-53 Revision 5 MODERATE IMPACT BASELINE
trestle.core.commands.author.ssp:96 ERROR: Error while assembling SSP: 'VirtualMachine'
Traceback (most recent call last):
  File "/home/ec2-user/.local/lib/python3.9/site-packages/trestle/core/commands/author/ssp.py", line 499, in _run
    CatalogReader.read_ssp_md_content(md_path, ssp, comp_dict, part_id_map_by_label, context)
  File "/home/ec2-user/.local/lib/python3.9/site-packages/trestle/core/catalog/catalog_reader.py", line 364, in read_ssp_md_content
    ssp, control_id, comp_dict[comp_name], comp_info_dict, part_id_map_by_label
KeyError: 'VirtualMachine'