search

intel / dffml

The easiest way to use Machine Learning. Mix and match underlying ML libraries and data set sources. Generate new datasets or modify existing ones with ease.
https://intel.github.io/dffml/main/
MIT License
253 stars 138 forks source link

service: sw: src: change: notify: Service to facilitate poly repo pull model dev tooling #1315

Open johnandersen777 opened 2 years ago

johnandersen777 commented 2 years ago

Related:

johnandersen777 commented 2 years ago

ability to publish commit history across projects with email lists or chat notifcations to send on a per repo (or custom function) basis.

The goal here is that you're able to figure out what people are working on more quickly and collaborate on problems faster. Breaks down silos caused by pre-pr work. If everyones changes show up as code somewhere. You can quickly understand who is working on what at all times in your org. This allows you to map pre-pr dev branches in docs, infra as code (or gitops), code, etc. anything you have an API for you put it as code. Now all the sudden you have complete visibility into what everyone in your org is working on every time they save their work. This could be extended to provide daemon functionality which watches the filesystem in a repo (fanotify, inotify, nodemon, etc.) and sends out diffs for dirty repos (http://git-scm.com/docs/gitglossary#def_dirty). You can then map branches to OKRs. You can also map them to ANYTHING, via DID's and personal datastores. We then implement DID and personal datastore support as df.*Network objects. Then your code can dynamically just add whatever it wants where ever it wants and a dataflow anywhere else can pick it up (propagates through the blockchain). So you map the branches to the OKRs, issues (requirements, bugs, etc.), etc. Map OKRs and requirements then follow the links in the chain to see who is working on what objectives. Then follow the chain of DIDs to see if anyone is working on the same thing technically. Do this by comparing feature sets of projects as identified by metric collection (similar dependencies, similar API calls, walks like http server, quacks like http server, probably is http server, usage of aiohttp, http.server, common libs for language, check if in depenendnency tree, then move to source analysis). This model is useful for human developers working on projects but will also be useful when we start to get into the cross between automl auto feature engineering used here to come up with modificaitions to dataflows in order to create a distributed system for attempting to build new programs or refactor or patch security vulns (understanding of threat model via architecture of entrie solution as dataflow required to understand context around usage, intent). Need to rememeber to add the ability for any part of a dataflow run to output metrics which can be used to train a model to optimize for any high level goal as defined by the direction to automl that we'd like to move this metric this way. That metric can be as abitrary as "security" via model stacking, development time is then equivilant to time consumed on the distributed compute, or developer hours, whoever is faster. There are several important metrics in our final system.

Probably want to make the collection of these metrics done by flows. Look into doing this by just running a dataflow at any point within the execution of the dataflow to hit output stage operations, theres a method for this. We can then just trigger the metric collection flow periodically, maybe via running in a subflow with same contexts triggered by the running of an operation within the parent flow, or just triggered by caller whenever if running with #919 async for style.

johnandersen777 commented 2 years ago

Question around what to do client side when you receive a change notification from a peer.

Modes:

Example: Two devs: CR0/4

For example, devs Alice and Bob are working on a dataflow which has a SUT being used to run CR0/4 patchest validation (KSPP Issue 19).

They both have their own SUT which they are using actively for development. Alice is working in kexec-tools and Bob is working in QEMU. Recall that there were multiple failure conditions when this patch was first introduced which could be pursued in parallel with enough developers.

Start with the desired end state of the the system. The state the system (defined as end state after executing flow) would be given by a pass result from the test cases: https://github.com/pdxjohnny/protecting_cr4_host_setup/blob/master/test.sh

If either Alice or Bob make a change which results in one of those test cases passing, it is promoted to the best known data flow (BKD). The data flow defining the test run is of course also stored upstream, which is the BKD. However, what's new about this pull model approach is that Alice and Bob can now define what their BKD should be based on, dynamically. Bob can say if Alice finds any changes that advance the upstream flow, auto apply them to my local copy. To advance the BKD one must provide essentially a more useful thought than the current most useful thought. Where a thought here is a dataflow, and in the way that we are pulling in from other devs it's likely an overlay, or a diff between dataflows.

Let's go back in time before any of the work had been done. Alice and Bob both have dffml service sw change notify daemon repo running in each git repository.

graph TD

  subgraph tought_0[Thought 0]

    subgraph Alice
      alice_kexec_tools
      alice_linux
      alice_qemu
    end

    subgraph Bob
      bob_kexec_tools
      bob_linux
      bob_qemu
    end

  end
johnandersen777 commented 2 years ago

Dataflows as a web3 adapter via input network

johnandersen777 commented 2 years ago

Source: https://www.linkedin.com/posts/manisha-s-b72713200_data-datascience-datavisualization-ugcPost-6908783417899806720-IiA5

Example for chaos metric visualization:

image

johnandersen777 commented 2 years ago 
graph TD

    subgraph client
      http_request
      print
    end

    subgraph http_server
      json_to_dict
      listen_for_http_requests
      get_form_value
      get_mapping
      format_message
      respond_to_http_request
    end

    http_request --> listen_for_http_requests
    listen_for_http_requests --> get_form_value
    get_form_value --> get_mapping
    json_to_dict --> get_mapping
    get_form_value --> format_message
    get_mapping --> format_message
    format_message --> respond_to_http_request

    http_request --> print
johnandersen777 commented 2 years ago

Cross ref to https://github.com/intel/dffml/discussions/1406#discussioncomment-3181956

johnandersen777 commented 2 years ago

Cross ref to https://github.com/intel/dffml/discussions/1406#discussioncomment-3595322

💊 [![Alice,Mary,TheSophia](https://user-images.githubusercontent.com/5950433/218511948-a4e453f8-9ddd-44c8-b84e-deae3ee0d449.png)](https://pdxjohnny.github.io/redpill/)

Anarchy Elmo Says “Chaos for the Chaos God”

johnandersen777 commented 2 years ago

Policy on policy on downstream triggers to support sort of cached rebuild, minimal resource usage to complete full validation.

johnandersen777 commented 2 years ago

Source: https://github.com/ietf-scitt/use-cases/blob/main/scitt-components/scitt-rbac.md#promotion-across-rbac-instances PR: https://github.com/ietf-scitt/use-cases/pull/21 LICENSE: https://github.com/ietf-scitt/use-cases/blob/main/LICENSE Related: #1400

SCITT Role Based Access Control Enabling Decentralized Development

TODO:

  • Why we need limited access to information submitted to SCITT
  • Will SCITT support partitioned access within a SCITT instance, or will a SCITT instance be limited and only approved content is promoted to another SCITT instance that has broader access?

Promotion Across RBAC Instances

Whether a SCITT instance supports RBAC partitioning or not, users will configure multiple SCITT instances to support their dev through production workflows, or their internal systems being maintained separate from their public services. Users can promote the content from internal, secured SCITT instances, to SCITT instances with different permissions, or public access.

SCITT Instance Promotion

Design Options

RBAC at the SCITT Instance

Any access to a SCITT instances means access to all content

Pros

Simpler to implement

Cons

RBAC Partitioning a SCITT Instance

Pros

Cons

johnandersen777 commented 1 year ago

activitypubsecuritytxt

Pull Request Review (WIP: IETF SCITT: Use Case: OpenSSF Metrics): https://github.com/ietf-scitt/use-cases/pull/18

Collection of metric data into shared (crowdsourcable) DB. There are many repos to search, we want to enable self reporting and granularity as applicable to ad-hoc formed policy as desired by end-user.

activitypub extensions for security.txt

A methodology allowing organizations to nominate security contact points and policies via ActivityPub Actors. This allows for notifications to be federated of new lifecycle events. These lifecycle events might be VEX, SBOM, CSAF security advisory information, repository events, etc.

VEX documents should be aligned with the either the https://docs.oasis-open.org/csaf/csaf/v2.0/csaf-v2.0.html or OpenVEX specs: https://www.chainguard.dev/unchained/accelerate-vex-adoption-through-openvex. We can then communicate the IDs via ActivityPub like so.

Summary

When entities find security issues in source code, the correct channel to report security issues can be found if the repo has an RFC 9116 security.txt file with a Contact field. This contact field can be a URL which points to an ActivityPub Actor.

Via traversal of ActivityPub AcivityStream objects, reporters are enabled to discover reporting endpoints. Researchers are also enabled to receive up to date events by following declared ActivityPub Actors. When a researcher finds a vulnerability, they can submit their evidence to an eNotary (could be self notarized). The eNotary attests validity of the vuln and then replys to ActivityPub threads to facilite communication of valid vuln to upstream.

Imagine the following YAML as a directed graph whose upleveled pesudocode form is:

bob_vcs_repo:
  security.txt:
    Contact: https://activitypub.securitytxt.activitypub.example.org/bob

activitypub_service:
  endpoint_url: https://activitypub.securitytxt.activitypub.example.org
  actors:
    bob:
      attachment:
        type: "PropertyValue"
        name: "activitypubextensions"
        value: "<a href=\"https://activitypub.securitytxt.activitypub.example.org/users/bob/statuses/1\" target=\"_blank\" rel=\"nofollow noopener noreferrer me\"><span class=\"invisible\">https://</span><span class=\"\">activitypub.securitytxt.activitypub.example.org/users/bob/statuses/1</span><span class=\"invisible\"></span></a>"
      statuses:
        - id: "https://activitypub.securitytxt.activitypub.example.org/users/bob/statuses/1"
          content: "activitypubextensions"
          replies:
          - id: "https://activitypub.securitytxt.activitypub.example.org/users/bob/statuses/1/replies"
            type: "Collection"
            first:
              type: "CollectionPage"
              items:
              - "https://activitypub.securitytxt.activitypub.example.org/users/bob/statuses/2"
        - id: "https://activitypub.securitytxt.activitypub.example.org/users/bob/statuses/2"
          inReplyTo: "https://activitypub.securitytxt.activitypub.example.org/users/bob/statuses/1"
          content: "activitypubsecuritytxt"
          replies:
          - id: "https://activitypub.securitytxt.activitypub.example.org/users/bob/statuses/2/replies"
            type: "Collection"
            first:
              type: "CollectionPage"
              items:
              - "https://activitypub.securitytxt.activitypub.example.org/users/bob/statuses/3"
        - id: "https://activitypub.securitytxt.activitypub.example.org/users/bob/statuses/3"
          inReplyTo: "https://activitypub.securitytxt.activitypub.example.org/users/bob/statuses/2"
          content: "https://schema.example.org/vcs.push.1.0.0.schema.json"
          replies:
          - id: "https://activitypub.securitytxt.activitypub.example.org/users/bob/statuses/3/replies"
            type: "Collection"
            first:
              type: "CollectionPage"
              items:
              - "https://activitypub.securitytxt.activitypub.example.org/users/bob/statuses/4"
        - id: "https://activitypub.securitytxt.activitypub.example.org/users/bob/statuses/4"
          inReplyTo: "https://activitypub.securitytxt.activitypub.example.org/users/bob/statuses/3"
          content: "bob.registry.example.org/src_repo_name_contents_are_webhook_translated_to_vcs_push_manifest:sha256@babebabe"
    alice:
      statuses:
        - id: "https://activitypub.securitytxt.activitypub.example.org/users/alice/statuses/1"
          inReplyTo: "https://activitypub.securitytxt.activitypub.example.org/users/bob/statuses/4"
          content: "alice.registry.example.org/vex_contents_are_openvex_from_scratch:sha256@babebabe"

Scratch work upstream: https://github.com/intel/dffml/discussions/1406?sort=new#discussioncomment-4819872

graph TD
    subgraph bob[Bob's Cool Software]
      actor[ActivityPub Actor - &#x0040 bob&#x0040 forge.mycoolsoftware.example.com]
      actor_attachment[Attachment PropertyValue activitypubsecuritytxt]
      activitypubsecuritytxt_root_post[activitypubsecuritytxt root post]
      activitypubsecuritytxt_vcs_push[vcs.push root post]
      activitypubsecuritytxt_vcs_push_content[vcs.push content - content address of manifest instance in registry]

      actor --> actor_attachment
      actor_attachment -->|Link| activitypubsecuritytxt_root_post
      activitypubsecuritytxt_vcs_push -->|inReplyTo| activitypubsecuritytxt_root_post
      activitypubsecuritytxt_vcs_push_content -->|inReplyTo| activitypubsecuritytxt_vcs_push
    end

    subgraph alice[Alice]
      alice_shouldi_contribute[Static Analysis Result] -->|inReplyTo| activitypubsecuritytxt_vcs_push_content
    end
graph LR

  subgraph vcs_source[Version Controled Software]
    subgraph dffml_vcs_source[dffml.git]
      subgraph dffml_vcs_source_security_txt[security.txt]
        dffml_vcs_source_security_txt_contact[Contact: https://example.org/dffml]
      end
      subgraph dffml_vcs_source_dockerfile[dffml.Dockerfile]
        dffml_vcs_source_dockerfile_from_base[FROM upstream as dffml]
      end
      subgraph dffml_vcs_source_dockerfile_example[dffml.example.Dockerfile]
        dffml_vcs_source_dockerfile_example_from_base[FROM dffml &#x0040 sha:latest]
      end
      subgraph vcs_source_alice[dffml.git/entities/alice]
        subgraph alice_vcs_source_security_txt[security.txt]
          alice_vcs_source_security_txt_contact[Contact: https://example.org/alice]
        end
        subgraph alice_vcs_source_dockerfile[alice.Dockerfile]
          alice_vcs_source_dockerfile_from_base[FROM dffml &#x0040 sha:latest]
        end
        subgraph alice_vcs_source_dockerfile_shouldi_contribute[alice_shouldi_contribute.Dockerfile]
          alice_vcs_source_dockerfile_shouldi_contribute_from_base[FROM alice &#x0040 sha:latest]
          subgraph alice_shouldi_contribute[alice shoulid contribute -keys ARG_REPO_URL]
            alice_shouldi_contribute_git_clone[git clone ...]
            alice_shouldi_contribute_read_security_txt[grep Contact: security.txt]
            alice_shouldi_contribute_result[Static Analysis Result]

            alice_shouldi_contribute_git_clone --> alice_shouldi_contribute_read_security_txt
            dffml_vcs_source_security_txt_contact --> alice_shouldi_contribute_read_security_txt
            alice_shouldi_contribute_read_security_txt --> alice_shouldi_contribute_result
          end
        end
      end
    end
  end

  subgraph schema[Manifest ADRs]
    subgraph manifest_build_images_contianers[Build Image Container]
      manifest_build_images_contianers_intent[README.md/THREATS.md]
      manifest_build_images_contianers_schema[1.0.0.schema.json]
    end
  end

  subgraph manifest_instances[Manifest Instances]
    alice_manifest_build_images_contianers_alice_shouldi_contribute
  end

  subgraph transparency_logs[Transparency Logs]
    dffml_scitt[dffml.scitt.example.org]
    alice_scitt[alice.scitt.example.org]
  end

  subgraph factory[Secure Software Factories]
    subgraph build_images_contianers[build_images_contianers.yml]
    end

    subgraph factory_container_image_registries[Container Image Registry https://oras.land]
      subgraph dffml_factory_container_image_registries_project[DFFML Images]
        dffml_container_image[dffml:latest]
      end
      subgraph alice_factory_container_image_registries_project[Alice Images]
        alice_container_image[alice:latest]
        alice_shouldi_contribute_scan_results[shouldicontribute &#x0040 sha384:babebabe]
      end
    end

    build_images_contianers --> dffml_scitt
    build_images_contianers --> alice_scitt
  end

  subgraph protocol_knowledge_graph_activity_pub[ActivityPub]
    subgraph ActivityPubExtensionsForSecurityTXT[activitypub extensions for security.txt]
      subgraph dffml_security_txt_contact[dffml.git/security.txt:Contact]
        dffml_actor[ActivityPub Actor - &#x0040 dffml &#x0040 example.org]
        dffml_actor_attachment[Attachment PropertyValue activitypubsecuritytxt]
        dffml_activitypubsecuritytxt_root_post[activitypubsecuritytxt root post]
        dffml_activitypubsecuritytxt_vcs_push[vcs.push root post]
        dffml_activitypubsecuritytxt_vcs_push_content[vcs.push content - content address of manifest instance in registry]

        dffml_actor --> dffml_dffml_actor_attachment
        dffml_actor_attachment -->|Link| dffml_activitypubsecuritytxt_root_post
        dffml_activitypubsecuritytxt_vcs_push -->|inReplyTo| dffml_activitypubsecuritytxt_root_post
        dffml_activitypubsecuritytxt_vcs_push_content -->|inReplyTo| dffml_activitypubsecuritytxt_vcs_push
      end

      subgraph alice_security_txt_contact[dffml.git/entites/alice/security.txt:Contact]
        alice_actor[ActivityPub Actor - &#x0040 alice &#x0040 example.org]
        alice_actor_attachment[Attachment PropertyValue activitypubsecuritytxt]
        alice_activitypubsecuritytxt_root_post[activitypubsecuritytxt root post]
        alice_activitypubsecuritytxt_vcs_push[vcs.push root post]
        alice_activitypubsecuritytxt_vcs_push_content[vcs.push content - content address of manifest instance in registry]

        alice_actor --> alice_actor_attachment
        alice_actor_attachment -->|Link| alice_activitypubsecuritytxt_root_post
        alice_activitypubsecuritytxt_vcs_push -->|inReplyTo| alice_activitypubsecuritytxt_root_post
        alice_activitypubsecuritytxt_vcs_push_content -->|inReplyTo| alice_activitypubsecuritytxt_vcs_push
      end
    end

    alice_actor -->|follow| dffml_actor
  end

  subgraph render_knowledge_graph_agora[Agora]
  end

  alice_vcs_source_dockerfile_shouldi_contribute

  dffml_vcs_source_security_txt_contact --> dffml_actor
  alice_vcs_source_security_txt_contact --> alice_actor

  alice_shouldi_contribute_result --> alice_shouldi_contribute_scan_results
  alice_shouldi_contribute_scan_results --> |inReplyTo| dffml_vcs_source_dockerfile_example_from_base

  dffml_container_image --> dffml_vcs_source_dockerfile_example_from_base
  alice_container_image --> alice_vcs_source_dockerfile_example_from_base

  dffml_vcs_source_dockerfile_example_from_base --> dffml_activitypubsecuritytxt_vcs_push
  dffml_activitypubsecuritytxt_vcs_push --> build_images_contianers_trigger
  alice_vcs_source_dockerfile_example_from_base --> alice_activitypubsecuritytxt_vcs_push

  alice_shouldi_contribute
{
    "@context": [
        "https://www.w3.org/ns/activitystreams",
        "https://w3id.org/security/v1",
    ],
    "id": "https://mastodon.social/users/alice",
    "type": "Person",
    "following": "https://mastodon.social/users/alice/following",
    "followers": "https://mastodon.social/users/alice/followers",
    "inbox": "https://mastodon.social/users/alice/inbox",
    "outbox": "https://mastodon.social/users/alice/outbox",
    "featured": "https://mastodon.social/users/alice/collections/featured",
    "featuredTags": "https://mastodon.social/users/alice/collections/tags",
    "preferredUsername": "alice",
    "name": "Alice",
    "summary": "An ActivityPub Actor",
    "url": "https://mastodon.social/@alice",
    "publicKey": {
        "id": "https://mastodon.social/users/alice#main-key",
        "owner": "https://mastodon.social/users/alice",
        "publicKeyPem": "-----BEGIN PUBLIC KEY-----\nMIIBIjANBgk\n-----END PUBLIC KEY-----\n"
    },
    "attachment": [
        {
            "type": "PropertyValue",
            "name": "activitypubextensions",
            "value": "<a href=\"https://mastodon.social/users/alice/statuses/1\" target=\"_blank\" rel=\"nofollow noopener noreferrer me\"><span class=\"invisible\">https://</span><span class=\"\">mastodon.social/users/alice/statuses/1</span><span class=\"invisible\"></span></a>"
        }
    ],
    "endpoints": {
        "sharedInbox": "https://mastodon.social/inbox"
    }
}
{
    "@context": [
        "https://www.w3.org/ns/activitystreams"
    ],
    "id": "https://mastodon.social/users/alice/statuses/1",
    "type": "Note",
    "summary": null,
    "inReplyTo": null,
    "published": "2022-11-11T04:40:17Z",
    "url": "https://mastodon.social/@alice/1",
    "attributedTo": "https://mastodon.social/users/alice",
    "to": [
        "https://www.w3.org/ns/activitystreams#Public"
    ],
    "cc": [
        "https://mastodon.social/users/alice/followers"
    ],
    "sensitive": false,
    "content": "activitypubextensions",
    "updated": "2022-11-11T04:42:27Z",
    "attachment": [],
    "replies": {
        "id": "https://mastodon.social/users/alice/statuses/1/replies",
        "type": "Collection",
        "first": {
            "type": "CollectionPage",
            "next": "https://mastodon.social/users/alice/statuses/1/replies?min_id=1&page=true",
            "partOf": "https://mastodon.social/users/alice/statuses/1/replies",
            "items": [
                "https://mastodon.social/users/alice/statuses/2"
            ]
        }
    }
}
{
    "@context": [
        "https://www.w3.org/ns/activitystreams"
    ],
    "id": "https://mastodon.social/users/alice/statuses/2",
    "type": "Note",
    "summary": null,
    "inReplyTo": "https://mastodon.social/users/alice/statuses/1",
    "published": "2022-11-11T04:40:17Z",
    "url": "https://mastodon.social/@alice/2",
    "attributedTo": "https://mastodon.social/users/alice",
    "to": [
        "https://www.w3.org/ns/activitystreams#Public"
    ],
    "cc": [
        "https://mastodon.social/users/alice/followers"
    ],
    "sensitive": false,
    "content": "activitypubsecuritytxt",
    "updated": "2022-11-11T04:42:27Z",
    "attachment": [],
    "replies": {
        "id": "https://mastodon.social/users/alice/statuses/2/replies",
        "type": "Collection",
        "first": {
            "type": "CollectionPage",
            "next": "https://mastodon.social/users/alice/statuses/2/replies?min_id=2&page=true",
            "partOf": "https://mastodon.social/users/alice/statuses/2/replies",
            "items": [
                "https://mastodon.social/users/alice/statuses/3"
            ]
        }
    }
}
{
    "@context": [
        "https://www.w3.org/ns/activitystreams"
    ],
    "id": "https://mastodon.social/users/alice/statuses/3",
    "type": "Note",
    "summary": null,
    "inReplyTo": "https://mastodon.social/users/alice/statuses/2",
    "published": "2022-11-11T04:40:17Z",
    "url": "https://mastodon.social/@alice/3",
    "attributedTo": "https://mastodon.social/users/alice",
    "to": [
        "https://www.w3.org/ns/activitystreams#Public"
    ],
    "cc": [
        "https://mastodon.social/users/alice/followers"
    ],
    "sensitive": false,
    "content": "vcs.push",
    "updated": "2022-11-11T04:42:27Z",
    "attachment": [],
    "replies": {
        "id": "https://mastodon.social/users/alice/statuses/3/replies",
        "type": "Collection",
        "first": {
            "type": "CollectionPage",
            "next": "https://mastodon.social/users/alice/statuses/3/replies?min_id=3&page=true",
            "partOf": "https://mastodon.social/users/alice/statuses/3/replies",
            "items": [
                "https://mastodon.social/users/alice/statuses/4"
            ]
        }
    }
}
{
    "@context": [
        "https://www.w3.org/ns/activitystreams"
    ],
    "id": "https://mastodon.social/users/alice/statuses/4",
    "type": "Note",
    "summary": null,
    "inReplyTo": "https://mastodon.social/users/alice/statuses/3",
    "published": "2022-11-11T04:54:56Z",
    "url": "https://mastodon.social/@alice/4",
    "attributedTo": "https://mastodon.social/users/alice",
    "to": [
        "https://www.w3.org/ns/activitystreams#Public"
    ],
    "cc": [
        "https://mastodon.social/users/alice/followers"
    ],
    "sensitive": false,
    "content": "registry.example.org/vex:sha256@babebabe",
    "attachment": [],
    "tag": [],
    "replies": {
        "id": "https://mastodon.social/users/alice/statuses/4/replies",
        "type": "Collection",
        "first": {
            "type": "CollectionPage",
            "next": "https://mastodon.social/users/alice/statuses/4/replies?only_other_accounts=true&page=true",
            "partOf": "https://mastodon.social/users/alice/statuses/4/replies",
            "items": []
        }
    }
}
{
  "@context": "https://openvex.dev/ns",
  "@id": "https://mastodon.social/users/alice/statuses/vex-sha256@feedface",
  "author": "GitHub Actions <actions@github.com>",
  "role": "GitHub Actions",
  "timestamp": "2023-02-02T14:24:00.000000000-07:00",
  "version": "1",
  "statements": [
    {
      "vulnerability": "vex-vcspush-sha256@feedface",
      "products": [
        "pkg:github/intel/dffml@ddb32a4e65b0d79c7561ce2bdde16d963c8abde1"
      ],
      "status": "not_affected",
      "justification": "vulnerable_code_not_in_execute_path"
      "impact_statement": "registry.example.org/vcspush:sha256@feedface",
    }
  ]
}
$ curl -sfL https://vcs.example.org/push/outbox | jq --unbuffered -r '.orderedItems[].object.content' | grep stream_of | grep modified | jq -r --unbuffered '.commits[].modified[]'
Dockerfile

Why?

Decentralized

Actors can be spun up ad-hoc, mirrors decentralized nature of OSS development.

Enables projects to update based on policy.

Upstream of following mermaid: https://github.com/intel/dffml/tree/alice/docs/tutorials/rolling_alice/0000_architecting_alice#what-is-alice

graph BT
    subgraph Alice[Alice the Entity]
        subgraph compute[Compute]
        Web5[Web 5]
        KCP
        CI_CD[CI/CD]
        end
        subgraph soul[Strategic Plans and Principles]
        Threat_Modeling[Threat Modeling]
        Debug
        end
        subgraph collector[Collector]
        subgraph dynamic_analysis[Dynamic Analysis]
            policy[policy.yml]
            sandbox_policy_generator[Adaptive Sandboxing]
        end
        subgraph static_analysis[Static Analysis]
            cve_bin_tool[CVE Binary Tool]
            SBOM
        end
        end
        Open_Architecture
        Open_Architecture[Alice the Open Architecture]
        snapshot_system_context[Alice the Overlay<br>Snapshot of System Context]
        orchestartor[Orchestartor]

        Open_Architecture --> Threat_Modeling
        Open_Architecture --> Debug

        Threat_Modeling --> orchestartor
        Debug --> orchestartor

        orchestartor --> KCP
        orchestartor --> Web5
        orchestartor --> CI_CD

        CI_CD --> snapshot_system_context
        KCP --> snapshot_system_context
        Web5 --> snapshot_system_context

        snapshot_system_context --> sandbox_policy_generator
        snapshot_system_context --> cve_bin_tool

        sandbox_policy_generator --> policy --> Open_Architecture
        cve_bin_tool --> SBOM --> Open_Architecture
        cve_bin_tool --> VEX -->|Trigger validation run of mitigation suggestion| orchestartor
        policy -->|Check if policy says out of scope<br>client vs. server usage| VEX
    end
$ curl -ku alice:$(cat ../password) -X POST -v http://localhost:8000/admin/follow/push/vcs.activitypub.securitytxt.dffml.chadig.com/443/https
* Uses proxy env variable no_proxy == 'localhost,127.0.0.0/8,::1'
*   Trying 127.0.0.1:8000...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 8000 (#0)
* Server auth using Basic with user 'alice'
> POST /admin/follow/push/vcs.activitypub.securitytxt.dffml.chadig.com/443/https HTTP/1.1
> Host: localhost:8000
> Authorization: Basic YWxpY2U6ODkyZTI1Y2MwMTMzYTcwYTEzMzRlYTIyNmQ2NDNkNTNhMDRjYzc5MDIwOWM0MzY1ZTUwMzA2Mjc3MGVmZTdmOWVlM2M3MDI4OWNlODdiYzJmZThiYzE2NGNlNTQxYTYx
> User-Agent: curl/7.68.0
> Accept: */*
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 204 No Content
< X-Powered-By: Express
< ETag: W/"a-bAsFyilMr4Ra1hIU5PyoyFRunpI"
< Date: Mon, 13 Feb 2023 14:50:51 GMT
< Connection: keep-alive
< Keep-Alive: timeout=5
<
* Connection #0 to host localhost left intact
Dumbo listening on port 8000…
GET /push 200 1493 - 7.432 ms
Data to compare (request-target): post /push/inbox
host: vcs.activitypub.securitytxt.dffml.chadig.com
date: Mon, 13 Feb 2023 14:50:49 GMT
digest: SHA-256=4byRebHbzxk6BlJopQYVQcI+9YiHojWKhaI2S0J8w68=
Data to sign (request-target): post /alice/inbox
host: d30a15e2d986dc.lhr.life
date: Mon, 13 Feb 2023 14:50:50 GMT
digest: SHA-256=QOPUiXd5oq6u0i+DNQu9TZRIydnRewGdlN1eoiaEsKs=
GET /push 200 1493 - 1.654 ms
POST /push/inbox 204 - - 1557.550 ms
$ curl -sfL https://github.com/pdxjohnny/activitypub-starter-kit/archive/refs/heads/alternate_port.tar.gz | tar xvz
$ cd activitypub-starter-kit-alternate_port
$ cat > .env <<'EOF'
# The Node environment
NODE_ENV="production"

# The path to the database schema
SCHEMA_PATH="db/schema.sql"

# The path to the database file
DATABASE_PATH="db/database.sqlite3"

# The hostname (i.e. the "example.com" part of https://example.com/alice)
HOSTNAME="vcs.activitypub.securitytxt.dffml.chadig.com"

# The account name (i.e. the "alice" part of https://example.com/alice)
ACCOUNT="push"
EOF
$ npm i
$ head -n 10000 /dev/urandom | sha384sum | awk '{print $1}' | tee ../webhook
$ head -n 10000 /dev/urandom | sha384sum | awk '{print $1}' | tee ../password
$ openssl genrsa -out keypair.pem 4096 && openssl rsa -in keypair.pem -pubout -out publickey.crt && openssl pkcs8 -topk8 -inform PEM -outform PEM -nocrypt -in keypair.pem -out pkcs8.key
$ mkdir node_modules/@types/simple-proxy-agent/
$ echo "declare module 'simple-proxy-agent';" | tee node_modules/@types/simple-proxy-agent/index.d.ts
$ npm run build
$ FDQN=vcs.activitypub.securitytxt.dffml.chadig.com WEBHOOK_PATH=$(cat ../webhook) NODE_ENV=production PORT=8000 ACCOUNT=push ADMIN_USERNAME=admin ADMIN_PASSWORD=$(cat ../password) PUBLIC_KEY=$(cat publickey.crt) PRIVATE_KEY=$(cat pkcs8.key) npm run start

> dumbo@1.0.0 start
> node build/index.js

Dumbo listening on port 8000…
GET /push 200 1493 - 8.201 ms
GET /push 200 1493 - 1.200 ms
POST /push/inbox 204 - - 1583.186 ms
$ rm -f db/database.sqlite3; ssh -R 80:localhost:8000 nokey@localhost.run 2>&1 | tee >(grep --line-buffered 'tunneled with tls termination' | awk -W interactive '{print $1}' | xargs -l -I '{}' -- sh -c 'reset; echo "{}"; PROTO=https FDQN="{}" WEBHOOK_PATH=$(cat ../webhook) NODE_ENV=production PORT=8000 ACCOUNT=alice ADMIN_USERNAME=alice ADMIN_PASSWORD=$(cat ../password) PUBLIC_KEY=$(cat publickey.crt) PRIVATE_KEY=$(cat pkcs8.key) npm run start &
c4d2dfa777b86f.lhr.life

> dumbo@1.0.0 start
> node build/index.js

Dumbo listening on port 8000…
GET /alice 200 1354 - 2.530 ms
GET /alice 200 1354 - 0.895 ms
POST /alice/inbox 204 - - 71.294 ms
POST /admin/follow/push/vcs.activitypub.securitytxt.dffml.chadig.com/443/https 204 - - 3183.157 ms
$ curl -ku alice:$(cat ../password) -X POST -v http://localhost:8000/admin/follow/push/vcs.activitypub.securitytxt.dffml.chadig.com/443/https
$ websocat --exit-on-eof --basic-auth alice:$(cat ../password) ws://localhost:8000/listen/websocket
$ cat > post.json <<'EOF'
{
    "object": {
        "type": "Note",
        "content": "OUR PROPHECY MUST BE FULFILLED!!! https://github.com/intel/dffml/pull/1401#issuecomment-1168023959"
    }
}
EOF
$ curl -u admin:$(cat ../password) -X POST --header "Content-Type: application/json" --data @post.json -v http://localhost:8000/admin/create
POST /admin/create 204 - - 133.004 ms
{
    "@context": "https://www.w3.org/ns/activitystreams",
    "type": "Create",
    "published": "2023-02-13T15:39:08.628Z",
    "actor": "https://vcs.activitypub.securitytxt.dffml.chadig.com/push",
    "to": [
        "https://www.w3.org/ns/activitystreams#Public"
    ],
    "cc": [
        "https://eb62a3437cf6a9.lhr.life/alice"
    ],
    "object": {
        "attributedTo": "https://vcs.activitypub.securitytxt.dffml.chadig.com/push",
        "published": "2023-02-13T15:39:08.628Z",
        "to": [
            "https://www.w3.org/ns/activitystreams#Public"
        ],
        "cc": [
            "https://vcs.activitypub.securitytxt.dffml.chadig.com/push/followers"
        ],
        "type": "Note",
        "content": "OUR PROPHECY MUST BE FULFILLED!!! https://github.com/intel/dffml/pull/1401#issuecomment-1168023959",
        "id": "https://vcs.activitypub.securitytxt.dffml.chadig.com/push/posts/15f4de9c-a582-4f9d-8372-a740a5ffe6a8"
    },
    "id": "https://vcs.activitypub.securitytxt.dffml.chadig.com/push/posts/58f883cd-0252-4319-a934-3ca2eb062f62"
}

hack-the-planet-hackers-gif

Side from Andrew Ng's Intel Innovation 2022 Luminary Keynote Source: https://www.intel.com/content/www/us/en/newsroom/news/2022-intel-innovation-day-2-livestream-replay.html#gs.iex8mr image

Gif of Alice on roller skates throwing a bowling ball which is a software vuln, strike, she frontflips throwing knife style throws the pins into pull requests. We zoom out and see her just doing this over and over again around the Entity Analysis Trinity. Intent/LTM is where the throwing board is. Bowling alley is static analysis and the end of the bowling ally where she frontflips over (through hoop of CI/CD fire?) is where she pics up the pins and throws them as pull request (titles and numbers maybe, pulls/1401 style maybe?) knives into the board at the top which is the LTM and codebase. Then from top, LTM to static analysis where bowling alley starts shes in the lab, cooking up the vuln or maybe out looking for it. Or maybe refactoring after pull requests!

Entity Analysis Trinity

Misc. Notes

Mike from OpenSSF has been thinking about SCITT as a schema and rules on how one would assert facts, weither it's confidential compute or traditional permissions is impelmenetation details.

BEGIN UNTOUCHED PRE-FORK CONENT

Fields

Deployment Options

Just as security.txt can be deployed into either the root or the .well-known directory of a webserver, DNS Security TXT can be deployed to either the apex of a domain, or under a specially created _security.domain.com subdomain. This approach allows organizations to decide the approach that suits them best.

Apex approach

Pros:

Cons:

Description Domain Type Content
Direct email reporting contact domain.com TXT "security_contact=mailto:security@domain.com"
Direct web form reporting contact domain.com TXT "security_contact=https://domain.com/report-security-issue"
3rd party web form reporting contact domain.com TXT "security_contact=https://bugcrowd.com/domain/report"
Direct policy URL .domain.com TXT "security_policy=https://domain.com/security-policy"
3rd party web form reporting URL domain.com TXT "security_policy=https://bugcrowd.com/domain"

_security.domain.com approach

Pros:

Cons:

Description Domain Type Content
Direct email reporting contact _security.domain.com TXT "security_contact=mailto:security@domain.com"
Direct web form reporting contact _security.domain.com TXT "security_contact=https://domain.com/report-security-issue"
3rd party web form reporting contact _security.domain.com TXT "security_contact=https://bugcrowd.com/domain/report"
Direct policy URL _security.domain.com TXT "security_policy=https://domain.com/security-policy"
3rd party web form reporting URL _security.domain.com TXT "security_policy=https://bugcrowd.com/domain"

Frequently Asked Questions**

Is this a replacement for security.txt?

Is this giving anyone permission to hack my organization?

Can I deploy this on a subdomain?

Who in my organization do I need to engage with to get these records in place?

Will adding an email address expose me to spam bots?

How do I put these entries into my DNS?

Created with <3 by John Carroll and Casey Ellis for The disclose.io Project.

Forked from dnssecuritytxt by John Carroll and Casey Ellis for The disclose.io Project.

johnandersen777 commented 1 year ago

Cross ref: Alice Engineering Coms 2023-02-13 @pdxjohnny Engineering Logs (reproduced below)

Data to sign (request-target): post /push/inbox
host: vcs.activitypub.securitytxt.dffml.chadig.com
date: Mon, 13 Feb 2023 14:38:08 GMT
digest: SHA-256=xvQlt8xT5UzECmeLhU94qWLWL6hHug6smeMqgqEihTE=
Data to compare (request-target): post /push/inbox
host: vcs.activitypub.securitytxt.dffml.chadig.com:80
date: Mon, 13 Feb 2023 14:38:08 GMT
digest: SHA-256=xvQlt8xT5UzECmeLhU94qWLWL6hHug6smeMqgqEihTE=
Error: Invalid request signature.
return `${header}: ${req.get(header)}`
$ FDQN=vcs.activitypub.securitytxt.dffml.chadig.com WEBHOOK_PATH=$(cat ../webhook) NODE_ENV=production PORT=8000 ACCOUNT=push ADMIN_USERNAME=admin ADMIN_PASSWORD=$(cat ../password) PUBLIC_KEY=$(cat publickey.crt) PRIVATE_KEY=$(cat pkcs8.key) npm run start

> dumbo@1.0.0 start
> node build/index.js

Dumbo listening on port 8000…
GET /push 200 1493 - 11.075 ms
Data to compare (request-target): post /push/inbox
host: :8000
date: Mon, 13 Feb 2023 14:44:32 GMT
digest: SHA-256=3TGS+O9ajWB71TSN6Tm5IBVBizH35dxrE1wDw7LAw9Y=
Error: Invalid request signature.
    at verify (file:///home/alice/activitypub-starter-kit-alternate_port/build/request.js:123:15)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
    at async file:///home/alice/activitypub-starter-kit-alternate_port/build/activitypub.js:36:16
POST /push/inbox 401 12 - 616.413 ms

use-the-source

$ git grep FDQN
src/index.ts:7:import { ADMIN_USERNAME, ADMIN_PASSWORD, ACCOUNT, HOSTNAME, PORT, PROTO, FDQN } from "./env.js";
src/index.ts:78:const endpoint: string = (FDQN != null ? FDQN: `${HOSTNAME}:${PORT}`);
       else if (FDQN != null && header === "host")
        return `host: ${FDQN}`;
$ curl -ku alice:$(cat ../password) -X POST -v http://localhost:8000/admin/follow/push/vcs.activitypub.securitytxt.dffml.chadig.com/443/https
* Uses proxy env variable no_proxy == 'localhost,127.0.0.0/8,::1'
*   Trying 127.0.0.1:8000...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 8000 (#0)
* Server auth using Basic with user 'alice'
> POST /admin/follow/push/vcs.activitypub.securitytxt.dffml.chadig.com/443/https HTTP/1.1
> Host: localhost:8000
> Authorization: Basic YWxpY2U6ODkyZTI1Y2MwMTMzYTcwYTEzMzRlYTIyNmQ2NDNkNTNhMDRjYzc5MDIwOWM0MzY1ZTUwMzA2Mjc3MGVmZTdmOWVlM2M3MDI4OWNlODdiYzJmZThiYzE2NGNlNTQxYTYx
> User-Agent: curl/7.68.0
> Accept: */*
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 204 No Content
< X-Powered-By: Express
< ETag: W/"a-bAsFyilMr4Ra1hIU5PyoyFRunpI"
< Date: Mon, 13 Feb 2023 14:50:51 GMT
< Connection: keep-alive
< Keep-Alive: timeout=5
<
* Connection #0 to host localhost left intact
Dumbo listening on port 8000…
GET /push 200 1493 - 7.432 ms
Data to compare (request-target): post /push/inbox
host: vcs.activitypub.securitytxt.dffml.chadig.com
date: Mon, 13 Feb 2023 14:50:49 GMT
digest: SHA-256=4byRebHbzxk6BlJopQYVQcI+9YiHojWKhaI2S0J8w68=
Data to sign (request-target): post /alice/inbox
host: d30a15e2d986dc.lhr.life
date: Mon, 13 Feb 2023 14:50:50 GMT
digest: SHA-256=QOPUiXd5oq6u0i+DNQu9TZRIydnRewGdlN1eoiaEsKs=
GET /push 200 1493 - 1.654 ms
POST /push/inbox 204 - - 1557.550 ms
$ curl -sfL https://github.com/pdxjohnny/activitypub-starter-kit/archive/refs/heads/alternate_port.tar.gz | tar xvz
$ cd activitypub-starter-kit-alternate_port
$ cat > .env <<'EOF'
# The Node environment
NODE_ENV="production"

# The path to the database schema
SCHEMA_PATH="db/schema.sql"

# The path to the database file
DATABASE_PATH="db/database.sqlite3"

# The hostname (i.e. the "example.com" part of https://example.com/alice)
HOSTNAME="vcs.activitypub.securitytxt.dffml.chadig.com"

# The account name (i.e. the "alice" part of https://example.com/alice)
ACCOUNT="push"
EOF
$ npm i
$ head -n 10000 /dev/urandom | sha384sum | awk '{print $1}' | tee ../webhook
$ head -n 10000 /dev/urandom | sha384sum | awk '{print $1}' | tee ../password
$ openssl genrsa -out keypair.pem 4096 && openssl rsa -in keypair.pem -pubout -out publickey.crt && openssl pkcs8 -topk8 -inform PEM -outform PEM -nocrypt -in keypair.pem -out pkcs8.key
$ mkdir node_modules/@types/simple-proxy-agent/
$ echo "declare module 'simple-proxy-agent';" | tee node_modules/@types/simple-proxy-agent/index.d.ts
$ npm run build
$ FDQN=vcs.activitypub.securitytxt.dffml.chadig.com WEBHOOK_PATH=$(cat ../webhook) NODE_ENV=production PORT=8000 ACCOUNT=push ADMIN_USERNAME=admin ADMIN_PASSWORD=$(cat ../password) PUBLIC_KEY=$(cat publickey.crt) PRIVATE_KEY=$(cat pkcs8.key) npm run start

> dumbo@1.0.0 start
> node build/index.js

Dumbo listening on port 8000…
GET /push 200 1493 - 8.201 ms
GET /push 200 1493 - 1.200 ms
POST /push/inbox 204 - - 1583.186 ms
$ rm -f db/database.sqlite3; ssh -R 80:localhost:8000 nokey@localhost.run 2>&1 | tee >(grep --line-buffered 'tunneled with tls termination' | awk -W interactive '{print $1}' | xargs -l -I '{}' -- sh -c 'reset; echo "{}"; PROTO=https FDQN="{}" WEBHOOK_PATH=$(cat ../webhook) NODE_ENV=production PORT=8000 ACCOUNT=alice ADMIN_USERNAME=alice ADMIN_PASSWORD=$(cat ../password) PUBLIC_KEY=$(cat publickey.crt) PRIVATE_KEY=$(cat pkcs8.key) npm run start &
c4d2dfa777b86f.lhr.life

> dumbo@1.0.0 start
> node build/index.js

Dumbo listening on port 8000…
GET /alice 200 1354 - 2.530 ms
GET /alice 200 1354 - 0.895 ms
POST /alice/inbox 204 - - 71.294 ms
POST /admin/follow/push/vcs.activitypub.securitytxt.dffml.chadig.com/443/https 204 - - 3183.157 ms
$ curl -ku alice:$(cat ../password) -X POST -v http://localhost:8000/admin/follow/push/vcs.activitypub.securitytxt.dffml.chadig.com/443/https
$ websocat --exit-on-eof --basic-auth alice:$(cat ../password) ws://localhost:8000/listen/websocket
$ cat > post.json <<'EOF'
{
    "object": {
        "type": "Note",
        "content": "OUR PROPHECY MUST BE FULFILLED!!! https://github.com/intel/dffml/pull/1401#issuecomment-1168023959"
    }
}
EOF
$ curl -u admin:$(cat ../password) -X POST --header "Content-Type: application/json" --data @post.json -v http://localhost:8000/admin/create
POST /admin/create 204 - - 133.004 ms
file:///home/alice/activitypub-starter-kit-alternate_port/build/request.js:19
        throw new Error(`Received ${res.status} fetching actor. Body: ${response_body}`);
              ^

Error: Received 503 fetching actor. Body: no ssh tunnel here :(
    at fetchActor (file:///home/alice/activitypub-starter-kit-alternate_port/build/request.js:19:15)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
    at async send (file:///home/alice/activitypub-starter-kit-alternate_port/build/request.js:31:19)
{
    "@context": "https://www.w3.org/ns/activitystreams",
    "type": "Create",
    "published": "2023-02-13T15:39:08.628Z",
    "actor": "https://vcs.activitypub.securitytxt.dffml.chadig.com/push",
    "to": [
        "https://www.w3.org/ns/activitystreams#Public"
    ],
    "cc": [
        "https://eb62a3437cf6a9.lhr.life/alice"
    ],
    "object": {
        "attributedTo": "https://vcs.activitypub.securitytxt.dffml.chadig.com/push",
        "published": "2023-02-13T15:39:08.628Z",
        "to": [
            "https://www.w3.org/ns/activitystreams#Public"
        ],
        "cc": [
            "https://vcs.activitypub.securitytxt.dffml.chadig.com/push/followers"
        ],
        "type": "Note",
        "content": "OUR PROPHECY MUST BE FULFILLED!!! https://github.com/intel/dffml/pull/1401#issuecomment-1168023959",
        "id": "https://vcs.activitypub.securitytxt.dffml.chadig.com/push/posts/15f4de9c-a582-4f9d-8372-a740a5ffe6a8"
    },
    "id": "https://vcs.activitypub.securitytxt.dffml.chadig.com/push/posts/58f883cd-0252-4319-a934-3ca2eb062f62"
}

hack-the-planet-hackers-gif