fossas / fossa-cli

Fast, portable and reliable dependency analysis for any codebase. Supports license & vulnerability scanning for large monoliths. Language-agnostic; integrates with 20+ build systems.
https://fossa.com
Other
1.26k stars 173 forks source link

Ane-1711 - [2/3] integrate jar fingerprint method #1444

Closed csasarak closed 2 months ago

csasarak commented 2 months ago

Overview

  1. This PR takes results from running millhone in #1442, inserts them in the appropriate places in the container scan, and then uploads them.
  2. The CLI now tries to contact the new analysis service first before falling back to the old one which doesn't support jars in containers.
  3. I also did some light refactoring to eliminate a call to the organization api.
  4. I provided a way to get both the stderr and stdout results of executing a command.

Acceptance criteria

  1. When scanning a container that contains Jar files, the CLI now inserts the relevant observations into the container scan and uploads them.
  2. If the new analysis service isn't available/fails, it falls back to the old one without jars in containers support and advises the user to contact FOSSA.
  3. The raw traces and raw output from millhone should be available in the debug bundle.

Testing plan

Testing in this PR is manual in the interest of getting the review process started. I plan to add automated tests in a later PR in this stack.

Before any testing you must build millhone and ensure that it is in the right location for GHC to pick it up:

make build-embedded-rust-bins

OR

cargo build --release --bin millhone

To test that the proxying is working:

  1. Check out the analysis service and then run sudo caddy run from its root directory.
  2. Start up the analysis service.
  3. Run cabal run fossa -- container analyze --revision rev-1 -e 'https://localhost' --debug jetty
  4. Check the UI and see that the project exists and has Jar results.
  5. The debug bundle should also have messages saying that it POSTed to the proxy.

To test that the fallback is working:

  1. Stop the caddy and analysis services if they are running locally
  2. Run cabal run fossa -- container analyze --revision rev-2 --debug jetty
  3. If you look in the UI at this project, you'll notice that there are no jars.
  4. Additionally, if you look in the debug bundle you should see a section like this:
    {
    "duration": "0.984492000",
    "events": [
    {
      "duration": "0.486788000",
      "events": [
        "POST https://app.fossa.com/api/proxy/analysis/api/container/upload",
        {
          "error": "Errata {errataHeader = Just \"The FOSSA endpoint returned an unexpected status code: 404\", errataBlocks = [], errataBody = Just \"Response:\\n 404\\nWhile HTTP responses typically come from the FOSSA API, it's also possible that some other device on the network sent this response.\\nThis error is often transient, so trying again in a few minutes may resolve the issue.\\n\\ESC[0;94mSupport: \\ESC[0mIf this issue persists, please contact FOSSA support at https://support.fossa.com\\nIn your bug report, please include FOSSA's debug bundle file: fossa.debug.json.gz.\\nYou can generate debug bundle by using `--debug` flag, for example: `fossa analyze --debug`\\n\\ESC[0;96mHelp: \\ESC[0mHTTP status codes - https://developer.mozilla.org/en-US/docs/Web/HTTP/Status\\n\\ESC[0;92mContext: \\ESC[0mRequest: POST https://app.fossa.com/api/proxy/analysis/api/container/upload?locator=custom%2Bquay.io%2Fkeycloak%2Fkeycloak%24rev-6&cliVersion=&managedBuild=true&scanType=native&title=quay.io%2Fkeycloak%2Fkeycloak\"}"
        }
      ],
      "scope": "Calling FOSSA API",
      "startTime": "2024-06-27 17:22:00.874124 UTC"
    },
    {
      "duration": "0.497552000",
      "events": [
        {
          "duration": "0.497493000",
          "events": [
            "POST https://app.fossa.com/api/container/upload"
          ],
          "scope": "Calling FOSSA API",
          "startTime": "2024-06-27 17:22:01.360964 UTC"
        }
      ],
      "scope": "Upload to core analysis service",
      "startTime": "2024-06-27 17:22:01.360938 UTC"
    }
    ],
    "scope": "Upload Container Scan",
    "startTime": "2024-06-27 17:22:00.87403 UTC"
    }

    The CLI should also emit a warning like this: Screenshot 2024-06-27 at 12 26 30 PM

Risks

The testing here is manual for the time being.

Metrics

References

ANE-1711

Checklist