GSoC Idea: Develop a Shared Data Resource Focused on Dependencies, Risk and Vulnerabilities in Open Source Software

[sgoggins]

The aim of this work is to understand the code based dependencies embedded within a piece of open source software. This metric explicitly excludes infrastructure focused dependencies like databases, and operating systems, which are defined in the “upstream infrastructure dependencies” metric.

Objectives

• This software project is aimed at understanding the language level, non-infrastructure packages, and other software which are required to run a piece of software in build, test, and runtime environments.
• What libraries or versions does my project explicitly depend on?
• What libraries or versions does my project implicitly depend on?
• Consolidate dependency information from the range of projects focused in this area.
• Provide a shared, open source software package for consolidating a subset of available resources (we don't expect one student to finish this entire project in a summer)
• Integrate this information on the CHAOSS Website

Implementation The expectation is that this would be implemented by using existing tools that examine package manager data for the languages in use (e.g., package.json for JavaScript npm, pyproject.toml / requirements.txt for Python, Gemfile / Gemfile.lock for Ruby). Ergo, dependencies will be analyzed using the project’s dependency file. This will be analyzed using dependency file in the project.

Note: C/C++ generally do not use system package managers. Things get more complex with multiple languages, insofar as several language specific dependency files will need to be scanned.

Micro-tasks and place for questions [will add link later]

Augur would be the tool that this is ultimately implemented in, although only as an accessed, shared data resources including informaiton form other tools, including:

Resource Link

• Scorecard https://github.com/ossf/scorecard
• Dependency Check https://owasp.org/www-project-dependency-check/
• Proactive Error Detection in Software https://github.com/google/oss-fuzz
• High Severity Vulnerability Detection https://github.com/google/tsunami-security-scanner
• Kubernetes focused supply chain security https://github.com/grafeas/kritis
• Verification from source to binary https://reproducible-builds.org/
• Dependency Check (Relies on Common Platform Enumeration (CPE) https://owasp.org/www-project-dependency-check/
• Securing Critical Projects OSSF Working Group https://docs.google.com/document/d/1MIXxadtWsaROpFcJnBtYnQPoyzTCIDhd0IGV8PIV0mQ/edit
• Preventing Supply Chain Attacks https://www.linuxfoundation.org/en/blog/preventing-supply-chain-attacks-like-solarwinds/
• Security Threats OSSF Working Group
• Best Practices OSSF Working Group
• National Vulnerabilities Database https://nvd.nist.gov/vuln/full-listing/2021/1
• https://nvd.nist.gov/vuln/data-feeds#JSON_FEED
• Libyears https://github.com/nasirhjafri/libyear & https://github.com/sesh/piprot
• Census II https://drive.google.com/file/d/1zyAdbftGhSUiddh1she3X_MDlKXDSIu5/view?usp=sharing
• 2021 State of Open Source Vulnerabilities https://drive.google.com/file/d/1BwJD3eqynwSms5b9WxzzHrzp-YRXMbLv/view?usp=sharing

• Difficulty: Medium
• Requirements: Interest in software analytics. Python programming. JavaScript programming. SQL knowledge.
• Recommended: Experience with Python
• Mentors: Sean Goggins, CHAOSS Risk Working Group, Vinod Ahuja

Microtasks

For becoming familiar with Augur, you can start by reading some documentation. You can find useful information at in the links, below.

Once you're familiar with Augur, you can have a look at the following microtasks.

• Microtask 0:
  Download and configure Augur, creating a dev environment using the general cautions noted here: https://oss-augur.readthedocs.io/en/dev/getting-started/installation.html and the full documentation here: https://oss-augur.readthedocs.io/en/dev/development-guide/toc.html

• Microstask 1: Work on any Augur Issue that's Open

• Microtask 2: Identify new issues you encounter during installation.

• Microstask 3: Build a jupyter notebook to identify software dependencies, or build software dependency metrics using one of the identified tools.

• Microtask 4: Anything you want to show us. Even if you find bugs in our documentation and want to issue a PR for those!

[Dhruv-Sachdev1313]

Hello @sgoggins I have a little doubt regarding this project's objective. So is this project about developing a software that identifies dependencies and vulnerabilities? Or using the existing tools to develop a database for the same ?If you can explain it a little that would be really helpful.

[Dhruv-Sachdev1313]

hello @sgoggins Does here implicit dependencies mean the dependencies not specified directly in package manager data ? So how do we get to know about those dependencies.

[sgoggins]

@Dhruv-Sachdev1313 : Simply put, there is more here than we can do in a summer. We will be guided by the Risk working group on what we choose. I think playing with the Libyear implementations against some augur data is a really good start!

[sgoggins]

There will be some design discussion at the start of this task for sure!!

[sgoggins]

This spreadsheet of the Risk working group's minimum viable metrics are a good place to start to understand what we will build first. https://docs.google.com/spreadsheets/d/1hNCtgHkA3uwB4OrUS0gXkXpOwWsJMbIBMzek3j_E3mw/edit#gid=855606942

[ADI10HERO]

@sgoggins Is it like creating GitHub's Dependabot? And, if it is, what differences are we looking for, like why not just use it? I hope its not too stupid a question 😅

[sgoggins]

@ADI10HERO I think any implementation of any small part of a tool that evaluates risk is going to be a sufficient. On this spreadsheet are a list of other tools to explore:

https://docs.google.com/spreadsheets/d/1hNCtgHkA3uwB4OrUS0gXkXpOwWsJMbIBMzek3j_E3mw/edit#gid=855606942

Risk working group information is located here: https://github.com/chaoss/wg-risk

[sgoggins]

I am leaving this issue open for the convenience of potential GSoC Students for the time being.