Revise Methodology

[neil-unomaha]

Awaiting Dr. Hale's feedback from milestone one. Dr. Hale did mention our methodology requires some work.

[skyemakable]

Feedback from Dr. Hale on Methodology:

• Too little details outside of "read documentation, setup testbed, and integrate"
• Little idea over what the entire project will entail
• Technical plan
  • Too short
  • Did not include information such as prototyping, testing, and iterating solution
  • Did not include details on testing any deployment infrastructure, measuring performance, etc.
  • Does not address research questions

Ideas for how to improve methodology

• Verify confidence on the research questions
• Outline methodology to help answer research questions
• Dive into risk list to integrate into methodology
  • How will we handle the risks if they come up?
  • Also look into expanding risk list to involve items such as compatibility, format, performance, etc. of CIF and PA
• Run CIF in testbed to see how it runs at the time being
  • Look at script that is run by cron job to gather information
  • Look at records generated by PA and what records are sent to CIF after running the cron job.
• Research other potential applications that use PA API
  • Check forums from PA of other customers who have use the API and see what questions they normally ask. Does anything related to compatibility, format, or performance come up?

[neil-unomaha]

Methodology Details to Discuss with Professor Hale

Developing code in the context of a capstone is a bit tricky. Two common software development approaches are to use the Waterfall Model or the Agile model. The main difference between the two:

• Waterfall approach: you deploy the entire, finished product all at once.
• Agile approach: you deploy small, iterative but complete features of code one at a time which move towards the final product.

The end result is the same for both. Agile is a common approach due to project management purposes. One large benefit of the agile approach is that users get to experience the features even before the final project, or the final major version, is complete.

Methodology

The methodology at a high level for our capstone project can, oddly enough, be somewhat modeled after a kickstarter campaign. So: it is somewhat a hybrid between the agile and waterfall model. We will have our main "goal" and then "stretch goals" beyond that.

Main Goal - Get an ad hoc solution up and running

We want to develop middleware that meets the needs of this specific environment:

• UNO's Palo Alto interfacing with their CIF server (or REN-ISAC CIF server, whichever way UNO has it setup)

Initial Steps: Acquisition, Configuration, Setup

• Acquire sufficient knowledge of CIF to be able to push/pull threat feed
• Acquire sufficient knowledge of Palo Alto Reports (identify appropriate dynamic report, or create custom report)
• Acquire sufficient knowledge of Palo Alto API to pull report
• Acquire sufficient knowledge of Palo Alto API to push new policies.
• Ensure configuration exists to access Palo Alto API.
• Set up test environment with both CIF and Palo Alto

Develop middleware

• CIF is written in bash and python so for consistency: it probably makes the most sense to stick with these languages.
• The two "workflows" to be developed are broken down in this comment of #22

Stretch Goal 1: Refactor for performance

The "main goal" is to just get a solution working. The code solution is likely messy and has room for improvement in the area of performance. In this stretch goal, we could likely go through many iterations in hopes of discovering the ideal solution, but due to time constraints we will need to enforce a deadline if we hope to get to the next stretch goal.

• Brain storm and research strategies of developing more efficient middleware code
• Perform A/B testing with same data inputs to determine which version is most efficient. Perform multiple tests of all versions and take the average

Stretch Goal 2: Abstract out "middleware design pattern" from the existing ad hoc design

The middleware is ultimately solving this problem: creating a bridge between two endpoints. In our scenario the middleware happens to be bridging a next gen firewall and a CIF server, but these two endpoints are ultimately irrelevant. A good middleware program would be accommodating to various endpoints.

It would be nice to abstract out this "middleware pattern" so that the middleware is no longer an ad hoc design. However, we don't want to abstract out too much or else the middleware would end up not providing any real value.

Key components

End Point 1 And End Point 2 "Pulling" Configuration

• Calling an API or does the data already exist in a file somewhere?
  • If API: write the API Call into the configuration file
  • If data already exists in from a file: specify path to file
  • Access to data if it exists in a database (outside of the scope of the capstone)

Endpoint 1 and Endpoint 2 Data Formatting

• Likely some sort of string manipulation strategy, like sed

Endpoint 1 and Endpoint 2 "Pushing" Configuration

[neil-unomaha]

Methodology Revision: Rough Sketch

Note: Brian mentioned that most Universities in this collective intelligence network only push every 24hrs. However, at least one University, Duke, pushes more often. This provides us the opportunity to perform before/after measurements as described below.

Baseline Measurement:

• Measure how many new threats are ingested into Palo Alto currently within a 24hr period. (The current process in place only pulls in new threats once every 24 hours).

Implement our solution

• Build an extension into CIF 3 and/or CIF 4 Python library which does the following:
  • Pulls in IP indicators into Palo Alto in ingestible format. Saves to file. No more than 5,000 indicators per file referencing page 60 of Palo Alto API.
  • Every 15 minutes: CIF pushes generated files to Palo Alto via the Palo Alto API. API section on Importing files.

Conduct Post-Solution Measurement and Compare

• Measure temporal threat ingestion differences.
  • Example: a threat reported in 24 hrs (before) vs a threat reported at 1am (23hrs savings)
  • Example: a threat reported in 24hrs (before) vs a threat reported at 5am (19hrs savings).

---

Technical breakdown of Implementing our solution:

In order to arrive at our solution, it is necessary to break it down. Here are a list of activities that will need to be conducted in order to arrive at our solution:

• Familiarize ourselves with Python
  • CIF is written in Python.
• Setup a test environment for CIF
  • Bryan will set us up with an environment that mirrors production and has CIF 3 preloaded.
• Familiarize ourselves with CIF at the user-interface level
  • We will want to familiarize ourselves with the major commands and features of CIF. Examples include: querying CIf database, adding/removing feeds
• Gain a sufficient level of understanding of CIF at the code-level
  • Like most software applications, CIF is a an aggregate application which is composed of a number of other libraries all working together. In order to extend CIF, it is necessary to gain sufficient knowledge of the CIF ecosystem. The requirements.txt file is a good starting place. Here we notice that CIF is modularized into at least 5 CIF-related libraries, and has additional dependencies, and also possesses dependencies on other libraries. Flask is one particular library that jumps out at us because that is a web application framework, so we will likely want to familiarize ourselves with Flask to see how that library might apply to developing our solution.
• Create the solution
• Test the solution
  • Ensure our extension is properly creating Palo Alto ingestible files of IP indicators within 15 minute or less intervals.
  • Ensure the files do not exceeed 5,000 IP indicators
  • Ensure our extension pushes the indicators to Palo Alto, and Palo Alto successfully ingests the indicators.
• Deploy the solution to production
• Submit pull request

[neil-unomaha]

8c7f81a3349a873d0dddc6a8cff067f59bb37c7d