adammontville / cis-controls-71-measures

0 stars 0 forks source link

Subcontrol 2.2 #54

Open adammontville opened 5 years ago

adammontville commented 5 years ago

Ensure that only software applications or operating systems currently supported by the software's vendor are added to the organization's authorized software inventory. Unsupported software should be tagged as unsupported in the inventory system.

Measures

M1 = # of authorized software
M2 = # of tracked software(tagged as supported or unsupported)
M3 = # of unsupported software in authorized list

Metrics

if M1 = M2,
Quality of authorized software Inventory, Q = (M1-M3) / M1,
otherwise Q = 0
apiperCIS commented 5 years ago

The first sentence of the Sub-Control suggests that unsupported software shouldn't be added to the authorized software list. The second sentence of the Sub-Control suggests unsupported is fine, so long as it's tagged appropriately.

Should we be defining unsupported? Or is that left to the implementing vendor or organization?

apiperCIS commented 5 years ago

Rough Proposal:

Input1) The Authorized Software list with supported/unsupported field for each entry Input2) Access to a database containing supported/unsupported information by product (could be scraping vendor websites?)

If either Input1 or Input2 are not available, this test fails. (I guess this is true of most or all of the proposals, so I probably don't need to restate this?)

Test1) For each piece of authorized software in Input1 that is labeled supported, do a lookup in Input2 to see if it's supported. Is an Input1 with the supported field blank a valid Input1? If so, Test1 would also do look-ups for any blanks. Would there be value in doing look-ups for the ones labeled unsupported?

Measure/Metric: The measures/metrics here depend on one's reading of the Sub-Control. If we read the Sub-Control as saying unsupported is bad, don't do it (the sentence 1 perspective), then a measure/metric could be: M1 = # of items in Input1 that are unsupported (combination of test1 results and those initially marked as unsupported in Input1?) M2 = total # of pieces of authorized software in Input1 Metric: (M2 - M1)/M2

If we read this sub-control as being more about accurate tagging, this could be: M1 = # of pieces of software mislabeled as determined by test1 (unlabeled too?) M2 = total # of pieces of authorized software in Input1 Metric: (M2 - M1)/M2 The UNCC metrics for false positives and false negatives might be useful and are worth considering if we are using this reading of the Sub-Control.

adammontville commented 5 years ago

I think the spirit of the sub-control is to ensure that security-related patches/updates are reasonably expected by the enterprise. Therefore, I think the first reading of the sub-control is more accurate (i.e. unsupported = bad).

I think M1 and M2 look sane. M1 is the count of unsupported software and M2 is the count of authorized software, and what we're doing is finding the percentage of unsupported software in use.

wmunyan commented 5 years ago

Consider adding the measures/metrics for false positives/negatives