search

libyal / libevt

Library and tools to access the Windows Event Log (EVT) format
GNU Lesser General Public License v3.0
57 stars 25 forks source link

Incorrect and misleading security advisories CVE-2018-8754 and DSA-4160 #5

Closed joachimmetz closed 5 months ago

joachimmetz commented 6 years ago

Incorrect and misleading security advisories

Recently I was made aware of CVE-2018-8754 and DSA-4160.

First of all I was surprised to see these "Security Advisories" (quotation intended) seeing neither Mitre (who are responsible for issuing CVEs) nor Debian Security had reached out me. Seeing I’m the maintainer of libevt.

First some context

Libevt clearly indicates it has alpha status and HEAD, which is work in progress. So it will likely contain bugs.

See Wikipedia for an explanation of alpha: https://en.wikipedia.org/wiki/Software_release_life_cycle#Alpha

You cannot expect normal (open source) development if every pre-release or development version is scrutinized as stable software. It will take time and effort to get to stable and secure.

Lack of due diligence

Neither Mitre nor Debian Security did reach out to me, as the project maintainer, before they made their "advisories" (quotation intended).

Until to date neither Mitre nor Debian Security has not answered me these questions:

Where the answer to the last questions seems no.

Mitre and NVD and their arbitrary CVE process

The status of CVE-2018-8754 initially read:

This vulnerability is currently awaiting analysis.

How can you post an advisory if have not done your analysis?

Now it says:

Additional Information: 
Allows unauthorized disclosure of information
Allows unauthorized modification
Allows disruption of service

Until date I have not seen any proof for the first 2 claims.

For the 3rd one you would have to run libevt as part of a service without having taken additional measures like sand-boxing.

To improve security it is important to get facts straight and not have this arbitrary process.

Per Mitre the vulnerability definition we currently use is:

  a weakness in the computational logic (e.g., code) found in software
  and hardware components that, when exploited, results in a negative
  impact to confidentiality, integrity, OR availability.

  http://cve.mitre.org/cve/cna/rules.html#Appendix_A

The “OR availability” part is important here, for the layman, this translate to any bug
that results in a segfault or an infinite loop.
Our current practice is to interpret "availability" in a certain way
if a software package is a library that can be used within arbitrary
applications. Specifically, any instance of an application crash is
considered an availability impact. The rationale is that somebody
could use libevt as part of a long-running daemon that accepts Event
Log data from throughout an enterprise network, and would crash with
an out-of-bounds read. In other words, one client could cause a denial
of service for all other users of this central log service. We are not
saying that such a log service has been built, only that this is a
realistic use case.

By contrast, if you wrote a command-line program (not in the form of a
library) for Event Log data and it crashed with an out-of-bounds read,
we would NOT assign a CVE ID.

We feel that we are maintaining this distinction consistently across
software packages from different maintainers.

https://cwe.mitre.org/about/faq.html#A.2 (which you mentioned) is the
documentation of the CWE project, not of CVE. We do know the people on
the CWE team, however, and can ask them to adjust that definition.

As you can read from their response Mitre applies arbitrariness within their over complicated definitions (that are also not aligned with the CWE). Also until date Mitre has not provided any evidence of their claims after numerous requests to do so.

Additional information from NVD.

The NVD uses the publicly available information to associate CVSS metric values
to CVE entries. When information is unclear, or there are conflicting points from
sources the NVD takes the position of representing the worst-case scenario until
further details can be provided to justify a modification to the score. Currently, the
information presented to the analyst does not provide enough detail to ensure that
the impacts are explicitly limited.

NVD basically says we have no information so we assume the worst based on no facts without any transparency.

The NVD does not participate in the vulnerability disclosure or the CVE publication
process. The CVSS metrics offered by the NVD represent the severity of a given
CVE based on the information that is publicly available.

Let's review https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2018-8754 and https://nvd.nist.gov/vuln/detail/CVE-2018-8754. There are details in the NVD that are not in the CVE such as "Allows unauthorized disclosure of information", "Allows unauthorized modification", "Attack Vector (AV): Network",

Also the NVD references https://www.debian.org/security/2018/dsa-4160, which only mentions "denial of service".

Where does the NVD get this information? After having repeatedly asked for the proof NVD has not provided me with any.

The information provided by the NVD has always represented the stance of what
the highest severity score would be when there is no information available to clarify
limitations to exploitability or impact as represented by CVSS metrics or when
information conflicts.

I read this as instead of providing an accurate and transparent advisory the NVD will knowingly make false claims (make up stuff) when there is no information. This sounds a lot like slander ("to make false and damaging statements") or defamation to me.

Thank you Mitre CVE and Nist NVD for having such a "responsible disclosure" process. (quotation intended)

Bureaucracy at its best

Nist NVD pointing to Mitre CVE

Unfortunately, the NVD does not have control over the CVE Description, CVE status
(Published, Disputed, Rejected) or references associated to the CVE entry as these
are maintained by the CVE Assignment Team. To dispute the legitimacy of a CVE or
to request modifications to the description or references, you will need to contact the
CVE Assignment Team.

Mitre CVE pointing to Nist NVD

The additional information and the nvd.nist.gov site are handled by the NIST NVD team.
You can contact the NIST NVD team at nvd@nist.gov for questions or concerns with this
impact data specifically.

Rectification (at last)

On July 11, 2018 NIST NVD removed the speculative claims from https://nvd.nist.gov/vuln/detail/CVE-2018-8754. Nothing in the NVD entry indicates that the advisory has been updated.

Also until date neither Mitre CVE or NIST NVD has apologized for putting their speculative claims out in the world in the first place, nor have they presented an assessment on what they plan to change going forward.

Untruthful claims in DSA-4160

I was also surprised and saddened that Debian Security had did not done their due diligence. From their initial DSA-4160 report:

It was discovered that insufficient input sanitising in libevt, a library to access the
Windows Event Log (EVT) format, could result in denial of service or the execution
of arbitrary code if a malformed EVT file is processed.

I've reached out to Debian Security seeing their website offers no direct feedback link. They replied after roughly 1 week after issuing DSA-4160.

We seem to have mis-triaged this, then. Sorry for that, I'll add a note to the entry
in the Debian Security Tracker.

It took about a month to get the DSA-4160 posting updated. There is no public evidence of the posting being changed, nor any statistics on how often this happens.

More hearsay

Some more hearsay I found by vulnerability exchange platforms that not even try to bother to keep up to date with their upstream

From: https://exchange.xforce.ibmcloud.com/vulnerabilities/140473

libevt-cve20188754-code-exec (140473)   reported Mar 17, 2018

Libevt could allow a remote attacker to execute arbitrary code on the system, caused
by the failure to properly check for out-of-bounds values of user SID data size, strings
size, or data size by the libevt_record_values_read_event() function. An attacker could
exploit this vulnerability to execute arbitrary code on the system.

From: https://packetstormsecurity.com/files/cve/CVE-2018-8754

Debian Linux Security Advisory 4160-1 - It was discovered that insufficient input
sanitising in libevt, a library to access the Windows Event Log (EVT) format, could
result in denial of service or the execution of arbitrary code if a malformed EVT file
is processed.

Even more disappointing the ability of the site to comment does not appear to be working. Until date packetstormsecurity has not responded or acted on any feedback provided directly to them.

Post-mortem

Who can I send the bill for all the time, effort and energy spent on this?

Mitre CVE and Nist NVD it is very nice of you want the software developers to meet your standards, but when are you going to self-impose quality standards to your own work?

More "security advisory" incompetence

Another update September 12, 2018

An update from Mitre CVE on August 24, 108, 5 months after the "advisory" went public:

Regarding your CVE service request, logged on 2018-06-25T15:07:03, we have the
following question or update:

We feel that the information currently published on cve.mitre.org web pages, in
conjunction with the information in the linked references, is reasonably usable by
consumers who have installed the affected software libraries, and allows them to
make initial risk assessments and/or better understand the code quality. However,
we also understand your concerns and feel they are valid enough to warrant marking
the items as DISPUTED. We will make this change to all of the applicable CVE entries
with the next 5 business days.

Please do not hesitate to contact the CVE Team by replying to this email if you have any
questions, or to provide more details. Please do not change the subject line, which allows
us to effectively track your request. Regards, CVE Assignment TeamM/S M300, 202
Burlington Road, Bedford, MA 01730 USA[A PGP key is available for encrypted
communications athttp://cve.mitre.org/cve/request_id.html]
--

First of all finally some more transparency. However Mitre CVE, please add the CVE number(s) you are referencing to your responses. From the email I have no idea which CVE number(s) you are referring to. So looking through all the CVE numbers I know about I checked CVE-2018-8754 and it now states:

** DISPUTED ** The libevt_record_values_read_event() function in libevt_record_values.c
in libevt before 2018-03-17 does not properly check for out-of-bounds values of user SID
data size, strings size, or data size. NOTE: the vendor has disputed this as described in
libyal/libevt issue 5 on GitHub.

So this is a misrepresentation of the dispute I have with the CVE. I do not dispute there is a bug, I dispute that this is worth a CVE number. For reasons mentioned before. Honestly this only confirms the complete arbitrariness of the CVE program.

Where are the links to the assessment done by Mitre CVE? Or does Mitre CVE do no assessment at all? Then where is the link to the assessment by the reporter? Oh wait, did the reporter just provided the output created by the fuzzing tool?

Please do not hesitate to contact the CVE Team by replying to this email if you have any questions, or to provide more details.

I did (multiple times), and what do you expect no answer so far.

Another update August, 2021

Mitre still has not made any visible improvements they now even confirm that they do not do any due diligence. They leave it up to the reporter to reach out to the "vendor" and there is no validation that this happened. Also see: https://github.com/libyal/libexe/issues/1#issuecomment-904888577

If you cannot guarantee software issues will be fixed then why bother? Stop wasting everyone else their time so people can focus their time and energy on efforts that really make software more secure

joachimmetz commented 6 years ago

Why does this matter:

joachimmetz commented 5 years ago

More data points that CVE and CVSS scoring is not helping to improve security and that Mitre CVE and NIST NVD are clearly not up to the task of providing reliable security advisories.

A recent academic study[1] found that only 57% of security questions with regards
to CVE vulnerability scoring presented to participants in the study were accurately
answered. The study further elaborates on which information cues are crucial for
better accuracy, as well as which would actually drive confusion and lower
accuracy in scoring.

It is not unusual to find false positive in a CVE or inaccuracies in scores assigned
to any of the metrics groups, which introduces a risk of losing trust in a CVE or
creating panic for organizations which is uncalled for.
Another study[2] from Carnegie Mellon University reported similar findings with
regards to the accuracy of scoring. The report states: “More than half of survey
respondents are not consistently within an accuracy range of four CVSS points”
where 4 points alone moves the needle from a high or critical severity to lower
levels.
[1] Allodi, Luca & Banescu, Sebastian & Femmer, Henning & Beckers, Kristian.
     (2018). Identifying Relevant Information Cues for Vulnerability Assessment
     Using CVSS. 119-126. 10.1145/3176258.3176340.
[2] Spring, J.M, Hatleback, E. Householder, A. Manion, A. Shick, D. “TOWARDS
     IMPROVING CVSS” Carnegie Mellon University, December 2018
VideoLAN blamed a reporter for running VLC on an old version of Ubuntu with out-of-date
libraries, and security firm MITRE for issuing a CVE before it could examine the
reporter's claims
NIST NVD has rated this issue incorrectly at CVSS v3 base score of 9.8 and "Network"
exploitable. This evaluation is not correct, the issue is only local exploitable, which
gets it a CVSS v3 base score of 7.8.
CVE stands for Common Vulnerability and Exposures and is scored using the CVSS
(Common Vulnerability Scoring System) standard. This standard is a bit complicated
to grasp at first, and (on the surface) seems a bit arbitrary. 
The CVE scoring method is complicated, so much so that the average user shouldn't
even bother trying to calculate scores for vulnerabilities.
joachimmetz commented 5 years ago

Things that would make the CVE better suited for security advisories (at least from an Open Source perspective):

Things Mitre CVE could be doing to reduce their error rate:

Things NIST NVD could be doing to reduce their CVSS scoring error rate: