OpenCTI-Platform / connectors

OpenCTI Connectors
https://www.opencti.io
Apache License 2.0
341 stars 370 forks source link

[CVE] Import associated CPEs when importing CVEs #1949

Open d1zanv opened 3 months ago

d1zanv commented 3 months ago

Use case

When you want to see the CPEs associated with a vulnerability or vice versa.

Current Workaround

There is no workaround.

Proposed Solution

When importing the CVEs datas into OpenCTI, included the configurations field, with the CPEs.

Would you be willing to submit a PR?

I am willing to submit a PR if needed to help, but the last big update of the connector was made by @helene-nguyen. I think she'll be in the best position to modify her code.

Jipegien commented 3 months ago

Hello @d1zanv! You can submit a PR, as we are currently working on other connectors. Feel free to ask any question you may have here :)

iFrozenPhoenix commented 3 months ago

I'm currently working on this functionality. Currently I have it as a pure python script where all vulnerabilities are fetched from octi, then the vulnerability from the nist api is fetched and the cpe names extracted. These cpe names are then looked up in octi for their ids and the relationship is created.

Think I can convert it to an enrichment connector. @Jipegien what do you suggest, converting it to an enrichment connector, or combining the cpe and cve connector and adding this functionality?

Jipegien commented 3 months ago

@iFrozenPhoenix Why don't you fetch all "new" CVEs from NIST, along with the related CPEs and then create the corresponding stix data in OpenCTI. Deduplication mechanism will then take care of the matching in OpenCTI. You don't need to compare the data inside the connector logic. It will cover the use case "I know what Softwares are impacted by CVEs"

The logic you have implemented id more adequate for an enrichment connector, but an enrichement connector works entity by entity.

iFrozenPhoenix commented 3 months ago

Well because there was first an cve connector that just imported the cves and scores, then an additional cpe connector was created that just imported the cpes. From this perspective it was the consequence that if one wants to relate these entities an enrichment connector is needed, or the cpe and cve connectors must be combined to 1. Therefore I asked how it should be implemented, as an enrichment connector or if the existing conectors should be combined.

Jipegien commented 3 months ago

I don't think it is mandatory to collect all the existing CPEs and I also think having a sort of CVE->CPE library from NIST into OpenCTI is more valuable than having enrichment of already imported CVEs. So IMHO, you should focus on improving the current CVE connector for retrieving related CPEs. Retrieval of CPE should be configurable.

Currently, I am not sure about the value of the CPE connector by itself. Why having a Software created in the platform if it is not related to a threat/vuln/IT_infra context.

d1zanv commented 3 months ago

I don't think it is mandatory to collect all the existing CPEs and I also think having a sort of CVE->CPE library from NIST into OpenCTI is more valuable than having enrichment of already imported CVEs. So IMHO, you should focus on improving the current CVE connector for retrieving related CPEs. Retrieval of CPE should be configurable.

Currently, I am not sure about the value of the CPE connector by itself. Why having a Software created in the platform if it is not related to a threat/vuln/IT_infra context.

@Jipegien you are right ! For now, the CPEs doesn't have any value in OpenCTI because they can't be linked to the vulnerabilities. When the CVE connector will be updated to import the related CPEs of a CVE, it will be a great enrichment to the platform.

d1zanv commented 3 months ago

I will let you update the CVE connector @iFrozenPhoenix, if you already have a python script to do it. If you need any help, just let me know !

iFrozenPhoenix commented 3 months ago

Well one use case we use the other cpes is to map them against our infrastructure to have software asset management capabilities. Another one is that the complete dataset also contains the base cpes for the software (without version no). While these use cases are specific to our needs someone might also have additional use cases that we currently have not in mind. Don't know if it makes sense to reduce the available knowledge that someone can use.

While thinking what the functionality that should be provided, it sounds from the characteristics that we want to enrich existing data, either by connecting a cve to a cpe or by creating and relating a cpe to a cve.

So while this functionality can also be built in the cve connector logic (if entity does not exist then create cpe and relate, else only relate) I would prefer to also keep the cpe connector, due to additional use cases that cannot be reflected with this approach.

Jipegien commented 3 months ago

It is a matter of priority. A lot of things are relevant enough to be done, but only one thing can be develop at a time. In my opinion, we should begin by importing the most relevant data, meaning softwares (CPEs) impacted by known vulnerabilities (CVEs), because OpenCTI is about threat management first, not assets management. For the same reason, we cannot prioritize building upon "possible" use cases.

So I agree that it'd be great to have all these things. I am just saying, and it is just my opinion, that we should begin by the most relevant part. That's currently how we prioritize at Filigran. But we are not alone in this :) So if you have an important specific use case implying that importing all CPEs is more relevant, nothing should stop you :)

iFrozenPhoenix commented 3 months ago

@Jipegien I think I don't understand what you mean exactly? @d1zanv has developed the cpe connector and it is published in this repository https://github.com/OpenCTI-Platform/connectors/tree/master/external-import/cpe. I also made a minor improvement and fixed a bug. So we currently have ALL cpes and ALL cves in the platform, because they are imported by two separate connectors. To relate these two entities I wrote a script that creates the relations, because that is the missing part. This is also done, but not published as an enrichment connector here. I asked you if I should publish my script as an Enrichtment Connector or if I should modify the CVE Connector to map the entities. So if you are talking about time, for me it is less time to provide it here for the community as an enrichment connector then looking into the code of the cve connector, implementing it and refactoring code.

d1zanv commented 3 months ago

@Jipegien I think I don't understand what you mean exactly? @d1zanv has developed the cpe connector and it is published in this repository https://github.com/OpenCTI-Platform/connectors/tree/master/external-import/cpe. I also made a minor improvement and fixed a bug. So we currently have ALL cpes and ALL cves in the platform, because they are imported by two separate connectors. To relate these two entities I wrote a script that creates the relations, because that is the missing part. This is also done, but not published as an enrichment connector here. I asked you if I should publish my script as an Enrichtment Connector or if I should modify the CVE Connector to map the entities. So if you are talking about time, for me it is less time to provide it here for the community as an enrichment connector then looking into the code of the cve connector, implementing it and refactoring code.

@iFrozenPhoenix For me, it should be implement in the CVE connector as the data you want to use for your enrichment connector are coming from the exact same source as the CVE connector. It's just that when the CVE connector was developped, there was no way of adding the CPEs to OpenCTI, so the dev had no reason to extract the CPEs from the CVE API. Now that the CPE connector exist, the CVE connector can be updated to use the list of CPEs related to the CVE and created the relations between them.

Jipegien commented 3 months ago

Sorry for the misunderstanding. Do what you think is the most relevant for you. On my side, I don't currently see the value of importing all CPEs in the platform and use a connector to do so. The CPE connector is not deployed in our instances currently, thus I see more value to improve the CVE connector because it is the use case I see as important.

d1zanv commented 3 months ago

Sorry for the misunderstanding. Do what you think is the most relevant for you. On my side, I don't currently see the value of importing all CPEs in the platform and use a connector to do so. The CPE connector is not deployed in our instances currently, thus I see more value to improve the CVE connector because it is the use case I see as important.

@Jipegien If I understand you correctly, you would have just updated the CVE connector, so that when a CPE is find related to a CVE, you create it and add a relation between the 2 ?

Jipegien commented 3 months ago

Yes

iFrozenPhoenix commented 3 months ago

@d1zanv I created a repo under my account with the current status. You can use, copy, modify,... it as needed. https://github.com/iFrozenPhoenix/octi-cve-enrichment Thank you for creating the cpe connector 👍️

iFrozenPhoenix commented 3 months ago

@Jipegien I wanted to come back to this issue, also calmed down, sorry for the language. Do you still think it is valuable to eliminate the CPE connector and integrate into the CVE connector? I ask it again because your company currently focuses on vulnerability management and asset management integration (tenable I think). Further I think it would be valuable to see the lifecycle of a vulnerability (i.e. disclosed, proof of concept, exploited). I wanted to discuss how all these things could be integrated best into the platform. Because I really don't know what would be best. Single Connector with options or multiple connectors. Would be cool to discuss these things and find the best solution that fits all our needs. Regards

iFrozenPhoenix commented 3 months ago

Additions: Currently I add the lifecycle state also via a script with data from explitdb.

Jipegien commented 3 months ago

I still think there is more value to import CPE and Software IF they are Threat related or "Infrastructure" related. In respect with that, i see more value in the CVE connector (importing Software that are actually linked to CVE) and connector like Tanium Asset management (importing Software part of the Infrastructure the user must protect). CPE connector will import every known Softwares (my understanding is it will include Softwares not related to CVE, but perhaps I'm wrong here). I think it will result to a lot of unrelevant objects in the platform (because not related to threats or infrastructures to protect). So, It is not about eliminate CPE connector, it is more a priority matter.

But again, if it is more relevant to integrate CPE connector for your specific use case, I see no harm in developing it. It is just not a priority on our side :)

I also see a lot of value on everything that could help assess how a vulnerability is exploited in the wild (EPSS and so on)

iFrozenPhoenix commented 3 months ago

Ok, then I will open a PR at the weekend for the CPE connector that adds a exploit label at exploited label on exploited vulnerabilities (my current approach). I would make this optional and configurable. Is this in your sense?

Mit freundlichen Grüßen / Kind regards


From: Jean-Philippe Salles @.> Sent: Friday, April 5, 2024 5:21:59 PM To: OpenCTI-Platform/connectors @.> Cc: iFrozenPhoenix @.>; Mention @.> Subject: Re: [OpenCTI-Platform/connectors] [CVE] Import associated CPEs when importing CVEs (Issue #1949)

I still think there is more value to import CPE and Software IF they are Threat related or "Infrastructure" related. In respect with that, i see more value in the CVE connector (importing Software that are actually linked to CVE) and connector like Tanium Asset management (importing Software part of the Infrastructure the user must protect). CPE connector will import every known Softwares (my understanding is it will include Softwares not related to CVE, but perhaps I'm wrong here). I think it will result to a lot of unrelevant objects in the platform (because not related to threats or infrastructures to protect). So, It is not about eliminate CPE connector, it is more a priority matter.

But again, if it is more relevant to integrate CPE connector for your specific use case, I see no harm in developing it. It is just not a priority on our side :)

I also see a lot of value on everything that could help assess how a vulnerability is exploited in the wild (EPSS and so on)

— Reply to this email directly, view it on GitHubhttps://github.com/OpenCTI-Platform/connectors/issues/1949#issuecomment-2040077237, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AITUPUSTMAYHFW4KJR7NBOLY326RPAVCNFSM6AAAAABEZTQ55WVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANBQGA3TOMRTG4. You are receiving this because you were mentioned.Message ID: @.***>

d1zanv commented 2 months ago

Ok, then I will open a PR at the weekend for the CPE connector that adds a exploit label at exploited label on exploited vulnerabilities (my current approach). I would make this optional and configurable. Is this in your sense? Mit freundlichen Grüßen / Kind regards

Do you need my help @iFrozenPhoenix ?

V1D1AN commented 2 weeks ago

Hi the teams, I have try your enrichment script python for make the association between CVE and CPE. I have do the request https://github.com/OpenCTI-Platform/opencti/issues/7200 for the same thing but i didn't see you have already the same idea.

I have try to make a connector-enrichment with the template of OpenCTI. I have always error 404 when i add the NIST_API_KEY

Without NIST_API_KEY , i have some result ok image

Result: image