search

SAP / risk-explorer-for-software-supply-chains

A taxonomy of attacks on software supply chains in the form of an attack tree, based on and linked to numerous real-world incidents and other resources. The taxonomy as well as related safeguards can be explored using an interactive visualization tool.
https://sap.github.io/risk-explorer-for-software-supply-chains/
Apache License 2.0
71 stars 14 forks source link

[New Attack Vector] AI Package Hallucination #88

Closed piergiorgioladisa closed 1 year ago

piergiorgioladisa commented 1 year ago

Probably you've heard about the AI Package Hallucination strategy. In a nutshell, as per my understanding, LLMs can "invent" names of packages when users ask to generate some code. Thus, attackers may deploy malicious packages under such names.

I personally think this could be included as a Create Name Confusion with Legitimate Package technique. But before creating a PR to add this new AV, I would like to hear your opinion @henrikplate @serenaponta

henrikplate commented 1 year ago

I think it is a relevant new technique, and since the hallucinated package name probably repeats the name of some technology the user asked about (as in the case of arangodb), it's fair to add it below "Create Name Confusion".

piergiorgioladisa commented 1 year ago

Great! Then I will create a PR and of course feel free to adjust it if you find some inconsistency or mistake