search

li-zhenyuan / Knowledge-enhanced-Attack-Graph

AttacKG: Constructing Knowledge-enhanced Attack Graphs from Cyber Threat Intelligence Reports
MIT License
139 stars 38 forks source link

Failed to install requirements #3

Closed deg closed 2 years ago

deg commented 2 years ago

Does this need a particular version of Python, or any other setup? pip install -r requirements.txt fails in Python 3.10.4 and several other versions I tried.

Also, it looks like the README.md file may be out-of-date. The instructions suggest ... -R ./reports_sample/Log4Shell.html, but neither that file nor directory seem to be included in this repo.

li-zhenyuan commented 2 years ago

I use Python 3.8 in this project. The missing dataset is added. And the Readme is updated. Please try it again.

deg commented 2 years ago

Thank you for your rapid response! The install made it a lot further now, but it still failed with the following conflict:

ERROR: Cannot install -r requirements.txt (line 13), -r requirements.txt (line 4) and numpy==1.22.3 because these package versions have conflicting dependencies.

The conflict is caused by:
    The user requested numpy==1.22.3
    blis 0.7.6 depends on numpy>=1.15.0
    coreferee 1.0.1 depends on numpy~=1.19.2
deg commented 2 years ago

I tried to work around that problem by requesting coreferee==1.2.0 instead. This let me finish installing all the requirements.

But, when I then tried

python main.py -M attackGraphGeneration -R "./Dataset/Evaluation/Frankenstein Campaign.txt" -O ./output.pdf

I got the error

ImportError: cannot import name '_imaging' from 'PIL'
deg commented 2 years ago

Ok, I got around that by upgrading Pillow to 9.2.0.

But now, I'm getting the error:

Can't find model './new_cti.model'
li-zhenyuan commented 2 years ago

You can find the model in the google drive link mentioned in Readme.

li-zhenyuan commented 2 years ago

The problem with the requirement installation seems to be a generic one, which I have encountered before. I'll upload a working virtual environment later.

deg commented 2 years ago

I'm making progress, but not quite there. So far, I've updated requirements.txt per above, and copied in and unzipped the three zip files (the model directory, templates and technique_template. (It would be helpful if you could add a line to the readme saying that those three files need to be copied in and unzipped. As is, you just offer the clue of the links to their existence).

I can now run the first command and it generates a .pdf. But the results look very incorrect output.pdf

deg commented 2 years ago

And, when I try the second command:

python main.py -M techniqueIdentification -T ./templates -R "./Dataset/Evaluation/Frankenstein Campaign.txt" -O ./output.pdf

the console shows INFO:root:---Done!--- and a output.pdf_techniques.json file is generated, but there is no output.pdf

deg commented 2 years ago

FWIW, here are the generated json files for the two commands: jsons.zip

deg commented 2 years ago

Any thoughts on why the blank and missing pdf outputs? Can I do anything to help debug?

li-zhenyuan commented 2 years ago

Any thoughts on why the blank and missing pdf outputs? Can I do anything to help debug?

Hi Deg, thanks for offering help. The project here have two versions. The first one is in the directory Archive-0.1. This version has more complete functions and better parameter adjustment. But, as you may see it is not well-organized. Also graphviz-based visualization used in the first version has some issues that cannot be described with overlapping attack techniques. Thus I create a new version (in the root directory). This version has more readable features but has not been carefully tweaked. That is why there are more problem. I think I will fix them when I have time. You may help with this. For the specific question why miss pdf output. Maybe you can check if the .gml file is generated. If so, maybe you are lack of graphviz.

deg commented 2 years ago

Hmm. You are correct that I did not have graphviz installed. I was running on a fresh Ubuntu VM.

So I have installed it now, and tried again. But I still see the same results:

python main.py -M attackGraphGeneration -R "./Dataset/Evaluation/Frankenstein Campaign.txt" -O ./output.pdf generates a 23kb .pdf - a blank page with some lines but no text. This time it is slightly different, so I've attached it again: output.pdf

python main.py -M techniqueIdentification -T ./templates -R "./Dataset/Evaluation/Frankenstein Campaign.txt" -O ./output.pdf does not generate any pdf

Neither appears to have generated any .gml files. The only .gml files I see are the ones that came from the git repo.

li-zhenyuan commented 2 years ago

I don't understand what do you mean by "no text". Maybe you can zoom in the image? By the way, the graph is disconnected because the threshold for graph merge is not optimized. And the techniqueIdentification does not generate image for now.

li-zhenyuan commented 2 years ago

Also I think it is a new issue. So I am closing this one.

deg commented 2 years ago

Aha. I've zoomed way in now and see that there is text by each of the disconnected graph components. My apologies for missing this.

I'm happy to open a new issue if you'd like. But, I'm not sure how you want to divide this issue.

From my perspective, the single problem is a lack of details in the README.md. You've already addressed part of this (thank you!), but some bit still remain unclear:

Remaining open issues: