search

center-for-threat-informed-defense / adversary_emulation_library

An open library of adversary emulation plans designed to empower organizations to test their defenses based on real-world TTPs.
https://ctid.io/adversary-emulation
Apache License 2.0
1.6k stars 292 forks source link

Create dead-links-check.yml #137

Open dcaldr opened 1 year ago

dcaldr commented 1 year ago

CI that checks for dead links as suggested by Issue #60 I used work from https://github.com/lycheeverse/lychee-action has some false positives i.e. www.bitdefender.com as 403 error that I'm not able to fix. but most reported links are really broken. Further additions could be using cache or try to auto-solve links via internet archive as presented: more commandline arguments I could put more time and effort, but as this is my first pull request I'm not sure if it's useful.

cat-alyst commented 1 year ago

Thank you for making a PR request!! 🤩

Links are a tricky thing, for sites we own (MITRE & CTID) it makes sense to check and a great call out. For sites we do not own...we will probably always come up with errors.

Here is the reason, vendors (the main supplier of reports) can and do remove published reports 💔 . Annoying but since it's their report it's also their right. It's not uncommon for us to be using a report during development and suddenly find the report 💨 gone 😿 . Our work around ❤️‍🩹 has been to download reports earmarked as useful so we do not rely on the online version. This way if anyone has questions regarding citations, we can promptly provide the documentation even if the links are broken 🔗 . However GitHub is not the best place for document storage. So we don't upload those here.

Any thoughts on other solutions? I haven't looked too deep in this project yet but it's now on my docket. If there is a way to ignore some links while verifying others, that would be helpful. This is also a good call out for a documentation update. Thank you! 🙏

dcaldr commented 12 months ago

I did some trial and error testing on the tool. The tool can also suggest for the dead links their saved version in wayback machine (Internet Archive)it is way slower but can get the job done. Here is example of summary report from my testing (hopefully) summary-106err Ignoring links - can be done via arguments, with special "config file" or allowing http error codes as good -> what I have right now. I will try to explain what it does now: