search

digital4rensics / Malformity

Malformity is a Maltego project based on the Canari framework for malicious binary and infrastructure research.
116 stars 35 forks source link

Cleanup Entities #24

Closed leadZERO closed 11 years ago

leadZERO commented 11 years ago

Original "entities.mtz" replicated all default Maltego and Casefile entities. This had the effect of overwriting the user's default Maltego (and possibly Casefile) entities when Canari was run. Not a bad thing if they don't differ, but should they be updated before Malformity was updated it could cause issues. Also remove all unused entities from both "entities.mtz" and "entities.py." Also removed a "canari.conf" that seemed to be mistakenly placed in the "etc" directory. This file is handled by canari automatically and should not be placed by each canari package.

Ryan

digital4rensics commented 11 years ago

Thanks for the work Ryan. Can you confirm that you tested both these changes without any issues?

Thanks again!

leadZERO commented 11 years ago

Yes, if you look when you run "canari install-package Malformity" it will overwrite all the default entities (and getting them back is challenging :)). The new entities.mtz solves that and only writes the ones used by current transforms.

If you look at "canari.conf" it even says that it is automatically generated. When you run "install-package" canari updates the current canari.conf with the necessary elements.

Ryan

digital4rensics commented 11 years ago

Cool, completely understand not wanting to overwrite current changes. Also, is it strictly limited to entities in use, or does it include all entities in the ohdae set that was included? Ref: https://github.com/ohdae/Analyst-CaseFile

I would prefer to keep all those included since they're not default entities. Thoughts?

leadZERO commented 11 years ago

It does only include the entities in use. The reason was that the entities from Ohdae aren't attributed to anything other than "analysis." So, when you go to try and find what package they belong to, they only show up as "analysis.Hash." In the pull requested I renamed them to "malformity.hash." So it's easier to tell where they came from.

I only included the ones in use just because it's a decent amount of work to rename them (requires edits to the files in your setting folder, then edits to entities.py). Rather than miss something in all of them, I figured to just merge in the ones in use for now. Others can be brought in lateras needed.

The only downside to renaming them is that it will break any currently saved mtgx files. :-/

I figured that might be better to break a few early and name attribute entities rather than have more confusion later.

Ryan

On Wed, Feb 27, 2013 at 2:53 PM, digital4rensics notifications@github.comwrote:

Cool, completely understand not wanting to overwrite current changes. Also, is it strictly limited to entities in use, or does it include all entities in the ohdae set that was included? Ref: https://github.com/ohdae/Analyst-CaseFile

I would prefer to keep all those included since they're not default entities. Thoughts?

— Reply to this email directly or view it on GitHubhttps://github.com/digital4rensics/Malformity/pull/24#issuecomment-14198751 .

Ryan P Sommers ryans@rpsommers.com

digital4rensics commented 11 years ago

Alright. I have no problem going through and doing all the edits for those ones. I figure it's worth it now rather than potentially forgetting they exist and re-creating them down the road.

I agree about breaking a few graphs early, even though it's more than a "few" on my side :-p

Thanks for all the work!

leadZERO commented 11 years ago

One thing I would watch for if you do is to not overlap with the default entities... I don't recall which, but there were some that I thought might overlap with default entities... The worry that being that if you don't properly inherit from default entities you won't be able to use the default transforms... For instance, if you make your own "ip address" but don't inherit from IPv4Address you won't be able to apply transforms that use IPv4Address inputs.

No worries, I was writing some for the VirusTotal private API and then found your project. Figured I'd just contribute there.

Ryan

On Wed, Feb 27, 2013 at 3:06 PM, digital4rensics notifications@github.comwrote:

Alright. I have no problem going through and doing all the edits for those ones. I figure it's worth it now rather than potentially forgetting they exist and re-creating them down the road.

I agree about breaking a few graphs early, even though it's more than a "few" on my side :-p

Thanks for all the work!

— Reply to this email directly or view it on GitHubhttps://github.com/digital4rensics/Malformity/pull/24#issuecomment-14199612 .

Ryan P Sommers ryans@rpsommers.com

digital4rensics commented 11 years ago

After thinking about it again, decided this all made sense :-D Thanks again.