search

alexandreborges / malwoverview

Malwoverview is a first response tool used for threat hunting and offers intel information from Virus Total, Hybrid Analysis, URLHaus, Polyswarm, Malshare, Alien Vault, Malpedia, Malware Bazaar, ThreatFox, Triage, InQuest and it is able to scan Android devices against VT.
https://github.com/alexandreborges/malwoverview
GNU General Public License v3.0
2.85k stars 439 forks source link

Update tool to prepare for pip install, use config file. #9

Closed digitalsleuth closed 3 years ago

digitalsleuth commented 3 years ago

First, great tool! I was recently informed about this tool when someone recommended we add it to the REMnux tool suite. In order to do this, I've had to make some changes, which I'd like to merge with your repo.

The updates made will allow for a modular approach to modify API keys based on a specific config file, automatically copied to the users home directory. This means the user will not have to go to the /usr/lib .. directory to change/modify the API key. Update will also allow a user to specify a config file of their choice.

This update will also work on Windows inherently by determining the running OS and importing/installing the required modules for the OS. Removed some standalone requirements as well (configmalw.py and magic.py) in favor of installable modules and the config file modification.

alexandreborges commented 3 years ago

First, great tool! I was recently informed about this tool when someone recommended we add it to the REMnux tool suite. In order to do this, I've had to make some changes, which I'd like to merge with your repo.

The updates made will allow for a modular approach to modify API keys based on a specific config file, automatically copied to the users home directory. This means the user will not have to go to the /usr/lib .. directory to change/modify the API key. Update will also allow a user to specify a config file of their choice.

This update will also work on Windows inherently by determining the running OS and importing/installing the required modules for the OS. Removed some standalone requirements as well (configmalw.py and magic.py) in favor of installable modules and the config file modification.

Dear @digitalsleuth

Good afternoon. I've reviewed the purposed changes and all of them are great contributions. I've already updated the README.md, but I've just found several points to be considered:

  1. Eventually, not all distribution has USER_HOME_DIR (for example, Kali Linux doesn't have), but they have HOME. I didn't changed it yet because I don't want to broken the REMnux pattern.
  2. I've changed the two packages' version because Polyswarm has its package (2.1.2), but this new version doesn't work with this version of Malwoverview. Thus, I've fixed both "requests" and "polyswarm-api" versions for while:
  1. I"ve added the -c version (introduced by you) in the short help.
  2. I"ve added the requirements.txt again in the Github version because many people continue installing the Malwoverview from Github. Of course, this file not exists in the pip version.
  3. I couldn't test it on Windows and I need to verify the .malwapi.conf issue there.
  4. I've changed the version to 3.1.2 because these few and small changes.
  5. I've added your name as a formal contributor in the README.md
  6. The PIP package has already been created: pip3.8 install malwoverview

Your contribution has been great.

In the next two weeks I'm releasing the new version (4.0) of Malwoverview, which it will include many new features, changes and fixes. Thus, I'll change my new version for reflecting your suggested changes for 3.1.2 version.

Thank you for your time and really great contributions.

Have an excellent day.

Alexandre.

digitalsleuth commented 3 years ago

Hi Alexandre,

Thanks for the feedback, and I'm glad to have helped. The only reason I deleted the requirements.txt was because the requirements are in the setup.py, and the tool can then be installed using pip3.8 install git+https://github.com/alexandreborges/malwoverview (when online) and pip3.8 install . (when done offline). The setup.py will then pull the requirements based on the OS.

I was able to successfully install and configure malwoverview on Windows using the method I had used in the script, and it worked with, and created, the .malwapi.conf file in the C:\Users directory. I'm unsure what the issue was, but hopefully you've managed to correct it. If not, I'd be glad to help there as well.

I look forward to the next version, and thanks again!

Cheers

Corey

digitalsleuth commented 3 years ago

I just realized I had forgotten to mention one thing about the USER_HOME_DIR. I assigned that variable to the pathlib Path.home() function to identify the current users home directory. While Kali, and many other OS's don't have a variable called 'USER_HOME_DIR', that's only what I opted to call it in the setup.py. They do, however, provide a valid response to str(Path.home()). In the case of Kali, if you're running as root, it returns '/root', otherwise it will return '/home/'.

I hope that helps clarify my reasoning for using it.

Corey

alexandreborges commented 3 years ago

Corey,

Thank you for your replies.

I think the pending issues were cleared, mainly about USER_HOME_DIR. I've just updated the README.md once more.

Please, let me know when the Malwoverview will be added to REMnux.

I hope you like the next new 4.0 version.

Have an excellent day.

Cheers,

Alexandre.

digitalsleuth commented 3 years ago

Hi Alexandre,

We've already added malwoverview to REMnux (our initial release candidate was announced on Monday). It's currently listed on the docs page for REMnux here.

Currently, it's using the version from my github repo. We only opted to use the modified version from my repo to resolve a few installation and configuration issues we encountered, which were resolved by some of the changes made. Once you've updated to 4.0 (since you'll be making some substantial changes soon), we'll move the installation to pull from pip and use yours here on in (following some testing, of course, to ensure there are no issues with the installation and conflict with other tools in the suite).

Please feel free to reach out if you have any questions or concerns!

And again, thank you for your work and your contribution to the open source community!

Corey

alexandreborges commented 3 years ago

Corey,

Good morning. Thank you for the information.

I'd like to ask you to reference the malwoverview website as soon you can because your changes have been merged into the master branch and the README.md has been updated to reflect the new version installation.

I'll take more two or three weeks to release the version 4.0 (I've been trying to make it my spare time) and certainly I'll let you know about it.

Thank you for the kind words and have an excellent day.

Alexandre.

digitalsleuth commented 3 years ago

Hi Alexandre,

Changes have been made, and the salt state for the software install now points back to your repo. I've tested the installation in our environment and everything tested well! The pull request has been merged, and can be found here.

Thanks for the great work. Cheers!

Corey

alexandreborges commented 3 years ago

Hi Corey,

Thank you for the change and for your kind words.

Have an excellent day.

Alexandre.