search

MISP / misp-modules

Modules for expansion services, enrichment, import and export in MISP and other tools.
http://misp.github.io/misp-modules
GNU Affero General Public License v3.0
341 stars 233 forks source link

Newly created import module is not showing up #77

Open jaegeral opened 7 years ago

jaegeral commented 7 years ago

My idea is to create a new import modules for misp.

I did a cp 

    cp /usr/local/src/misp-modules/misp_modules/modules/import_modtestimport.py /usr/local/src/misp-modules/misp_modules/modules/import_mod/testimport2.py

Changed 

vi /usr/local/src/misp-modules/misp_modules/modules/import_mod/__init__.py

to:

    __all__ = ['testimport2', 'ocr', 'stiximport']

But:

    /usr/local/bin/misp-modules -p 6666 -l 127.0.0.1 -t

Gives me:

uwhois module not installed.
2016-12-01 14:22:29,125 - misp-modules - INFO - Launch MISP modules server from current directory.
2016-12-01 14:22:29,126 - misp-modules - INFO - Helpers loaded cache.py 
2016-12-01 14:22:29,127 - misp-modules - INFO - MISP modules eupi imported
2016-12-01 14:22:29,319 - misp-modules - INFO - MISP modules countrycode imported
2016-12-01 14:22:29,320 - misp-modules - INFO - MISP modules vmray_submit imported
2016-12-01 14:22:29,320 - misp-modules - INFO - MISP modules cve imported
2016-12-01 14:22:29,321 - misp-modules - INFO - MISP modules circl_passivessl imported
2016-12-01 14:22:29,321 - misp-modules - INFO - MISP modules wiki imported
2016-12-01 14:22:29,322 - misp-modules - INFO - MISP modules virustotal imported
2016-12-01 14:22:29,322 - misp-modules - INFO - MISP modules asn_history imported
2016-12-01 14:22:29,322 - misp-modules - INFO - MISP modules shodan imported
2016-12-01 14:22:29,323 - misp-modules - INFO - MISP modules ipasn imported
2016-12-01 14:22:29,323 - misp-modules - INFO - MISP modules dns imported
uwhois module not installed.
2016-12-01 14:22:29,324 - misp-modules - INFO - MISP modules whois imported
2016-12-01 14:22:29,325 - misp-modules - INFO - MISP modules sourcecache imported
2016-12-01 14:22:29,325 - misp-modules - INFO - MISP modules circl_passivedns imported
2016-12-01 14:22:29,326 - misp-modules - INFO - MISP modules passivetotal imported
2016-12-01 14:22:29,326 - misp-modules - INFO - MISP modules reversedns imported
2016-12-01 14:22:29,327 - misp-modules - INFO - MISP modules testimport imported
2016-12-01 14:22:29,328 - misp-modules - INFO - MISP modules stiximport imported
2016-12-01 14:22:29,328 - misp-modules - INFO - MISP modules ocr imported
2016-12-01 14:22:29,329 - misp-modules - INFO - MISP modules vmray_import imported
2016-12-01 14:22:29,330 - misp-modules - INFO - MISP modules testexport imported
2016-12-01 14:22:29,330 - misp-modules - INFO - MISP modules cef_export imported
2016-12-01 14:22:29,334 - misp-modules - INFO - MISP modules server started on 127.0.0.1 port 6666
2016-12-01 14:22:29,335 - misp-modules - INFO - MISP modules started in test-mode, quitting immediately.

So I am missing testimport2 showing up in loaded modules. Any ideas?

Rafiot commented 7 years ago

I generally use a virtual env for testing, it makes everything easier.

Can you try to do the following? 

pip install -I -e .
cudeso commented 7 years ago

I've had to do a couple of times

sudo pip3 install --upgrade .
sudo -u www-data misp-modules -s

to get modules working

Rafiot commented 7 years ago

the -s option uses the modules listed in the __init__.py. If you are developing new modules, I recommend to do pip install -e: -e, --editable <path/url> Install a project in editable mode (i.e. setuptools "develop mode") from a local project path or a VCS url.

jaegeral commented 7 years ago 
misp@misp:/usr/local/src/misp-modules$ sudo -E -u www-data misp-modules -s
uwhois module not installed.
invalid syntax (__init__.py, line 2)
2016-12-01 14:43:59,027 - misp-modules - INFO - Launch MISP modules server from package.
2016-12-01 14:43:59,028 - misp-modules - INFO - Helper loaded cache
2016-12-01 14:43:59,028 - misp-modules - INFO - Unable to load MISP modules from package.

misp@misp:/usr/local/src/misp-modules$ sudo -E pip install -I -e .
Obtaining file:///usr/local/src/misp-modules
  Running setup.py (path:/usr/local/src/misp-modules/setup.py) egg_info for package from file:///usr/local/src/misp-modules

Downloading/unpacking tornado (from misp-modules==1.0)
  Downloading tornado-4.4.2.tar.gz (460kB): 460kB downloaded
  Running setup.py (path:/tmp/pip_build_root/tornado/setup.py) egg_info for package tornado

    no previously-included directories found matching 'docs/build'
    warning: no files found matching 'tornado/test/README'
Downloading/unpacking dnspython3 (from misp-modules==1.0)
  Downloading dnspython3-1.15.0.zip
  Running setup.py (path:/tmp/pip_build_root/dnspython3/setup.py) egg_info for package dnspython3
    Traceback (most recent call last):
      File "<string>", line 17, in <module>
      File "/tmp/pip_build_root/dnspython3/setup.py", line 25
        """+"="*78, file=sys.stdout)
                        ^
    SyntaxError: invalid syntax
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):

  File "<string>", line 17, in <module>

  File "/tmp/pip_build_root/dnspython3/setup.py", line 25

    """+"="*78, file=sys.stdout)

                    ^

SyntaxError: invalid syntax

----------------------------------------
Cleaning up...
Command python setup.py egg_info failed with error code 1 in /tmp/pip_build_root/dnspython3
Storing debug log for failure in /home/misp/.pip/pip.log

It is a virtual test MISP, so I would not really care about virtualenv etc.

Rafiot commented 7 years ago

pip3, sorry.

jaegeral commented 7 years ago

Bam, @cudeso your suggestions solved it.

Thx, will make a short update to the readme to help the next developer take that step easier.

adulau commented 7 years ago

@deralexxx Thank you.