intelowlproject / IntelOwl

IntelOwl: manage your Threat Intelligence at scale
https://intelowlproject.github.io
GNU Affero General Public License v3.0
3.81k stars 432 forks source link

[Analyzer] Domaincheck #1682

Open mlodic opened 1 year ago

mlodic commented 1 year ago

Name

Domaincheck

Link

https://github.com/zuzazuza/domaincheck

Type of analyzer

domain only

Why should we use it

cool tool to check DNS related problems

Possible implementation

integrate the open source tool: https://github.com/zuzazuza/domaincheck

g4ze commented 4 months ago

hi! @mlodic since the tool seems like a standalone command, it cant be imported as a package after cloning and installing. Could you suggest some workaround for it?

mlodic commented 4 months ago

yes the tool is pretty raw but it is still much useful and clever.

One idea, that I have already done before in such cases, is that we could create a package for this and maintain it.

I created a fork here. What you could do is to add the necessary items to publish it on pypi and then use the pypi package in this analyzer.

It's something fast to do once you know what to touch. Check what I did here for pdfid. I added the github workflow, managed the version and setup.py configuration and that's all.

Thoughts?

g4ze commented 4 months ago

That might be a bit of an overkill imo, cant we just import subprocess and use the shell command to directly fetch results, we could install it sipmly via the dockerfile..? Is that going to increase the processing time significantly?

g4ze commented 4 months ago

my bad, accidental click

mlodic commented 4 months ago

I honestly do not like too much and would prefer to avoid it if possibile. Anyway, even if we went in that direction, we would still have a problem with the installation. We cannot make the binary directly in our image (too expensive) and, even if we build it beforehand, we cannot embed it in IntelOwl itself without a proper installation method (that would mean distribution -> license implications).

For some python code, even if they are a little, makes sense to do what I suggested and it is really easy and plain, it's not an overkill. Have you ever created a python package before? That is also a chance to try to do that.

g4ze commented 4 months ago

okay, i understand, it makes more sense now. Installation would expensive in our case. Let's do it the way you proposed! Yes I've published a small package key-proxy . https://github.com/g4ze/key-proxy/tree/package_support not a really sophisticated one tho

g4ze commented 4 months ago

referencing latest update

mlodic commented 3 months ago

I have just finalized the creation of the package: https://pypi.org/project/domaincheck-fork/ you can now use it to complete the issue :)

g4ze commented 3 months ago

Woohoo!

g4ze commented 2 months ago

hey @mlodic after importing domaincheck, I'm not able to access its modules or anything. Are we expected to run it as a command?

mlodic commented 2 months ago

the package is domaincheck-fork, are you sure to have used the correct one?

g4ze commented 2 months ago

after installing the published package pip install domaincheck-fork:

Image Neither of the import are working. I'm not able to access its modules. the command domaincheck is functional. I might be missing something here, but i have deleted all the domaincheck packages and did a fresh reinstall of domaincheck-fork

mlodic commented 2 months ago

ok that's happening cause the python script name does not end with .py............ I fixed it, pls try again with version 0.2

g4ze commented 2 months ago

there's still a slight problem with approach we are taking, domaincheck is a standalone script and we cant use its modules as it is. we need to make the functions inside more modular and reusable outside. I'll try to fix that and make a PR in the fork. Please let me know if what I'm saying is valid.

g4ze commented 2 months ago

referencing the proposed fix this works as expected, not the best solution but gets the work done.

mlodic commented 2 months ago

yes, I have to admit that the code isn't portable at all. I have just given you maintainers right to that repo so you can create a release and try it. Thank you