InQuest / iocextract

Defanged Indicator of Compromise (IOC) Extractor.
https://inquest.readthedocs.io/projects/iocextract/
GNU General Public License v2.0
498 stars 91 forks source link

Found IPs being parsed as URLs #77

Closed bee-gees closed 1 year ago

bee-gees commented 1 year ago

Hey! Currently working with iocextract to read from a text file and convert to a query. I just now ran in the issue where the IPs were being extracted as IPs but then they were also being extracted and formatted as URLs. Input: 101.28[.]225[.]248 ---> Output: RemoteIP =~ "101.28.225.248" or RemoteUrl has "http://101.28.225.248"

battleoverflow commented 1 year ago

Hi, @bee-gees!

This issue should now be resolved and will be available in the next release of iocextract. You can see an example below of how to achieve this.

The best method for this is to use extract_ips(), but a simplified version is available if you’d like to include extract_urls() as well. This is ultimately left up to preference and testing to see which method works best for your use case.

data = \
“”"
101[.]28[.]225[.]248
http://102.35.235.240
https://105.55.245.180
“”"

# Using extract_urls()
# refang and no_scheme must be set to true
print(list(iocextract.extract_urls(data=data, refang=True, no_scheme=True)))

# Using extract_ips()
print(list(iocextract.extract_ips(data=data)))

The new version can be installed here: https://pypi.org/project/iocextract/1.16.1

You can also install the update using pip: pip install iocextract -U