search

Intevation / intelmq

IntelMQ is a solution to process data feeds, pastebins, tweets throught a message queue.
http://www.enisa.europa.eu/activities/cert/support/incident-handling-automation
GNU Affero General Public License v3.0
3 stars 0 forks source link

Downloading feed #35

Closed kodebrain closed 1 year ago

kodebrain commented 1 year ago 
2023-02-15 17:24:09,324 - HTTP-Collector-TeamCymru - INFO - Response body: '<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">\n<html><head>\n<title>404 Not Found</title>\n</head><body>\n<h1>Not Found</h1>\n<p>The requested URL /csirt-/csirt-{time[%Y%M%d]}.txt was not found on this server.</p>\n</body></html>\n'.
2023-02-15 17:24:09,324 - HTTP-Collector-TeamCymru - INFO - url: 'https://www.cymru.com/csirt-***/csirt-{time[%Y%M%d]}.txt'.

File "/usr/lib/python3/dist-packages/intelmq/bots/collectors/http/collector_http.py", line 92, in process raise ValueError('HTTP response status code was %i.' % resp.status_code) ValueError: HTTP response status code was 404.

I was able to download the file intelmq run HTTP-Collector-TeamCymru process --show-sent --dryrun by pointing to the individual link https://www.cymru.com/csirt-***/csirt-***_20****.txt but it is not allowing to download using the format csirt-{time[%Y%M%d]}.txt This is working on the older version of intelmq

kodebrain commented 1 year ago

After removing a number of default values, and only assigning the below ones, I was able to download the data


bottype 
extract_files   
http_password   
http_url    
http_url_formatting 
http_username   
logging_path    
name    
provider    
rate_limit  
ssl_client_certificate