PhishTank analyzer doesn't work

ilyaglow commented 6 years ago

Request Type

Bug

Work Environment

Question	Answer
Cortex Analyzer Name	PhishTank
Cortex Analyzer Version	2.0
Cortex Version	latest

Description

PhishTank analyzer fails because http://checkurl.phishtank.com is redirected to main website. Also it seems that summary function call fails too.

Steps to Reproduce

curl -v http://checkurl.phishtank.com
* Rebuilt URL to: http://checkurl.phishtank.com/
*   Trying 107.23.214.106...
* Connected to checkurl.phishtank.com (107.23.214.106) port 80 (#0)
> GET / HTTP/1.1
> Host: checkurl.phishtank.com
> User-Agent: curl/7.47.0
> Accept: */*
> 
< HTTP/1.1 302 Found
< Date: Wed, 01 Nov 2017 12:36:19 GMT
< Content-Type: text/html; charset=iso-8859-1
< Content-Length: 293
< Connection: keep-alive
< Server: Apache/2.4.10 (Debian)
< Location: https://www.phishtank.com/
< 
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>302 Found</title>
</head><body>
<h1>Found</h1>
<p>The document has moved <a href="https://www.phishtank.com/">here</a>.</p>
<hr>
<address>Apache/2.4.10 (Debian) Server at www.phishtank.com Port 80</address>
</body></html>
* Connection #0 to host checkurl.phishtank.com left intact

Possible Solutions

HTTP scheme should be replaced with HTTPS.

It looks like summary() call is unnecessary now, so it may be removed from the main function.

saadkadhi commented 6 years ago

Hi @ilyaglow. That's a good catch. @ecapuano were you aware that PhishTank changed their API URL to HTTPS instead of HTTP while not indicating so in https://www.phishtank.com/api_info.php?

@ilyaglow can you rebase your PR on develop instead of master please? Also, the summary() function is not working because of a var eval error (boolean instead of a string):

--- phishtank_checkurl.py.orig  2017-11-01 20:28:39.220451912 +0100
+++ phishtank_checkurl.py   2017-11-01 21:05:17.922238589 +0100
@@ -19,7 +19,7 @@
                                     'Missing PhishTank API key')

     def phishtank_checkurl(self, data):
-        url = 'http://checkurl.phishtank.com/checkurl/'
+        url = 'https://checkurl.phishtank.com/checkurl/'
         postdata = {'url': data, 'format':'json','app_key': self.phishtank_key}
         r = requests.post(url, data=postdata)
         return json.loads(r.content)
@@ -29,7 +29,7 @@
         value = "\"False\""
         level = ""

-        if 'in_database' in raw and raw['in_database'] == "True":
+        if 'in_database' in raw and raw['in_database'] == True:
             value = "\"{}\"".format(raw['in_database'])
             if raw.get('verified'):

You can add that change to your PR or I'll do it. Thanks.

ilyaglow commented 6 years ago

Hey @saadkadhi. Sure, hope you don't mind that I sent a new PR.

ecapuano commented 6 years ago

Gotta love undocumented API changes! Nice catch @ilyaglow

saadkadhi commented 6 years ago

Closed by PR #128

TheHive-Project / Cortex-Analyzers