'favicon' key received inside filteredResponse made Pandas throw an AttributeError

[dacian16]

Hi guys, I cloned the repo and install the dependencies correctly but for some reason it gave me an AttributeError: 'int' object has no attribute 'items' error while trying to create the dataframe with the remaining keys stored in the filteredResponse dictionary (pd.DataFrame.from_dict) - see urlReport(arg) function. Looking into the filteredResponse that's being passed to the DataFrame Pandas function, it seems that it didn't like the "favicon" attribute so I added it to the _keys_toremove array and now everything works fine.

The error:

└─$ python3 vt-ip-url-analysis.py -s google.com

+++++ Print filteredResponse +++++
{'favicon': {'raw_md5': '1631ff32e63a6dcc15469d14c7c94e42', 'dhash': 'e89e931939338ee8'}, 'times_submitted': 166200, 'redirection_chain': ['http://google.com/', 'http://www.google.com/'], 'reputation': 2615, 'has_content': False, 'last_analysis_stats': {'harmless': 81, 'malicious': 0, 'suspicious': 0, 'undetected': 8, 'timeout': 0}}
+++++++++++
Traceback (most recent call last):
  File "/home/kali/Desktop/Python-VT-Script/vt-ip-url-analysis.py", line 456, in <module>
    urlReport(args.single_entry)
  File "/home/kali/Desktop/Python-VT-Script/vt-ip-url-analysis.py", line 167, in urlReport
    dataframe = pd.DataFrame.from_dict(filteredResponse, orient='index')
  File "/home/kali/Desktop/Python-VT-Script/lib/python3.10/site-packages/pandas/core/frame.py", line 1747, in from_dict
    data = _from_nested_dict(data)
  File "/home/kali/Desktop/Python-VT-Script/lib/python3.10/site-packages/pandas/core/frame.py", line 12029, in _from_nested_dict
    for col, v in s.items():
AttributeError: 'int' object has no attribute 'items'

Below is the filteredResponse received for google.com:

{'favicon': {'raw_md5': '1631ff32e63a6dcc15469d14c7c94e42', 'dhash': 'e89e931939338ee8'}, 'times_submitted': 166198, 'redirection_chain': ['http://google.com/', 'http://www.google.com/'], 'reputation': 2615, 'has_content': False, 'last_analysis_stats': {'harmless': 81, 'malicious': 0, 'suspicious': 0, 'undetected': 8, 'timeout': 0}}

After adding "favicon" to the _keys_toremove attribute:

┌──(Python-VT-Script)─(kali㉿kali)-[~/Desktop/Python-VT-Script]
└─$ python3 vt-ip-url-analysis.py -s google.com

+++++Print filteredResponse +++++
{'times_submitted': 166200, 'redirection_chain': ['http://google.com/', 'http://www.google.com/'], 'reputation': 2615, 'has_content': False, 'last_analysis_stats': {'harmless': 81, 'malicious': 0, 'suspicious': 0, 'undetected': 8, 'timeout': 0}}
+++++++++++
                                                                     0
times_submitted                                                 166200
redirection_chain         [http://google.com/, http://www.google.com/]
reputation                                                        2615
has_content                                                      False
last_analysis_stats  {'harmless': 81, 'malicious': 0, 'suspicious':...
                                                            google.com
community score      0/89  :  security vendors flagged this as mali...
has_content                                                      False
last_analysis_date                            Thu Oct  6 10:32:49 2022
last_analysis_stats  {'harmless': 81, 'malicious': 0, 'suspicious':...
redirection_chain         [http://google.com/, http://www.google.com/]
reputation                                                        2615
times_submitted                                                 166200
virustotal report    https://www.virustotal.com/gui/url/cf4b367e49b...

[ArishJawaid]

This error was caused by inconsistencies in the FilterResponse dictionary data, where certain keys were occasionally found in different positions that raised n AttributeError exception. In my case, the index of Reputation key in the FilterResponse dict is inconsistent. To address this issue, I made some modifications to the code as a workaround. Since implementing these changes, I have not encountered the issue again.

Add the following code before the line 165:

if list(filteredResponse.keys()).index('reputation') != 0:
    rep = filteredResponse.pop('reputation')
    items = list(filteredResponse.items())
    items.insert(0, ('reputation', rep))
    filteredResponse = dict(items)