[Bug] shodan module broken

[leebaird]

Bug

The shodan module no longer returns any results. Tried both the Kali and GitHub versions. No errors shown.

To Reproduce Kali: theHarvester -d yale.edu -s Added API key to /etc/theHarvester/api-keys.yaml

GitHub: ./theHarvester.py -d yale.edu -s Added API key to api-keys.yaml

Expected Behavior 15 results

System Information OS: Kali Linux Version 2021.3

Additional Information I just tested the built in shodan tool in Kali with my API and it works fine.

[py2n]

i have found the problem. in the main main.py there is a shodan section which uses api to fetch required data. there is a for loop which search for ip addresses in shodan.io. if the IP address is not present in shodan Database theHarvester crashes and it does not continue for searching other IP addresses. i have solved the problem by surrounding the api call with try/except block which resolved the bug.

Shodan

shodanres = []
if shodan is True:
    import json
    print('\033[94m[*] Searching Shodan. \033[0m')
    try:
        for ip in host_ip:
            try:
                print(('\tSearching for ' + ip))
                shodan = shodansearch.SearchShodan()
                shodandict = await shodan.search_ip(ip)
                await asyncio.sleep(2)
                rowdata = []
                for key, value in shodandict[ip].items():
                    if str(value) == 'Not in Shodan' or 'Error occurred in the Shodan IP search module' in str(value):
                        break
                    if isinstance(value, int):
                        value = str(value)

                    if isinstance(value, list):
                        value = ', '.join(map(str, value))
                    rowdata.append(value)
                shodanres.append(rowdata)
                print(json.dumps(shodandict[ip], indent=4, sort_keys=True))
                print('\n')
            except Exception as e:
                print('[SHODAN-error] {e}')
    except Exception as e:
        print(f'\033[93m[!] An error occurred with Shodan: {e} \033[0m')

[py2n]

@L1ghtn1ng could you please check this? if it's OK then i create a pull request?

[cstyrsyn]

I was having the same issue and the above fix appears to work in allowing the module to continue after not finding an IP in shodan.

To Reproduce theHarvester -d yale.edu -b censys -s
(I used censys as just an example I could hit a few times for testing)

System Information theHarvester 4.0.3 OS: Kali Linux Version 2022.1

Error output Searching for 52.86.62.235 52.86.62.235: Not in Shodan [!] An error occurred with Shodan: 'str' object has no attribute 'items'

Perhaps shodan has changed the output for not found that the module can't catch now?