search

massimocandela / geofeed-finder

Utility to find geofeed files linked from rpsl.
BSD 3-Clause "New" or "Revised" License
76 stars 7 forks source link

result.csv file is not being outputted #33

Closed ac-hj closed 8 months ago

ac-hj commented 9 months ago

Hi, I'm currently trying to download geofeeds from all five RIR's. As per README, it seems like running ./geofeed-finder-linux-x64 should do the trick. However, I'm not seeing any result.csv produced after the run is finished. The same goes when I use -i with all 5 RIR's. Interestingly, however, result.csv is produced when I specify one RIR with -i.

In the case where result.csv isn't produced, I only see that logs like below are produced:

[ripe] Downloading whois data
[afrinic] Downloading whois data
[apnic] Downloading whois data
[afrinic] Parsing whois data: inetnum,inet6num
[ripe] Parsing whois data: inetnum,inet6num
...

But when result.csv is outputted, there are more logs being produced as such:

[apnic] Using cached whois data
[apnic] Parsing whois data
<geofeed link> [download]
<geofeed link> [download]
<geofeed link> [download]
<geofeed link> [download]
<geofeed link> [download]
<geofeed link> [download]
<geofeed link> [download]
...

I've also tried testing with and without cache, and the outcome was the same. Another thing to note is that running ./geofeed-finder-linux-x64 has never outputted result.csv during the past few days I've tried running, but running ./geofeed-finder-linux-x64 -i ripe,apnic,lacnic,afrinic,arin has outputted result.csv until yesterday and has randomly stopped doing so today.

massimocandela commented 9 months ago

Hi @ac-hj,

Can you provide the logs of rm -rf .cache/ && ./geofeed-finder-linux-x64 -i arin? Even jus the initial logs up to when it starts fetching netranges

ac-hj commented 9 months ago

Hi, after removing the cache and running the command you have above, it only took 2.15s and outputted only the logs below.

[arin] Downloading stat file
[arin] Fetching sub allocations ipv4

No error logs were produced as well.

ac-hj commented 9 months ago

To add onto the above, ./geofeed-finder-linux-x64 -i apnic produces more logs with result.csv file too. arin did not produce result.csv file. afrinic, lacnic and ripe also all produced result.csv when run individually. 

[apnic] Using cached whois data
[apnic] Parsing whois data
https://geofeed.tools.telstra.net/geofeed.csv [download]
https://www.spintel.net.au/geofeed/geofeed.csv [download]
https://pentanet.com.au/geofeed.csv [download]
https://livertonnetworks.com/geofeed.csv [download]
https://www.perthmint.com/geofeed.csv [download]
https://nettools.vocus.network/vocus-geo-feed.csv [download]
https://geofeed.swoop.com.au/geofeed.csv [download]
https://geofeed.constant.com/ [download]
https://sggs.network/geolocation.csv [download]
http://geofeed.llnw.net/ [download]
...
sid6mathur commented 9 months ago

Hi, after removing the cache and running the command you have above, it only took 2.15s and outputted only the logs below.

[arin] Downloading stat file
[arin] Fetching sub allocations ipv4

No error logs were produced as well.

Facing the same issue for the past 48hours; the above takes about 5s or less. echo $? is zero, so the tool isn't returning an error to the shell.

massimocandela commented 8 months ago

I'm not sure I was able to reproduce the exact issue you were facing. Please, let me know if the latest release fixes the issue.

sid6mathur commented 8 months ago

I'm not sure I was able to reproduce the exact issue you were facing. Please, let me know if the latest release fixes the issue.

The latest release is working well for me. Thank you! :)