search

InternetHealthReport / internet-yellow-pages

A knowledge graph for the Internet
https://iyp.iijlab.net
GNU General Public License v3.0
44 stars 18 forks source link

Remove/Transform Wikidata leftovers #23

Closed m-appel closed 8 months ago

m-appel commented 1 year ago

There are some Wikidata files left that should be removed and some of the crawlers need to be refactored for neo4j usage.

Bhardwaj-Himanshu commented 1 year ago

Hi @m-appel, If this issue is still open and you need some help, I would love to do so.

I had some queries before proceeding towards the issue: -could you please provide a link or more details over where this data exactly is located?

Thanks

SAHELISAHAA commented 8 months ago

@m-appel is this issue still open ? can i work on this ?

m-appel commented 8 months ago

In principle, this issue it still open, but more as a reminder for us to eventually decide on what to do with the leftovers.

Basically, everything that uses the wikihandy module needs to be removed or ported to neo4j, but as far as I can see this would just include deleting files, since I don't think we want to port any of the leftovers.

I'll have to discuss this with @romain-fontugne eventually, bit this is not a high priority at the moment.

romain-fontugne commented 8 months ago

I believe now this is just a matter of deleting old files. I just checked and the only crawlers we haven't migrated are either outdated (e.g. rapid7 that is not public anymore) or have been replaced (e.g. bgp is not needed anymore, we get that data from bgpkit and hoping to soon get spamhaus via firebog)