search

yacy / yacy_search_server

Distributed Peer-to-Peer Web Search Engine and Intranet Search Appliance
http://yacy.net
Other
3.32k stars 424 forks source link

Yacy for alternate TLDs #231

Open rastislavcore opened 5 years ago

rastislavcore commented 5 years ago

It is in roadmap list onion sites like AHMIA is listing it as well as alternate TLDs? https://github.com/juhanurmi/ahmia

Some alternative domains will be nice, if we speak about idea of decentralisation. Example of TLDs: onion, eth (https://ens.domains/)

luccioman commented 5 years ago

Hi @raisty , up to now the idea for supporting alternate naming systems in YaCy was to write a new YaCy network definition file and configure the willing peers (at /ConfigNetwork_p.html page) to use it instead of the 'freeworld' network that is designed to support only standard domain names.

An example network definition is notably available for ZeroNet which operates on .bit domains.

rastislavcore commented 5 years ago

As far as I know it is not enough definition, but gateway as well. It should be implemented sofisticated system to browse alternative networks. For onion TLD should be necessary some type of machine, which can handle all of the network. Maybe it is connected to issue #232 as well.

luccioman commented 5 years ago

Yes you are completely right, YaCy network definition customization in itself is not sufficient, you also need to pass through some kind of gateway or alternate names resolver. You did not mention it, but there is notably the long running OpenNIC project that manages its owns TLDs and also appears to be usable as a gateway to some peered naming systems.

By the way, as far as I know, in YaCy history it has been assumed that to keep things simple, support for non standard naming systems or protocols should be handled with external tools. Otherwise it would represent too much effort to integrate new libraries and code to support every new type of network appearing every year.

Maybe @Orbiter could complete on this, but I think that was roughly its explanation for how ZeroNet support should be added, and it still sounds reasonable to me.

rastislavcore commented 5 years ago

Absolutely! We need looking forward for decentralisation. Great tool which can work is lite nodes - they will collect informations from different areas of network. Let's say it can be vertical line - from deep web to heaven web. Anybody can then fork lite nodes and deliver informations from craweling. But in this case we need integration in YaCy and probably splitting crawelers in new project. Contributing then can be simple.

Example of scheme:

- lite node - craweler populate P2P database
- node will store crawling results
- YaCy will connect to X amount of nodes
- YaCy will serve to frontend

OpenNIC is great example as well. I am looking into anonymity and Blockchain structure.

Please comment, if you are open to discussion about decentralised googling. :godmode:

luccioman commented 5 years ago

I am still not sure to truly get your idea. Indeed do you mean that current YaCy operations should be split into micro services, like in the YaCy Grid?