yacy / yacy_search_server

Distributed Peer-to-Peer Web Search Engine and Intranet Search Appliance
http://yacy.net
Other
3.38k stars 426 forks source link

no man page, forum reg is insecure, no real instructions on how to open crawl results to the network. #138

Open Lew-Rockwell-Fan opened 6 years ago

Lew-Rockwell-Fan commented 6 years ago

I salute you all for a great idea & I hope it works out & kills google. Much of this post would probably be more appropriate at the forum, but I can't register there, and that IS a bug, albeit not in yacy itself. Maybe y'all can pass this on to whoever runs it. Firefox tells me the registration page isn't secure. Apparently it was refusing an https connection and falling back to http. For a registration page where I'm putting in email & pass, that's not so good. In Pale Moon, it says the certificate isn't valid.

Anyway, yacy itself: It didn't install a man page, or if it did, it isn't accessible in the normal way. I'd call that a bug, although you may disagree. I can't absolutely swear to it - maybe trying to get this working has driven me crazy - but I thought when I started messing with this, that at first the various http://localhost:8090/* pages included links to all the other such pages near the upper left. Now they don't. Did I inadvertently change a setting to do this, or was I just hallucinating to begin with? It's hard to find my way back to specific pages now,

Anyway, for anyone struggling with a Belkin router, the attached image shows what worked for me, try going to: http://192.168.2.1/fw_virt.html May not be the same address for you, dunno. If you haven't set the pass, try just clicking "submit" with the field empty. Then try setting it like the attached photo. Good chance you'll need a different number in the "provide ip" field. Still, examples help. This is what worked for me: belkin_router_setup_that_worked_for_me Don't forget to check your software firewall too. Might be easier to turn it off until you get this set up & working. Then get the firewall rule right. For people who use ufw, if you have trouble getting it to cooperate, try the gui front end "gufw". I've used the port yacy defaulted to, but I'm wondering if it wouldn't be more secure to pick a random number. I haven't tried that.

On this page: http://localhost:8090/ConfigBasic.html it says: "Configure your router for YaCy using UPnP: Configuration was not successful. This may take a moment." This message doesn't change in any reasonable number of moments, let alone "a moment".

But on the same page it also says: "good You are running a server in senior mode and you support the global internet index, which you can also search yourself. " and on http://localhost:8090/Status.html (I think that's where it was, my notes aren't perfect) it says: ""ok Your peer can be reached by other peers" Actually it doesn't say "good" or "ok". Those must be the name of the "!" icons that preceded the text I was trying to copy.

My router firewall config page offers only: TCP UDP BOTH

"BOTH" seems to work. I haven't tried the other settings. Is UPnP some kind of fallback in case the other(s) don't work? Is "BOTH" the proper setting or should I use one of the others?

So I finally figured out how to set up the router and the ufw software firewall, so that it works to some degree, but it took me about 12 hours & while I wouldn't call myself a hardcore geek, I'm hardly a technophobe, having had a PC since before the internet existed. People who aren't gamers mostly don't know diddly about network protocols, ports, & so on, unless they were sys admins back in the bad old days when setting up a LAN was HARD. This is intended as suggestion rather than criticism, but if I had this much trouble getting this (kinda sorta) working, I suggest that making the setup more user-friendly needs to be a high priority. I suspect there are a lot more people willing to give some drive space, cpu time, and bandwidth to make this work, than there are who will wade through the setup without giving up. FWIIW,

I know you can't do everything at once, but I suggest you might want to put a high priority on at least explaining better how to setup routers & software firewalls. Would it be possible to detect the brand of router and any software firewalls present and script the needed changes? Or at least give detailed instructions? "Explaining" what needs to be done by saying "go search youtube" isn't all that helpful. I suspect you lose a lot of people who would be crawling and contributing the results at this one step.

You might at least give some representative examples. Text with screen shots are a lot easier for most people to dig info out of than having to stop & start a low rez video of somebody mumbling while pointing and clicking.

I suspect that is your chokepoint in getting a larger user base contributing crawl results. Anyway, thanks for your efforts, & best of luck.

luccioman commented 6 years ago

Hi @Lew-Rockwell-Fan , thanks for this detailed feedback. For sure YaCy is still far from easy to set up for anyone who is not found of technology, especially when everything is not working fine at the first step.

About networking and ports configuration, that's why UPnP is enabled by default. With this technology (when properly working) you don't have to bother with complex configuration on your router : everything required to make ports open is done by communications between YaCy (or any other network application) and the UPnP enabled router. But for this to work, UPnP has first to be enabled (if available) on the router. Are you sure your Belkin one doesn't provide a setting for this?

By the way, when UPnP is not an available option, things become more complex, and this not specific to YaCy (yes YaCy has not advanced NAT traversal capabilities, which does not help).

There are quite many different hardware and software combinations, and this would be difficult to cover by a reduced number of volunteer developers. To my mind this is where the user feedback can come in action... but it takes some time to create a comprehensible text or video tutorial and publish it in a relevant place (as far as I know YaCy wiki can to be a good one).

Lew-Rockwell-Fan commented 6 years ago

Thank you. Especially for the clarification on UPnP & for suggesting the Wiki. I'll contribute at least a stub there shortly.

I hope my observations re problems with forum registration get passed on. I realize most of what was in my post would be more appropriate there than here. I put it here out of desperation since I couldn't register.

JeremyRand commented 6 years ago

FYI, if YaCy were capable of automatically setting up a Tor onion service, that would have the side benefit of making NAT traversal a lot easier (in addition to the obvious privacy benefits). See https://github.com/yacy/yacy_search_server/issues/84 for some relevant discussion.

funkyfuture commented 6 years ago

i don't know whether this is known, but there also seems to be an issue with the database of the forum which is hence not usable.

what about a [yacy] tag on superuser.stackexchange.com?