search

Ornias1993 / fetlife-aslsearch-reborn

Tampermonkey user script offering an interface to perform pseudo-automatic searches of the FetLife.com user base filtered by age, sex, location, and role.
6 stars 2 forks source link

Search for London, city of gives no results #70

Closed Slofe closed 4 years ago

Slofe commented 4 years ago

Description

Category

BUG, TODO or feature?

Detailed Bug Report

Steps to Reproduce

Please enter the steps to reproduce the bug or behaviour: 1.BUG

  1. Not sure you got my previous report but I get no results no matter what search I do. I search London, the capital of the UK and get no results.

  2. On Fetlife London is listed as London, City of, United Kingdom. I put London in the city field, City of in the state/province field and United Kingdom in the country field and get zero results. I've also tried the same search but leaving out the state/province field and still get zero results. This is using violent monkey script addon in Firefox latest version.

Slofe commented 4 years ago

Should mention it's the same for tampermonkey on chrome.

Ornias1993 commented 4 years ago

The correct notation should be: "London, City of" under state/province".

"London, City of" is not listed on fetlife as a city but as a province, it's a discrepency on their end.

Slofe commented 4 years ago

Ah I see! Thanks, I'll give that a try and report back.

Slofe commented 4 years ago

So it does one batch, many of which are deactivated profiles, but doesn't load batch 2.

Ornias1993 commented 4 years ago

The "batch" system has been removed in the new version and is not tested in the current version.

The pages should work (although there is a bug where it sometimes seems to freeze when changing pages), batches does not.

You can craft batches yourself based on age, for example you can search for 18-25 and 26-30 to get 2 batches containing everyeone between 18 and 30 (if that fits)

Slofe commented 4 years ago

I dunno, I just tried it again and it showed 6 people in a city of 7 million. I thought maybe my adblocker was causing a problem but there's no change when I turn it off.

Ornias1993 commented 4 years ago

There might only be 6 people in our database.

Slofe commented 4 years ago

Oh it's a database search and not a whole site search? Is site search coming in a future update?

Ornias1993 commented 4 years ago

Previous versions of fetlife ASL got people banned, because you need to request a LOT of data from fetlife to "search" the site manually.

The current versions of Fetlife ASL, work by community scraping. Everyone using the plugin sends profile details of all profiles they come across to a central database. Every search executed on said database.

It's actually in the FAQ, although under a different question: https://github.com/Ornias1993/fetlife-aslsearch-reborn/wiki/FAQ

Slofe commented 4 years ago

Oh I didn't realise it was like that I thought it was going to be the same as the previous one. I've used a few different scripts like this over the years and I never once got banned, perhaps I ws lucky? Hoping enough people click on profiles so a database gets filled isn't very useful, it will be quicker to just sift through profiles by hand! Oh well, hope you add the option to search fetlife properly in the future.

Ornias1993 commented 4 years ago

@Slofe I think you don't fully understand it. The previous versions also already did it this way since about 2016-2017. It was a VERY old version that actually opened profiles itself.

It takes to long to fully explain, but it doesn't "just" rely on people clicking profiles. Every user list someone opens gets scraped. If you see a user (name, profile pictures etc) it gets scraped every times. That means user lists already upload about 20 users in one click. The database fills very fast.

But you are right, thats not enough. Thats why next version also includes 4 miljoen (about 60% of fetlife) of prescraped profiles.

"it will be quicker to just sift through profiles by hand" Thats certainly not true, because it aggregates the combined "clicks by hand" of everyone in the same region. but also future manual scraping, such as the 4 milion profiles incomming with next version.

Slofe commented 4 years ago

You're right I didn't understand but have a better understanding now. Thank you for explaining it to me and for the work you're doing on the script :)