AnthonyBloomer / daftlistings

A library that enables programmatic interaction with daft.ie. Daft.ie has nationwide coverage and contains about 80% of the total available properties in Ireland.
MIT License
171 stars 54 forks source link

Inconsistent number of results #102

Closed sdfordham closed 3 years ago

sdfordham commented 3 years ago

Related to (#101) the following code:

from daftlistings.daft import Daft
from daftlistings.enums import SearchType, PropertyType, Ber
from daftlistings.location import Location

daft = Daft()
daft.set_search_type(SearchType.RESIDENTIAL_RENT)
daft.set_property_type(PropertyType.APARTMENT)
daft.set_property_type(PropertyType.HOUSE)
daft.set_location("Galway")
daft.set_min_beds(1)
daft.set_max_beds(2)
daft.set_min_price(1500)
daft.set_max_price(2000)

listings = daft.search(max_pages=4)
print(len(listings))

Usually gives output:

Got 1477 results.
200

but annoyingly sometimes:

Got 1477 results.
177
AnthonyBloomer commented 3 years ago

The logic seems fine. Is it always 177 or some other number? Perhaps some properties were removed during your testing.

sdfordham commented 3 years ago

PAGE_SZ=50 so the 'last' page should have 27 results, but it looks like sometimes this last page is actually the first page so we get 27 + 50 + 50 + 50 results instead of 50 + 50 + 50 + 50 results. Looking at the daft website, this doesnt seem to really matter since they are using infinite scrolling. But I could be missing something here.

AnthonyBloomer commented 3 years ago

I couldn't replicate this. Perhaps explicity setting the sort type may fix it. That's all I can really think of at the moment. It may be a problem with Daft's API 😄

The default sort type is best match so I'm thinking their search algorithm omits results or shows listings first based on what the user last seen.