search

khwang0 / 2018F-COMP3111

19 stars 13 forks source link

[Basic 2 & 3] Incorrect Number of Item Display on Craigslist Search Result #68

Open billy1624 opened 5 years ago

billy1624 commented 5 years ago

  1. When I search google pixel 5 on Craigslist (url: https://newyork.craigslist.org/search/sss?query=google+pixel+5&sort=rel). The page showing there are in total 17 items. However, there are 52 items on the result page. Should we scrape all the result? image

  2. When I search google pixel 5 on Craigslist (url: https://newyork.craigslist.org/search/sss?query=abc&sort=rel). There are in total 443 item, but each page was showing different total item count. This could lead to incorrect calculation of total page, when we do Basic Task 3. Page 1. image Page 2. image Page 3. image Page 4. image

comp3111ta commented 5 years ago

Q1. Following your links given in link one, there are some item suggest below the regular search. You are allowed to include them or exclude them. #uptoyou .

Q2. That means the total number of items given is not reliable. You should not rely on that one. Similar problem appears in Google web engine. This is only an estimation I afraid.

tkim0719 commented 5 years ago

@comp3111ta So do I have to seek for another way for pagination?

comp3111ta commented 5 years ago

May be. I am sorry that I am less supportive in providing technical solution here. Mostly clarification of requirement here, or that is really too technical that it is out of your ability.