search

khwang0 / 2018F-COMP3111

19 stars 13 forks source link

[Basic 2 & Basic 4] #73

Open tkim0719 opened 5 years ago

tkim0719 commented 5 years ago

the additional website chosen is carousel, but the html class element names are keep changing, and the program that used to work doesn't work. should I change the website? and Moreover, webpages other than carousel does not provide date of post. Is it fine if we display the latest item among craiglist items?

comp3111ta commented 5 years ago

You can cache carosell pages (abt 20) for us to test. Find a way to get the post or modify date. If you cant switch it quick

HYUNJS commented 5 years ago

You can cache carosell pages (abt 20) for us to test. Find a way to get the post or modify date. If you cant switch it quick

What do you mean "cache the pages"? How can we store the website at the time to our program in order to test the scraped items and multipagination which are changed by the time?

comp3111ta commented 5 years ago

Cache - download and save the webpage in your hard drive so that the content will never change.

In test, you use the cache In the live program, you use the real URL

If you want to make it a little more elegant, try use parametized test. https://stackoverflow.com/questions/35220100/how-can-i-apply-a-single-test-case-to-all-the-pages-of-the-website-using-seleniu