search

TechAndCheck / zorki

MIT License
0 stars 0 forks source link

LONG hang when a page is not available #14

Open cguess opened 2 years ago

cguess commented 2 years ago

When there's no post found i.e. https://www.instagram.com/p/sfhslsfjdls/ it hangs for awhile looking for elements. Let's figure out a bit faster way to figure that out.

oneroyalace commented 2 years ago

A page.title check might be quick. The check might fail if Instagram changed its content-not-found page, but if that happens, worst case we wait for the 60 second timeout here: https://github.com/cguess/zorki/blob/5b08badcf927cf0423abf94a2d2f2bd5541c310d/lib/zorki/scrapers/scraper.rb#L87

We could also change the line above to read return if find_field("Search", wait: n).present?. I've never run into trouble setting n at 15