JustAnotherArchivist / snscrape

A social networking service scraper in Python
GNU General Public License v3.0
4.31k stars 698 forks source link

Fix KeyError when a user is unavailable #956

Closed novucs closed 1 year ago

novucs commented 1 year ago

KeyError: 'result' reproducible when scraping previously existing user: TwitterProfileScraper(229692331)

JustAnotherArchivist commented 1 year ago

I'm getting an snscrape.base.EntityUnavailable: User unavailable as expected. But there have been similarly silly responses from Twitter before (#828), so I'm not surprised it might happen temporarily.

That said, rather than comparing to an empty dict, this needs handling similar to the Empty response exceptions in the entity fetching.

novucs commented 1 year ago

I'm getting an snscrape.base.EntityUnavailable: User unavailable as expected. But there have been similarly silly responses from Twitter before (#828), so I'm not surprised it might happen temporarily.

That said, rather than comparing to an empty dict, this needs handling similar to the Empty response exceptions in the entity fetching.

Thanks, updated the check for existence of result rather than empty dict

JustAnotherArchivist commented 1 year ago

Sorry for not being clearer, I meant raising a ScraperException('Empty response') in that particular case of result missing.