Open loganwilliams opened 2 years ago
2022-04-02 13:43:43.000 | ERROR | cisticola.scraper.base:scrape_channels:398 - An error has been caught in function 'scrape_channels', process 'MainProcess' (63504), thread 'MainThread' (139778754606912):
Traceback (most recent call last):
File "/home/loganw/cisticola/app.py", line 136, in <module>
scrape_channels(args)
│ └ Namespace(command='scrape-channels', gsheet=None, media=False)
└ <function scrape_channels at 0x7f20b9d52a60>
File "/home/loganw/cisticola/app.py", line 100, in scrape_channels
controller.scrape_all_channels(archive_media = args.media)
│ │ │ └ False
│ │ └ Namespace(command='scrape-channels', gsheet=None, media=False)
│ └ <function ScraperController.scrape_all_channels at 0x7f20bb9fd280>
└ <cisticola.scraper.base.ScraperController object at 0x7f20b9d1bcd0>
File "/home/loganw/cisticola/cisticola/scraper/base.py", line 332, in scrape_all_channels
return self.scrape_channels(channels, archive_media=archive_media)
│ │ │ └ False
│ │ └ [Channel(name='Qanonfighters', platform_id='1412770923', category='qanon', platform='Telegram', url='https://ttttt.me/qanonfi...
│ └ <function ScraperController.scrape_channels at 0x7f20bb9fd3a0>
└ <cisticola.scraper.base.ScraperController object at 0x7f20b9d1bcd0>
> File "/home/loganw/cisticola/cisticola/scraper/base.py", line 398, in scrape_channels
for post in posts:
│ └ <generator object TwitterScraper.get_posts at 0x7f20b71f2c10>
└ ScraperResult(scraper='TwitterScraper 0.0.1', platform='Twitter', channel=189, platform_id='1389019453803933697', date=dateti...
File "/home/loganw/cisticola/cisticola/scraper/twitter.py", line 26, in get_posts
for tweet in scraper.get_items():
│ │ └ <function TwitterProfileScraper.get_items at 0x7f20ba836790>
│ └ <snscrape.modules.twitter.TwitterProfileScraper object at 0x7f20b74a4d90>
└ Tweet(url='https://twitter.com/MohamedDiallo_/status/1389019453803933697', date=datetime.datetime(2021, 5, 3, 0, 50, 19, tzin...
File "/home/loganw/.local/share/virtualenvs/cisticola-BRujq-3x/lib/python3.9/site-packages/snscrape/modules/twitter.py", line 933, in get_items
yield from self._graphql_timeline_instructions_to_tweets(instructions)
│ │ └ [{'type': 'TimelineClearCache'}, {'type': 'TimelineAddEntries', 'entries': [{'entryId': 'tweet-1509192130408980498', 'sortInd...
│ └ <function _TwitterAPIScraper._graphql_timeline_instructions_to_tweets at 0x7f20ba839e50>
└ <snscrape.modules.twitter.TwitterProfileScraper object at 0x7f20b74a4d90>
File "/home/loganw/.local/share/virtualenvs/cisticola-BRujq-3x/lib/python3.9/site-packages/snscrape/modules/twitter.py", line 678, in _graphql_timeline_instructions_to_tweets
yield self._graphql_timeline_tweet_item_result_to_tweet(entry['content']['itemContent']['tweet_results']['result'])
│ │ └ {'entryId': 'tweet-1509192130408980498', 'sortIndex': '1509192130408980498', 'content': {'entryType': 'TimelineTimelineItem',...
│ └ <function _TwitterAPIScraper._graphql_timeline_tweet_item_result_to_tweet at 0x7f20ba839dc0>
└ <snscrape.modules.twitter.TwitterProfileScraper object at 0x7f20b74a4d90>
File "/home/loganw/.local/share/virtualenvs/cisticola-BRujq-3x/lib/python3.9/site-packages/snscrape/modules/twitter.py", line 668, in _graphql_timeline_tweet_item_result_to_tweet
kwargs['card'] = self._make_card(result['card'], _TwitterAPIType.GRAPHQL)
│ │ │ │ │ └ <_TwitterAPIType.GRAPHQL: 1>
│ │ │ │ └ <enum '_TwitterAPIType'>
│ │ │ └ {'__typename': 'Tweet', 'rest_id': '1509192130408980498', 'core': {'user_results': {'result': {'__typename': 'User', 'id': 'V...
│ │ └ <function _TwitterAPIScraper._make_card at 0x7f20ba839ca0>
│ └ <snscrape.modules.twitter.TwitterProfileScraper object at 0x7f20b74a4d90>
└ {}
File "/home/loganw/.local/share/virtualenvs/cisticola-BRujq-3x/lib/python3.9/site-packages/snscrape/modules/twitter.py", line 630, in _make_card
return Card(**cardKwargs)
│ └ {'url': 'https://twitter.com'}
└ <class 'snscrape.modules.twitter.Card'>
TypeError: __init__() missing 1 required positional argument: 'title'
2022-04-02 13:43:43.067 | INFO | cisticola.scraper.base:scrape_channels:407 - TwitterScraper 0.0.1 found 1 new posts from Channel(name='Mohamed Diallo Live', platform_id=None, category='qanon_content', platform='Twitter', url='https://twitter.com/MohamedDiallo_', screenname='MohamedDiallo_', country='FR(?)', influencer='Mohammed Diallo', public=True, chat=False, notes=None, source='researcher')
This bug is discussed in https://github.com/JustAnotherArchivist/snscrape/issues/407
Great, I'll wait for changes from justanotherarchivist.