CRITICAL:root:twint.run:Twint:Feed:noData'globalObjects'

joon612 commented 3 years ago

Initial Check

If the issue is a request please specify that it is a request in the title (Example: [REQUEST] more features). If this is a question regarding 'twint' please specify that it's a question in the title (Example: [QUESTION] What is x?). Please only submit issues related to 'twint'. Thanks.

Make sure you've checked the following:

[×] Python version is 3.6;
[√] Updated Twint with pip3 install --user --upgrade -e git+https://github.com/twintproject/twint.git@origin/master#egg=twint;
[√] I have searched the issues and there are no duplicates of this issue/question/request.

Command Ran

import twint
import pandas

c=twint.Config()
c.Search = "#cat"
c.Limit = 2000000000
c.Images= True
c.Output = r"D:\tmp\twitter\test.json"
c.Store_json = True
c.Pandas = True
twint.run.Search(c)
Tweets_df = twint.storage.panda.Tweets_df
t_df=pandas.DataFrame(Tweets_df)

Description of Issue

After I got 55k results, this err happens: CRITICAL:root:twint.run:Twint:Feed:noData'globalObjects'

Environment Details

Windows 10 Python 3.8.6

joon612 commented 3 years ago

I was search 2021-01-01~2021-08-17.

zxhhh97 commented 3 years ago

I have met the same case.

aksvisu commented 3 years ago

Here's a possible solution to your problem: Try git clone instead of pip install (I used Google Colab)

!git clone --depth=1 https://github.com/twintproject/twint.git !cd /content/twint && pip3 install . -r requirements.txt !pip3 install nest_asyncio !pip3 install -qq whatthelang import twint import pandas import nest_asyncio nest_asyncio.apply() c=twint.Config() c.Search = "#cat" c.Limit = _2000000000 c.Images= True c.Output = r"test.json" c.Store_json = True c.Pandas = True twint.run.Search(c) Tweets_df = twint.storage.panda.Tweets_df t_df=pandas.DataFrame(Tweets_df)

Capture

joon612 commented 3 years ago

Sometimes the same search criteria will return a different number of results. For example, adding a date condition with an interval of a week.

From: aksvisu @.> Sent: Monday, August 23, 2021 1:25:53 AM To: twintproject/twint @.> Cc: JXD @.>; Author @.> Subject: Re: [twintproject/twint] CRITICAL:root:twint.run:Twint:Feed:noData'globalObjects' (#1257)

Here's a possible solution to your problem: Try git clone instead of pip install (I used Google Colab)

!git clone --depth=1 https://github.com/twintproject/twint.git !cd /content/twint && pip3 install . -r requirements.txt !pip3 install nest_asyncio !pip3 install -qq whatthelang import twint import pandas import nest_asyncio nest_asyncio.apply() c=twint.Config() c.Search = "#cat" c.Limit = _2000000000 c.Images= True c.Output = r"test.json" c.Store_json = True c.Pandas = True twint.run.Search(c) Tweets_df = twint.storage.panda.Tweets_df t_df=pandas.DataFrame(Tweets_df)

[Capture]https://user-images.githubusercontent.com/48058736/130364241-0e6bc9eb-8670-4c0d-a3fb-c8e3953b351c.png

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/twintproject/twint/issues/1257#issuecomment-903302408, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AIGM4R72CXM5ZYYQ7XBGYBTT6EXKDANCNFSM5CK7MMNA. Triage notifications on the go with GitHub Mobile for iOShttps://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Androidhttps://play.google.com/store/apps/details?id=com.github.android&utm_campaign=notification-email.

Masaru-KMT commented 3 years ago

I'm facing the same problem. The same command option suddenly started giving me a lot of trouble about a week ago. I am guessing that Twitter has started some kind of interference. I think some changes need to be made to the collection method itself.

puddlejumper90 commented 3 years ago

I'm facing the same problem. The same command option suddenly started giving me a lot of trouble about a week ago. I am guessing that Twitter has started some kind of interference. I think some changes need to be made to the collection method itself.

I agree, I have been researching the Twitter API for my current employer. Twitter probably makes the majority of its money by selling enterprise access to its data.

twintproject / twint