twintproject / twint

An advanced Twitter scraping & OSINT tool written in Python that doesn't use Twitter's API, allowing you to scrape a user's followers, following, Tweets and more while evading most API limitations.
MIT License
15.65k stars 2.72k forks source link

Seems broken #1009

Open jtgd opened 3 years ago

jtgd commented 3 years ago

Running twint-2.1.19-1 from AUR. This happens with every feed:

$ twint -u MSNBC CRITICAL:root:twint.get:User:'NoneType' object is not subscriptable CRITICAL:root:twint.run:Twint:Feed:noDataExpecting value: line 1 column 1 (char 0) CRITICAL:root:twint.run:Twint:Feed:noDataExpecting value: line 1 column 1 (char 0) CRITICAL:root:twint.run:Twint:Feed:noDataExpecting value: line 1 column 1 (char 0) CRITICAL:root:twint.run:Twint:Feed:noDataExpecting value: line 1 column 1 (char 0) CRITICAL:root:twint.run:Twint:Feed:noDataExpecting value: line 1 column 1 (char 0) CRITICAL:root:twint.run:Twint:Feed:noDataExpecting value: line 1 column 1 (char 0) CRITICAL:root:twint.run:Twint:Feed:noDataExpecting value: line 1 column 1 (char 0) CRITICAL:root:twint.run:Twint:Feed:noDataExpecting value: line 1 column 1 (char 0) CRITICAL:root:twint.run:Twint:Feed:noDataExpecting value: line 1 column 1 (char 0) CRITICAL:root:twint.run:Twint:Feed:noDataExpecting value: line 1 column 1 (char 0) CRITICAL:root:twint.run:Twint:Feed:Tweets_known_error:Expecting value: line 1 column 1 (char 0) Expecting value: line 1 column 1 (char 0) [x] run.Feed [!] if get this error but you know for sure that more tweets exist, please open an issue and we will investigate it!

Moeh-Jama commented 3 years ago

Having same issue, seems that the whole project been impacted from Sept 18 due to twitters changes 😞

himanshudabas commented 3 years ago

@Moeh-Jama try installing from github instead of pip use the following command to install from github

pip3 install --user --upgrade git+https://github.com/twintproject/twint.git@origin/master#egg=twint
Moeh-Jama commented 3 years ago

Hey thanks 😊, I still got the issue but changing the call fixed @himanshudabas

For more information for anyone that sees this, after updating like Himanshu said I changed my query, removing the store and hide filters and manually saved results myself like below.

-- config.Store_csv = True doesn't work :disappointed:

++ search_list = config.search_tweet_list and saving that result.

himanshudabas commented 3 years ago

config.Store_csv = True doesn't work 😞

Huh! That's weird. It should work as expected.

Can you please share the script you ran and the error you get? (if there is some error)

jtgd commented 3 years ago

I originally installed from the Arch User Repository. This is the PKGBUILD:

pkgbase='twint' pkgname=('twint') _module='twint' pkgver='2.1.19' pkgrel=1 pkgdesc="An advanced Twitter scraping & OSINT tool." url="https://github.com/twintproject/twint" depends=('python' 'python-aiohttp' 'python-aiodns' 'python-beautifulsoup4' 'python-cchardet' 'python-elasticsearch' 'python-pysocks' 'python-pandas' 'python-geopy' 'python-googletransx' 'python-schedule' 'python-fake-useragent' 'python-aiohttp-socks') makedepends=('python-setuptools') license=('MIT') arch=('any') source=("https://files.pythonhosted.org/packages/source/${_module::1}/$_module/$_module-$pkgver.tar.gz") sha256sums=('61ff7f4e4b55cedfa36c65c7981638733ac2ad948db0dae9de07701013b40a72')

build() { cd "${srcdir}/${_module}-${pkgver}" python setup.py build }

package() { depends+=() cd "${srcdir}/${_module}-${pkgver}" python setup.py install --root="${pkgdir}" --optimize=1 --skip-build }

jtgd commented 3 years ago

Installing with the line given by himanshudabas seems to work.

I hope this can be propagated to the Arch repository.