twintproject / twint

An advanced Twitter scraping & OSINT tool written in Python that doesn't use Twitter's API, allowing you to scrape a user's followers, following, Tweets and more while evading most API limitations.
MIT License
15.69k stars 2.71k forks source link

Cannot connect to host twitter.com:443 #1097

Open Lirsakura opened 3 years ago

Lirsakura commented 3 years ago
(py36) C:\Users\Admin>twint -u beonyeok
CRITICAL:root:twint.get:User:Cannot connect to host twitter.com:443 ssl:True [Connect call failed ('2001::1f0d:45a0', 443, 0, 0)]
Traceback (most recent call last):
  File "d:\programdata\miniconda3\envs\py36\lib\site-packages\aiohttp\connector.py", line 936, in _wrap_create_connection
    return await self._loop.create_connection(*args, **kwargs)  # type: ignore  # noqa
  File "d:\programdata\miniconda3\envs\py36\lib\asyncio\base_events.py", line 798, in create_connection
    raise exceptions[0]
  File "d:\programdata\miniconda3\envs\py36\lib\asyncio\base_events.py", line 785, in create_connection
    yield from self.sock_connect(sock, address)
  File "d:\programdata\miniconda3\envs\py36\lib\asyncio\selector_events.py", line 439, in sock_connect
    return (yield from fut)
  File "d:\programdata\miniconda3\envs\py36\lib\asyncio\selector_events.py", line 469, in _sock_connect_cb
    raise OSError(err, 'Connect call failed %s' % (address,))
TimeoutError: [Errno 10060] Connect call failed ('2001::45ab:ea16', 443, 0, 0)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "d:\programdata\miniconda3\envs\py36\lib\runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "d:\programdata\miniconda3\envs\py36\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "D:\ProgramData\Miniconda3\envs\py36\Scripts\twint.exe\__main__.py", line 7, in <module>
  File "d:\programdata\miniconda3\envs\py36\lib\site-packages\twint\cli.py", line 305, in run_as_command
    main()
  File "d:\programdata\miniconda3\envs\py36\lib\site-packages\twint\cli.py", line 297, in main
    run.Search(c)
  File "d:\programdata\miniconda3\envs\py36\lib\site-packages\twint\run.py", line 327, in Search
    run(config, callback)
  File "d:\programdata\miniconda3\envs\py36\lib\site-packages\twint\run.py", line 226, in run
    get_event_loop().run_until_complete(Twint(config).main(callback))
  File "d:\programdata\miniconda3\envs\py36\lib\asyncio\base_events.py", line 488, in run_until_complete
    return future.result()
  File "d:\programdata\miniconda3\envs\py36\lib\site-packages\twint\run.py", line 154, in main
    await task
  File "d:\programdata\miniconda3\envs\py36\lib\site-packages\twint\run.py", line 199, in run
    await self.tweets()
  File "d:\programdata\miniconda3\envs\py36\lib\site-packages\twint\run.py", line 137, in tweets
    await self.Feed()
  File "d:\programdata\miniconda3\envs\py36\lib\site-packages\twint\run.py", line 45, in Feed
    response = await get.RequestUrl(self.config, self.init, headers=[("User-Agent", self.user_agent)])
  File "d:\programdata\miniconda3\envs\py36\lib\site-packages\twint\get.py", line 119, in RequestUrl
    response = await Request(_url, params=params, connector=_connector, headers=headers)
  File "d:\programdata\miniconda3\envs\py36\lib\site-packages\twint\get.py", line 143, in Request
    return await Response(session, url, params)
  File "d:\programdata\miniconda3\envs\py36\lib\site-packages\twint\get.py", line 148, in Response
    async with session.get(url, ssl=True, params=params, proxy=httpproxy) as response:
  File "d:\programdata\miniconda3\envs\py36\lib\site-packages\aiohttp\client.py", line 1012, in __aenter__
    self._resp = await self._coro
  File "d:\programdata\miniconda3\envs\py36\lib\site-packages\aiohttp\client.py", line 483, in _request
    timeout=real_timeout
  File "d:\programdata\miniconda3\envs\py36\lib\site-packages\aiohttp\connector.py", line 523, in connect
    proto = await self._create_connection(req, traces, timeout)
  File "d:\programdata\miniconda3\envs\py36\lib\site-packages\aiohttp\connector.py", line 859, in _create_connection
    req, traces, timeout)
  File "d:\programdata\miniconda3\envs\py36\lib\site-packages\aiohttp\connector.py", line 1004, in _create_direct_connection
    raise last_exc
  File "d:\programdata\miniconda3\envs\py36\lib\site-packages\aiohttp\connector.py", line 986, in _create_direct_connection
    req=req, client_error=client_error)
  File "d:\programdata\miniconda3\envs\py36\lib\site-packages\aiohttp\connector.py", line 943, in _wrap_create_connection
    raise client_error(req.connection_key, exc) from exc
aiohttp.client_exceptions.ClientConnectorError: Cannot connect to host twitter.com:443 ssl:True [Connect call failed ('2001::45ab:ea16', 443, 0, 0)]

can you tell me how to deal with this problem??

innocentius commented 3 years ago

You are providing too little information. Are you using IPv6 though?

Lirsakura commented 3 years ago

You are providing too little information. Are you using IPv6 though?

I am not using IPV6 and one more question: Suppose my server IP is 125.xxx.xxx.xxx, port is 1234 based on Socks5. Is the configuration in the Config.py file like this? Proxy_host: str = "125.xxx.xxx.xxx" Proxy_port: int = 1234 Proxy_type: object = ‘socks5’

innocentius commented 3 years ago

That looks correct to me. It looks to me that you are from China. Are you behind the GFW when running this? There is a step in the retrival (guest token retrival) in the current version of twint that is NOT using the proxy you provided, so this could be a reason.

Lirsakura commented 3 years ago

That looks correct to me. It looks to me that you are from China. Are you behind the GFW when running this? There is a step in the retrival (guest token retrival) in the current version of twint that is NOT using the proxy you provided, so this could be a reason.

I solve this problem,and can you tell why return is NoneType? CRITICAL:root:twint.get:User:'NoneType' object is not subscriptable CRITICAL:root:twint.run:Twint:Feed:noDataExpecting value: line 1 column 1 (char 0)

innocentius commented 3 years ago

Search the related issue to find an answer.

cantabile-kwok commented 3 years ago

That looks correct to me. It looks to me that you are from China. Are you behind the GFW when running this? There is a step in the retrival (guest token retrival) in the current version of twint that is NOT using the proxy you provided, so this could be a reason.

I solve this problem,and can you tell why return is NoneType? CRITICAL:root:twint.get:User:'NoneType' object is not subscriptable CRITICAL:root:twint.run:Twint:Feed:noDataExpecting value: line 1 column 1 (char 0)

Can you tell me how you solved it?

innocentius commented 3 years ago

That looks correct to me. It looks to me that you are from China. Are you behind the GFW when running this? There is a step in the retrival (guest token retrival) in the current version of twint that is NOT using the proxy you provided, so this could be a reason.

I solve this problem,and can you tell why return is NoneType? CRITICAL:root:twint.get:User:'NoneType' object is not subscriptable CRITICAL:root:twint.run:Twint:Feed:noDataExpecting value: line 1 column 1 (char 0)

Can you tell me how you solved it?

You want a way to send all traffics through the proxy tunnel. This is usually achieved through using a global routing policy (so no proxy setting in the program, but proxy all your traffics with your PC), or by using a VPN. Alternatively you could change the code of Twint to make sure the token retrival step would also use an ip address outside the GFW.

realQuiet commented 3 years ago

That looks correct to me. It looks to me that you are from China. Are you behind the GFW when running this? There is a step in the retrival (guest token retrival) in the current version of twint that is NOT using the proxy you provided, so this could be a reason.

I solve this problem,and can you tell why return is NoneType? CRITICAL:root:twint.get:User:'NoneType' object is not subscriptable CRITICAL:root:twint.run:Twint:Feed:noDataExpecting value: line 1 column 1 (char 0)

That looks correct to me. It looks to me that you are from China. Are you behind the GFW when running this? There is a step in the retrival (guest token retrival) in the current version of twint that is NOT using the proxy you provided, so this could be a reason.

I solve this problem,and can you tell why return is NoneType? CRITICAL:root:twint.get:User:'NoneType' object is not subscriptable CRITICAL:root:twint.run:Twint:Feed:noDataExpecting value: line 1 column 1 (char 0)

Can you tell me how you solved it?Dude,I really need your help.

changxy5 commented 3 years ago

That looks correct to me. It looks to me that you are from China. Are you behind the GFW when running this? There is a step in the retrival (guest token retrival) in the current version of twint that is NOT using the proxy you provided, so this could be a reason.

I solve this problem,and can you tell why return is NoneType? CRITICAL:root:twint.get:User:'NoneType' object is not subscriptable CRITICAL:root:twint.run:Twint:Feed:noDataExpecting value: line 1 column 1 (char 0)

hi can you tell me how did you sovle this problem?

h4m5t commented 3 years ago

That looks correct to me. It looks to me that you are from China. Are you behind the GFW when running this? There is a step in the retrival (guest token retrival) in the current version of twint that is NOT using the proxy you provided, so this could be a reason.

I solve this problem,and can you tell why return is NoneType? CRITICAL:root:twint.get:User:'NoneType' object is not subscriptable CRITICAL:root:twint.run:Twint:Feed:noDataExpecting value: line 1 column 1 (char 0)

I don not know how to solve it ,ai.

handeserve commented 3 years ago

你好,我遇到的问题和你一样,用了代理无法连接网络,能问下你是如何解决的吗?谢谢

h4m5t commented 3 years ago

你好,我遇到的问题和你一样,用了代理无法连接网络,能问下你是如何解决的吗?谢谢

I don not know how to solve it,too.

lichuana1 commented 3 years ago

我现在到和题主描述的这个阶段了 image 第一个问题我尝试了这些方法 image 1、从SSR换成了SStap的全局规则 image 2、Config.py中修改成这样 image 我不知道哪个是解决方法,但我更倾向于第一个,希望对你们有帮助 我现在在解决第二个问题

lichuana1 commented 3 years ago

我现在到和题主描述的这个阶段了 image 第一个问题我尝试了这些方法 image 1、从SSR换成了SStap的全局规则 image 2、Config.py中修改成这样 image 我不知道哪个是解决方法,但我更倾向于第一个,希望对你们有帮助 我现在在解决第二个问题

第二个问题的解决方法: 运行这个命令:pip3 install --user --upgrade git+https://github.com/twintproject/twint.git@origin/master#egg=twint 安装的过程中,我遇到了以下错误: image 使用这个命令解决了该问题:conda install git 成功后显示该界面:Successfully installed aiodns-3.0.0 aiohttp-3.7.4.post0 aiohttp-socks-0.6.0 async-timeout-3.0.1 cchardet-2.1.7 elasticsearch-7.13.0 fake-useragent-0.1.11 geographiclib-1.50 geopy-2.1.0 googletransx-2.4.2 multidict-5.1.0 pycares-4.0.0 python-socks-1.2.4 schedule-1.1.0 twint-2.1.21 yarl-1.6.3 然后就没有问题了。 一个成功的示例: in: twint -u -Heena1Zhou out: image 希望对你们有帮助

gigberg commented 3 years ago

我现在到和题主描述的这个阶段了 image 第一个问题我尝试了这些方法 image 1、从SSR换成了SStap的全局规则 image 2、Config.py中修改成这样 image 我不知道哪个是解决方法,但我更倾向于第一个,希望对你们有帮助 我现在在解决第二个问题

第二个问题的解决方法: 运行这个命令:pip3 install --user --upgrade git+https://github.com/twintproject/twint.git@origin/master#egg=twint 安装的过程中,我遇到了以下错误: image 使用这个命令解决了该问题:conda install git 成功后显示该界面:Successfully installed aiodns-3.0.0 aiohttp-3.7.4.post0 aiohttp-socks-0.6.0 async-timeout-3.0.1 cchardet-2.1.7 elasticsearch-7.13.0 fake-useragent-0.1.11 geographiclib-1.50 geopy-2.1.0 googletransx-2.4.2 multidict-5.1.0 pycares-4.0.0 python-socks-1.2.4 schedule-1.1.0 twint-2.1.21 yarl-1.6.3 然后就没有问题了。 一个成功的示例: in: twint -u -Heena1Zhou out: image 希望对你们有帮助

谢谢!sstap的代理很有效

YujiaDuan commented 3 years ago

我现在到和题主描述的这个阶段了 image 第一个问题我尝试了这些方法 image 1、从SSR换成了SStap的全局规则 image 2、Config.py中修改成这样 image 我不知道哪个是解决方法,但我更倾向于第一个,希望对你们有帮助 我现在在解决第二个问题

请问sstap的节点要如何设置呢

gigberg commented 3 years ago

我现在到和题主描述的这个阶段了 image 第一个问题我尝试了这些方法 image 1、从SSR换成了SStap的全局规则 image 2、Config.py中修改成这样 image 我不知道哪个是解决方法,但我更倾向于第一个,希望对你们有帮助 我现在在解决第二个问题

请问sstap的节点要如何设置呢

先在内网机器(比如本机、桥接的虚拟机或者内网其它设备)开一个科学网的代理,然后sstap添加这个代理服务器,最后sstap设置全局规则。sstap的特点是虚拟网卡支持,等于说所有浏览走sstap虚拟网卡所提供的代理服务器。

lichuana1 commented 3 years ago

我现在到和题主描述的这个阶段了 image 第一个问题我尝试了这些方法 image 1、从SSR换成了SStap的全局规则 image 2、Config.py中修改成这样 image 我不知道哪个是解决方法,但我更倾向于第一个,希望对你们有帮助 我现在在解决第二个问题

请问sstap的节点要如何设置呢

我使用的是自己买的SSR订阅链接,有相关节点 image 如图所示,如果存在问题的话,考虑一下是不是校园网的问题,我们校园网是只能用SSR协议的,v2ray和clash都不能使用

gigberg commented 3 years ago

我现在到和题主描述的这个阶段了 image 第一个问题我尝试了这些方法 image 1、从SSR换成了SStap的全局规则 image 2、Config.py中修改成这样 image 我不知道哪个是解决方法,但我更倾向于第一个,希望对你们有帮助 我现在在解决第二个问题

请问sstap的节点要如何设置呢

我使用的是自己买的SSR订阅链接,有相关节点 image 如图所示,如果存在问题的话,考虑一下是不是校园网的问题,我们校园网是只能用SSR协议的,v2ray和clash都不能使用

你试试使用单独的ssr客户端shadowsocks,并且开启ssr代理,一般shadowsocks会提供http代理端口是10808或者10809。然后同时启动sstap,在sstap中手动添加刚才提供的http代理”127.0.0.1:端口“,sstap设置全局代理,这时也许twint能直接运行

joker1024567 commented 2 years ago

无论怎样,在这里有三个软件的设置使用:twint的代理,SSTap的代理,shadowsocks的代理,这里的设置要怎样填写正确

changxy5 commented 2 years ago

这是来自QQ邮箱的假期自动回复邮件。   您好,您的邮件我已收到,我会尽快回复,祝好。

joker1024567 commented 2 years ago

无论怎样,在这里有三个软件的设置使用:twint的代理,SSTap的代理,shadowsocks的代理,这里的设置要怎样填写正确

经过测试,对于第一种问题,如果不使用SStap,使用代理软件的端口就可以解决,twint的代理设置具体是SS或SSR的端口是1080,v2ray的端口是10808,其他的为127.0.0.1和socks5。如果使用SSTap设置全局,在SSTap填写完成后直接使用,不要再填写twint的代理设置。推荐直接设置twint的代理设置,SSTap因为停止开发,所以并不安全和方便。 通俗来讲,任何解决方案只需要用到两个软件的设置即可,但我发现,愚蠢的不是第一个BUG问题,所有人的一半都是因为第二个BUG,即: 这是一个后续版本的BUG样式 CRITICAL:root:twint.get:User:'NoneType' object is not subscriptable CRITICAL:root:twint.feed:Follow:IndexError 这是一个原始版本的BUG样式 CRITICAL:root:twint.get:User:'NoneType' object is not subscriptable CRITICAL:root:twint.run:Twint:Feed:noDataExpecting value: line 1 column 1 (char 0) 使用:pip install --upgrade git+https://github.com/twintproject/twint.git@origin/master#egg=twint成功解决 现在是第三个BUG: