neverstoplearn
commented
1 year ago When I use run the web_demo.py it raise the error bellow, but when I run with cli_demo.py it works well. Is there something wrong with my configuration。
Query：why musk set the limit of twitter request times?
Output with cli_demo.py:
[System] Searching ... [System] Count of available urls: 8 [System] Fetching ... [System] Count of available fetch results: 2315866 [System] Extracting ... [System] Count of paragraphs: 136 [System] Filtering ... Reference 1: Twitter owner Elon Musk has limited the number of tweets that users can view each day — restrictions he described as an attempt to prevent unauthorized scraping of potentially valuable data from the social media platform. Reference 2: SAN FRANCISCO (AP) — Twitter owner Elon Musk has limited the number of tweets that users can view each day — restrictions he described as an attempt to prevent unauthorized scraping of potentially valuable data from the social media platform. Reference 3: The Tesla and SpaceX CEO, who is executive chairman and CTO of Twitter, said the limits are temporary, but verified accounts will be able to read 8,000 posts per day, unverified accounts will be able to read 800 posts per day and new unverified accounts will be limited to reading 400 posts per day. Reference 4: FILE - Elon Musk, who owns Twitter, Tesla and SpaceX, speaks at the Vivatech fair, June 16, 2023, in Paris. Thousands of people logged complaints about problems accessing Twitter on Saturday, July 1, after Musk limited most users to viewing 600 tweets a day — restrictions he described as an attempt to prevent unauthorized scraping of potentially valuable data from the site. (AP Photo/Michel Euler, File) Reference 5: FILE - Elon Musk, who owns Twitter, Tesla and SpaceX, speaks at the Vivatech fair, June 16, 2023, in Paris. Thousands of people logged complaints about problems accessing Twitter on Saturday, July 1, after Musk limited most users to viewing 600 tweets a day — restrictions he described as an attempt to prevent unauthorized scraping of potentially valuable data from the site. (AP Photo/Michel Euler, File)
Elon Musk set the limit of Twitter request times in order to prevent unauthorized scraping of potentially valuable data from the social media platform[1][2].The limits are temporary, with verified accounts being able to read 8,000 posts per day, unverified accounts being able to read 800 posts per day, and new unverified accounts being limited to reading 400 posts per day.[3]
=============================================
Output with web_demo.py
[System] Searching ... [System] Count of available urls: 8 [System] Fetching ... Task exception was never retrieved future: <Task finished name='Task-111' coro=<Connection.run() done, defined at /opt/miniconda39/lib/python3.9/site-packages/playwright/_impl/_connection.py:264> exception=NotImplementedError()> Traceback (most recent call last): File "/opt/miniconda39/lib/python3.9/site-packages/playwright/_impl/_connection.py", line 271, in run await self._transport.connect() File "/opt/miniconda39/lib/python3.9/site-packages/playwright/_impl/_transport.py", line 127, in connect raise exc File "/opt/miniconda39/lib/python3.9/site-packages/playwright/_impl/_transport.py", line 116, in connect self._proc = await asyncio.create_subprocess_exec( File "/opt/miniconda39/lib/python3.9/asyncio/subprocess.py", line 236, in create_subprocess_exec transport, protocol = await loop.subprocess_exec( File "/opt/miniconda39/lib/python3.9/asyncio/base_events.py", line 1676, in subprocess_exec transport = await self._make_subprocess_transport( File "/opt/miniconda39/lib/python3.9/asyncio/unix_events.py", line 188, in _make_subprocess_transport with events.get_child_watcher() as watcher: File "/opt/miniconda39/lib/python3.9/asyncio/events.py", line 766, in get_child_watcher return get_event_loop_policy().get_child_watcher() File "/opt/miniconda39/lib/python3.9/asyncio/events.py", line 602, in get_child_watcher raise NotImplementedError NotImplementedError Traceback (most recent call last): File "/opt/miniconda39/lib/python3.9/site-packages/gradio/routes.py", line 422, in run_predict output = await app.get_blocks().process_api( File "/opt/miniconda39/lib/python3.9/site-packages/gradio/blocks.py", line 1323, in process_api result = await self.call_function( File "/opt/miniconda39/lib/python3.9/site-packages/gradio/blocks.py", line 1067, in call_function prediction = await utils.async_iteration(iterator) File "/opt/miniconda39/lib/python3.9/site-packages/gradio/utils.py", line 336, in async_iteration return await iterator.anext() File "/opt/miniconda39/lib/python3.9/site-packages/gradio/utils.py", line 329, in anext return await anyio.to_thread.run_sync( File "/opt/miniconda39/lib/python3.9/site-packages/anyio/to_thread.py", line 31, in run_sync return await get_asynclib().run_sync_in_worker_thread( File "/opt/miniconda39/lib/python3.9/site-packages/anyio/_backends/_asyncio.py", line 937, in run_sync_in_worker_thread return await future File "/opt/miniconda39/lib/python3.9/site-packages/anyio/_backends/_asyncio.py", line 867, in run result = context.run(func, *args) File "/opt/miniconda39/lib/python3.9/site-packages/gradio/utils.py", line 312, in run_sync_iterator_async return next(iterator) File "/train/LLMs/webGLM/WebGLM/web_demo.py", line 49, in query for resp in webglm.stream_query(query): File "/train/LLMs/webGLM/WebGLM/model/modeling_webglm.py", line 43, in stream_query refs = self.ref_retriever.query(question) File "/train/LLMs/webGLM/WebGLM/model/retriever/init.py", line 27, in query fetch_results = self.fetcher.fetch(urls) File "/train/LLMs/webGLM/WebGLM/model/retriever/fetching/init.py", line 27, in fetch self.loop.run_until_complete(get_raw_pages(urls, close_browser=True)) File "/opt/miniconda39/lib/python3.9/asyncio/base_events.py", line 647, in run_until_complete return future.result() File "/train/LLMs/webGLM/WebGLM/model/retriever/fetching/playwright_based_crawl_new.py", line 50, in get_raw_pages context = await get_conetent() File "/train/LLMs/webGLM/WebGLM/model/retriever/fetching/playwright_based_crawl_new.py", line 27, in get_conetent playwright = await async_playwright().start() File "/opt/miniconda39/lib/python3.9/site-packages/playwright/async_api/_context_manager.py", line 52, in start return await self.aenter() File "/opt/miniconda39/lib/python3.9/site-packages/playwright/async_api/_context_manager.py", line 47, in aenter playwright = AsyncPlaywright(next(iter(done)).result()) File "/opt/miniconda39/lib/python3.9/site-packages/playwright/_impl/_connection.py", line 271, in run await self._transport.connect() File "/opt/miniconda39/lib/python3.9/site-packages/playwright/_impl/_transport.py", line 127, in connect raise exc File "/opt/miniconda39/lib/python3.9/site-packages/playwright/_impl/_transport.py", line 116, in connect self._proc = await asyncio.create_subprocess_exec( File "/opt/miniconda39/lib/python3.9/asyncio/subprocess.py", line 236, in create_subprocess_exec transport, protocol = await loop.subprocess_exec( File "/opt/miniconda39/lib/python3.9/asyncio/base_events.py", line 1676, in subprocess_exec transport = await self._make_subprocess_transport( File "/opt/miniconda39/lib/python3.9/asyncio/unix_events.py", line 188, in _make_subprocess_transport with events.get_child_watcher() as watcher: File "/opt/miniconda39/lib/python3.9/asyncio/events.py", line 766, in get_child_watcher return get_event_loop_policy().get_child_watcher() File "/opt/miniconda39/lib/python3.9/asyncio/events.py", line 602, in get_child_watcher raise NotImplementedError NotImplementedError
请问你解决了吗
dittops
When I use run the web_demo.py it raise the error bellow, but when I run with cli_demo.py it works well. Is there something wrong with my configuration。
Query：why musk set the limit of twitter request times?
Output with cli_demo.py:
[System] Searching ... [System] Count of available urls: 8 [System] Fetching ... [System] Count of available fetch results: 2315866 [System] Extracting ... [System] Count of paragraphs: 136 [System] Filtering ... Reference 1: Twitter owner Elon Musk has limited the number of tweets that users can view each day — restrictions he described as an attempt to prevent unauthorized scraping of potentially valuable data from the social media platform. Reference 2: SAN FRANCISCO (AP) — Twitter owner Elon Musk has limited the number of tweets that users can view each day — restrictions he described as an attempt to prevent unauthorized scraping of potentially valuable data from the social media platform. Reference 3: The Tesla and SpaceX CEO, who is executive chairman and CTO of Twitter, said the limits are temporary, but verified accounts will be able to read 8,000 posts per day, unverified accounts will be able to read 800 posts per day and new unverified accounts will be limited to reading 400 posts per day. Reference 4: FILE - Elon Musk, who owns Twitter, Tesla and SpaceX, speaks at the Vivatech fair, June 16, 2023, in Paris. Thousands of people logged complaints about problems accessing Twitter on Saturday, July 1, after Musk limited most users to viewing 600 tweets a day — restrictions he described as an attempt to prevent unauthorized scraping of potentially valuable data from the site. (AP Photo/Michel Euler, File) Reference 5: FILE - Elon Musk, who owns Twitter, Tesla and SpaceX, speaks at the Vivatech fair, June 16, 2023, in Paris. Thousands of people logged complaints about problems accessing Twitter on Saturday, July 1, after Musk limited most users to viewing 600 tweets a day — restrictions he described as an attempt to prevent unauthorized scraping of potentially valuable data from the site. (AP Photo/Michel Euler, File)
Elon Musk set the limit of Twitter request times in order to prevent unauthorized scraping of potentially valuable data from the social media platform[1][2].The limits are temporary, with verified accounts being able to read 8,000 posts per day, unverified accounts being able to read 800 posts per day, and new unverified accounts being limited to reading 400 posts per day.[3]
=============================================
Output with web_demo.py
[System] Searching ... [System] Count of available urls: 8 [System] Fetching ... Task exception was never retrieved future: <Task finished name='Task-111' coro=<Connection.run() done, defined at /opt/miniconda39/lib/python3.9/site-packages/playwright/_impl/_connection.py:264> exception=NotImplementedError()> Traceback (most recent call last): File "/opt/miniconda39/lib/python3.9/site-packages/playwright/_impl/_connection.py", line 271, in run await self._transport.connect() File "/opt/miniconda39/lib/python3.9/site-packages/playwright/_impl/_transport.py", line 127, in connect raise exc File "/opt/miniconda39/lib/python3.9/site-packages/playwright/_impl/_transport.py", line 116, in connect self._proc = await asyncio.create_subprocess_exec( File "/opt/miniconda39/lib/python3.9/asyncio/subprocess.py", line 236, in create_subprocess_exec transport, protocol = await loop.subprocess_exec( File "/opt/miniconda39/lib/python3.9/asyncio/base_events.py", line 1676, in subprocess_exec transport = await self._make_subprocess_transport( File "/opt/miniconda39/lib/python3.9/asyncio/unix_events.py", line 188, in _make_subprocess_transport with events.get_child_watcher() as watcher: File "/opt/miniconda39/lib/python3.9/asyncio/events.py", line 766, in get_child_watcher return get_event_loop_policy().get_child_watcher() File "/opt/miniconda39/lib/python3.9/asyncio/events.py", line 602, in get_child_watcher raise NotImplementedError NotImplementedError Traceback (most recent call last): File "/opt/miniconda39/lib/python3.9/site-packages/gradio/routes.py", line 422, in run_predict output = await app.get_blocks().process_api( File "/opt/miniconda39/lib/python3.9/site-packages/gradio/blocks.py", line 1323, in process_api result = await self.call_function( File "/opt/miniconda39/lib/python3.9/site-packages/gradio/blocks.py", line 1067, in call_function prediction = await utils.async_iteration(iterator) File "/opt/miniconda39/lib/python3.9/site-packages/gradio/utils.py", line 336, in async_iteration return await iterator.anext() File "/opt/miniconda39/lib/python3.9/site-packages/gradio/utils.py", line 329, in anext return await anyio.to_thread.run_sync( File "/opt/miniconda39/lib/python3.9/site-packages/anyio/to_thread.py", line 31, in run_sync return await get_asynclib().run_sync_in_worker_thread( File "/opt/miniconda39/lib/python3.9/site-packages/anyio/_backends/_asyncio.py", line 937, in run_sync_in_worker_thread return await future File "/opt/miniconda39/lib/python3.9/site-packages/anyio/_backends/_asyncio.py", line 867, in run result = context.run(func, *args) File "/opt/miniconda39/lib/python3.9/site-packages/gradio/utils.py", line 312, in run_sync_iterator_async return next(iterator) File "/train/LLMs/webGLM/WebGLM/web_demo.py", line 49, in query for resp in webglm.stream_query(query): File "/train/LLMs/webGLM/WebGLM/model/modeling_webglm.py", line 43, in stream_query refs = self.ref_retriever.query(question) File "/train/LLMs/webGLM/WebGLM/model/retriever/init.py", line 27, in query fetch_results = self.fetcher.fetch(urls) File "/train/LLMs/webGLM/WebGLM/model/retriever/fetching/init.py", line 27, in fetch self.loop.run_until_complete(get_raw_pages(urls, close_browser=True)) File "/opt/miniconda39/lib/python3.9/asyncio/base_events.py", line 647, in run_until_complete return future.result() File "/train/LLMs/webGLM/WebGLM/model/retriever/fetching/playwright_based_crawl_new.py", line 50, in get_raw_pages context = await get_conetent() File "/train/LLMs/webGLM/WebGLM/model/retriever/fetching/playwright_based_crawl_new.py", line 27, in get_conetent playwright = await async_playwright().start() File "/opt/miniconda39/lib/python3.9/site-packages/playwright/async_api/_context_manager.py", line 52, in start return await self.aenter() File "/opt/miniconda39/lib/python3.9/site-packages/playwright/async_api/_context_manager.py", line 47, in aenter playwright = AsyncPlaywright(next(iter(done)).result()) File "/opt/miniconda39/lib/python3.9/site-packages/playwright/_impl/_connection.py", line 271, in run await self._transport.connect() File "/opt/miniconda39/lib/python3.9/site-packages/playwright/_impl/_transport.py", line 127, in connect raise exc File "/opt/miniconda39/lib/python3.9/site-packages/playwright/_impl/_transport.py", line 116, in connect self._proc = await asyncio.create_subprocess_exec( File "/opt/miniconda39/lib/python3.9/asyncio/subprocess.py", line 236, in create_subprocess_exec transport, protocol = await loop.subprocess_exec( File "/opt/miniconda39/lib/python3.9/asyncio/base_events.py", line 1676, in subprocess_exec transport = await self._make_subprocess_transport( File "/opt/miniconda39/lib/python3.9/asyncio/unix_events.py", line 188, in _make_subprocess_transport with events.get_child_watcher() as watcher: File "/opt/miniconda39/lib/python3.9/asyncio/events.py", line 766, in get_child_watcher return get_event_loop_policy().get_child_watcher() File "/opt/miniconda39/lib/python3.9/asyncio/events.py", line 602, in get_child_watcher raise NotImplementedError NotImplementedError