-
I am using scrapy-splash to scrape a youtube video page. However, it seems the response object it's not complete when I use my spider. But I got a complete result when I use the scrapy shell.
I al…
-
-
I added this to my `settings.py` but it doesn't work
```python
SPIDER_SETTINGS = [
{
'endpoint': 'dmoz',
'location': 'spiders.dmoz',
'spider': 'DmozSpider',
…
-
None
Traceback (most recent call last):
File "video_download_run.py", line 38, in
douyin_crawl.grab_user_media(sys.argv[-1], "USER_LIKE")
File "../www_douyin_com/spiders/douyin_crawl.py",…
-
Node is great and V8 is great.
But why not take Mozilla's Spider/Eon/Odin/Monkey and create a real alternative to Node? Namely something that would be async/callback Node-compatible but have the alt…
-
Hello! Trying to get BuildStream 2.0.1 running on an aarch64 platform (Orange Pi 5) is being a bit of a challenge. I'm mostly just trying to follow the [official BuildStream installation instructions…
-
(env) E:\Spider\news-spider>scrapy crawl peopleNews -a kw=关键词 -a site=people.com.cn
2020-12-21 15:28:49 [scrapy.utils.log] INFO: Scrapy 2.1.0 started (bot: news_search)
2020-12-21 15:28:49 [scrapy.u…
-
code is same as readme,and python 3.5
import asyncio
from pyppeteer import launch
async def main():
browser = await launch()
page = await browser.newPage()
await page.goto('http://…
-
### Apache Iceberg version
0.6.0 (latest release)
### Please describe the bug 🐞
For reproduction using https://github.com/apache/iceberg-python/blob/main/tests/catalog/test_glue.py
Here is faili…
-
See http://stackoverflow.com/q/38378710 for motivation.
When using a `proxy` value without scheme, e.g. 'localhost:8080', scrapy breaks with an obscure exception on `to_bytes()`. Even if it's a wrong…