-
#We need to identify API endpoints for various pieces of information to completely build the dependencies for all of the courses.
### Status:
|Purpose|Endpoint |Docs/Wiki|Comments
|---|---|---|--…
-
## Summary
DNS Cache should have expire time.
## Motivation
I am using K8S to deploy scrapy to post some data to a API concurrently. However, when I update the api, the pod may change to anot…
-
2018-03-02 22:15:34 [scrapy.core.scraper] ERROR: Spider error processing (referer: https://www.safaribooksonline.com//api/v1/book/9781449342562/chapter/ch01.html)
Traceback (most recent call last):
…
-
As a scrapy user, I often want to extract the text content of an element. The default option in parsel is to either use the `::text` pseudo-element or XPath `text()`. Both options have the downside th…
-
# Why tf should I use this
Just saw this and was wondering what advantages this gives you over Scrapy's API...
-
There's no Yelp API for personal bookmarks, which is why we need Scrapy. But individual businesses are accessible in a structured way by API, with an API token. This is especially nice for figuring ou…
-
Написать list-парсер для манги https://anibel.net/manga
Тут скорее всего можно попробовать написать на обычных `request/scrapy`. Посмотрел на JS и на запросы и никаких API или JSON на странице нету
-
First of all many thanks for keeping the previous tags in Dockerhub
We run the Typesense Scanner in CI (EKS cluster in AWS with Amazon Linux nodes)
Up until 0.3.5 all our pipelines were working …
-
**Problem statement**
A typical scenario when using the Scrapy middleware to auto-extract e.g. product page URLs is that said URLs may respond with `404` status.
However, the library does not pr…
-
While working on documentation for https://docs.zyte.com/ about setting request metadata, I am starting to think that maybe we should not send `echoData` to the server, and instead keep track of it on…