-
```
import logging
from selenium import webdriver
from selenium.common.exceptions import NoSuchElementException, TimeoutException, WebDriverException
from selenium.webdriver.common.by import By
f…
-
This issue is for discussion/documentation of crawling site maps for story URL discovery.
I'm creating it in the rss-fetcher repo, since I think:
1. We will likely want the result to feed into the…
-
Make the speed modifier/boost effect crawling and pole speed.
-
Issue scraping the swift documentation:
Bug described by user on the /crawl endpoint
_Trying to scrape the documentation for Swift UI https://developer.apple.com/documentation/swiftui/ but it ke…
-
I've tried Digikey, Mouser, and LCSC, but none of them work.
Digikey brings me to what looks like a Lynx interactive for their login page. I can type in username and password, but I'm not sure why …
-
On BBOT v1.1.7.3251rc , on this site, bbot is not crawling at all.
[debug.log](https://github.com/blacklanternsecurity/bbot/files/15041363/debug.log)
-
-
- https://developers.google.com/search/docs/crawling-indexing/sitemaps/overview
- https://developers.google.com/search/docs/crawling-indexing/sitemaps/large-sitemaps
- https://developers.google.co…
-
### Describe the bug
Bedrock crawling behavior is inconsistent with Java Editon, because a Bedrock player can in certain cases, sprint-crawl. This is an issue because it can trigger anticheats falsel…
-
Description:
Enhance the existing web crawler to support crawling and extracting content from websites that rely heavily on JavaScript for rendering their content. This feature will involve integra…