-
The [Fragmention specification](https://indieweb.org/fragmention) does not explicitly say anything about hashbang (#!) URLs or similar common (including legacy) single-page-app routing patterns, and i…
-
[2014-11-27 12:11:06,621][ERROR][org.codelibs.robot.helper.impl.LogHelperImpl] Crawling Exception at http://www.qzlab.cn/html/504/507/2014/1103/35027.html
org.seasar.framework.exception.InstantiationR…
-
Thank you for creating pyspider. This is more a documentation request I suppose. I figured out how to crawl a site; following multiple page and links on pages using multiple self.crawl statements. I f…
Amain updated
6 years ago
-
Like SQLMap for SQLi, I think Commix is the de facto standard tool for Command Injection.
I am working on a project for automated mobile application security assessment called Mobile Security Framewor…
-
This project looks really awesome =), I was just wondering if you guys can share any insight on the future roadmap for Roach?
Also, is it something that's funded by PetroFeed, and actively used by th…
-
In very specific cases, the base url website that the user inputs will redirect to a new website containing a new base url. How should we handle that?
Example input /crawl: new.abb.com/sustainabili…
-
### Problem Statement
Refit is a populair library for making requests. If an exception occurs, only the Exception Message will be sent to Sentry. The Message is a very broad description (a 400 was re…
-
Hi, there,
I am working on Frontera these days, and Frontera is a great tool for cluster crawling!
But I still find there is something not that easy to understand/figure out, because of the lack…
-
Hello. Could you help me please? I use Single Activity Architecture. I have a fragment with ViewPager. How to make the main fragment and view pager fragments served by one presenter? Thank you
-
```
To grow our wikiteam collection of wikis, I have to increase our list of wikis.
To archive our first 4500 wikis, we've used Andrew Pavlo's list. Now I want to
adapt his crawling framework (see s…