-
### How are you running AnythingLLM?
Docker (local)
### What happened?
Subdomains seem to be only supported for normal TLDs.
Something like `https://confluence.myorg.at/display/SPACE/*` works.
So…
-
hykuup knapsack tenant covenant.hykuup.com, has communicated that the load times for their site are long. Please investigate.
-
### Is there an existing issue for this?
- [X] I have searched the existing issues
### Current Behavior
After a subdomain scanning I receive a multiples subdomain that not alive ,
This subdomain …
-
Follows on from discussions in #63 - currently the `HostAlias` setting is relatively limited, requiring an exact match before it crawls a link with that domain.
To make crawling a large number of s…
-
This is my idea page: https://pooriajr.com/ideas
Here's a gif showing how it doesn't work
![CleanShot 2024-02-26 at 19 50 53](https://github.com/lindylearn/aboutideasnow/assets/14811629/ebf693e5-71f…
-
Name of Crawler: ???
Settings:
- url: www.spiegel.de
- blacklist:
- sport
- dienste
- extra
- netzwelt
- karriere
- reise
- stil
- international
- follow subdomai…
-
For example, I was thinking of using this library to crawl a single site for pages.
This library looks great by the way - much higher quality than any of the other existing crawler libraries I've i…
-
Hello,
Over the past few months I've been working on an improved version of the [reconness-universal-wrapper](https://github.com/hiddengearz/reconness-universal-wrapper) I made a few months ago and…
-
Dear Sir,
Current version does not install gf, gau, waybackurls Ubuntu Or Kali Linux. It does not show installation path to add API, or other keys for censys, shodan based scripts or tools.
Seco…
-
we need to crawl many Internet sites and encountered an issue with `www` prefix:
some sites redirect to their domains without `www`, some other way round.
Unfortunately, such case cannot be handle b…