spider-rs spider issues

spider-rs / spider

A web crawler and scraper for Rust

https://spider.cloud

MIT License

1.16k stars 100 forks source link

issues

Newest

Newest Most commented Recently updated Oldest Least commented Least recently updated

Exception During Web Crawling

#229 cmauck10 closed 2 days ago
2
JSONDecodeError During Web Crawling

#228 cmauck10 closed 2 days ago
2
JSONDecodeError when crawling website with specific parameters

#227 tarujg closed 1 week ago
3
Access to reqwest internal structures ?

#226 rgrunbla opened 1 week ago
0
Delay not being respected?

#225 oco-adam closed 3 weeks ago
1
tcp keepalive issues

#224 ilpanich closed 2 weeks ago
4
Crawling tooks forever, does not end

#223 Arputikos closed 3 weeks ago
2
Use custom request library

#222 alexkreidler opened 1 month ago
2
Add CSS exclude selectors to /crawl endpoint

#219 mikoro closed 1 month ago
8
Publish spider CLI binaries

#217 alexkreidler opened 1 month ago
2
Panic with non ASCII string

#216 ronanM closed 1 month ago
1
It's colly not crolly

#214 melroy89 closed 2 months ago
2
Fix infinite loop

#213 hoagy-davis-digges closed 2 months ago
0
Retrieve crawled markdown via API

#211 culda closed 2 months ago
0
Broadcast never end when scraping with limit

#210 DimitriTimoz closed 2 months ago
2
Disable OpenSSL completely

#209 DimitriTimoz closed 2 months ago
14
Memory leak

#208 DimitriTimoz closed 2 months ago
0
Memory leak caused by hashbrown

#207 DimitriTimoz closed 2 months ago
8
Scrape with smart mode

#206 DimitriTimoz closed 2 months ago
1
Changed the callback functionality

#205 Rushmore75 closed 2 months ago
3
Help wanted: Reduce memory footprint

#204 Falumpaset closed 2 months ago
5
Update README.md

#203 James4Ever0 closed 3 months ago
0
Retrieve response cookies

#202 viktorholk closed 3 months ago
1
with_limit(1) does not work when "chrome" feature is enabled

#201 viktorholk closed 3 months ago
2
Store referring links

#199 LeoDog896 closed 3 months ago
1
Running the example code results in an error

#198 haijd closed 3 months ago
1
support file:// urls

#197 jmikedupont2 closed 2 months ago
4
Fixing clap issues #195

#196 jmikedupont2 closed 3 months ago
1
Command spider_cli: Short option names must be unique for each argument, but '-u' is in use by both 'url' and 'user_agent'

#195 jmikedupont2 closed 3 months ago
0
chore(deps): bump openssl from 0.10.64 to 0.10.66

#194 dependabot[bot] closed 4 months ago
1
CLI: download files as they arrive?

#192 gjtorikian closed 4 months ago
4
build.rs "wget" install in benches doesn't work on non-debian distros

#191 soulwa closed 4 months ago
1
Can transform work properly?

#190 ybsun0215 closed 5 months ago
1
Budget not respected

#187 CrazyDubya closed 5 months ago
1
Support COOKIE during the crawl [ENHANCEMENT]

#186 Zabrane closed 5 months ago
4
Add DEPTH level next to each debug line [ENHANCEMENT]

#185 Zabrane closed 5 months ago
3
robots.txt files are not being respected correctly

#184 div72 closed 5 months ago
6
Prebuilt binaries for Linux, macOS

#183 Zabrane closed 5 months ago
11
chore(deps): bump rustls from 0.21.10 to 0.21.11

#180 dependabot[bot] closed 7 months ago
1
docs: fix broken glob url link

#179 emilsivervik closed 7 months ago
0
chore(deps): bump h2 from 0.3.25 to 0.3.26

#178 dependabot[bot] closed 7 months ago
1
Fix typo in README file

#176 houseme closed 7 months ago
0
Is it possible to extract broken links from the crawl?

#175 metsis closed 7 months ago
6
Openai/chrome driver

#174 j-mendez closed 8 months ago
0
Already crawled URL attempted as % encoded

#172 apsaltis closed 8 months ago
3
Running with decentralized feature

#171 zmedelis closed 8 months ago
1
Is it possible to dynamicall add links to crawl?

#170 oiwn closed 8 months ago
7
chore(deps): bump mio from 0.8.10 to 0.8.11

#169 dependabot[bot] closed 8 months ago
1
Chrome flag chrome_intercept page hang.

#168 j-mendez closed 8 months ago
1
bench(perf): add local dev server for testing

#167 j-mendez closed 9 months ago
0