code4craft webmagic issues

code4craft / webmagic

A scalable web crawler framework for Java.

http://webmagic.io/

Apache License 2.0

11.34k stars 4.18k forks source link

issues

Newest

Newest Most commented Recently updated Oldest Least commented Least recently updated

关于Spider#setScheduler(Scheduler)存在bug

#1172 KamiNoYuki opened 2 weeks ago
0
0.10.3版本情况下，如果对方是 shtml页面，直接报错

#1171 derek-daye opened 3 weeks ago
0
调整重试逻辑

#1170 niuxiaozu opened 1 month ago
8
stopWhenComplete，增加动态修改完成时停止方法。

#1169 niuxiaozu closed 1 month ago
0
主要是修改请求间隔、错误状态码重试。按我的理解所做的小修改。

#1168 niuxiaozu closed 1 month ago
6
qq群无法加入，搜不到了，能提供一个新的群吗？感谢

#1167 sesames opened 1 month ago
0
SLF4J: No SLF4J providers were found

#1166 su1216 closed 1 month ago
1
Exception in thread "main" java.lang.NoSuchFieldError: INSTANCE

#1165 fuli66 opened 1 month ago
1
page status code error 404时, 没有进入process

#1164 PosiedChoss closed 3 weeks ago
0
如何触发重试

#1163 wangxinggui opened 2 months ago
1
请求本地服务响应报：400 Bad Request

#1162 maskainv opened 3 months ago
0
访问https form方式传参的post接口报错javax.net.ssl.SSLException: Connection reset

#1161 MindaDai opened 3 months ago
0
爬取百度百科需要耗时100多秒，请问怎么解决

#1160 yidasanqian opened 3 months ago
0
最新版本中测试用例无法运行：us.codecraft.webmagic.samples.HuabanProcessor

#1159 siqiniao opened 3 months ago
1
Refactored and implement of a template method pattern for logger config in webmagic-scripts

#1158 FrancoisGib closed 3 months ago
0
Changed refactor of processSingle again, this one is a better version

#1157 FrancoisGib closed 3 months ago
0
Changed my strategy for the refactoring of process Single and this one is a lot better

#1156 FrancoisGib closed 3 months ago
0
Refactor of processSingle in PageModelExtractor

#1155 FrancoisGib closed 3 months ago
0
Bump com.jayway.jsonpath:json-path from 2.8.0 to 2.9.0

#1154 dependabot[bot] closed 3 months ago
0
Revert "Refactored code for increased optimization."

#1153 sutra closed 3 months ago
0
Refactored Code to increase maintainability

#1152 ayushi250317 closed 3 months ago
0
Refactored Code to Resolve Implementation Code Smells

#1151 ayushi250317 closed 3 months ago
0
Added test cases for improving line and branch coverage

#1150 ayushi250317 closed 4 months ago
0
fix(sec): upgrade com.fasterxml.jackson.core:jackson-databind to

#1149 Ch3n4y closed 4 months ago
0
doCycleRetry not working when using immutable maps in extras

#1148 LeonardMeyer closed 4 months ago
0
支持socks代理

#1147 pengzhang closed 5 months ago
0
[Snyk] Security upgrade com.jayway.jsonpath:json-path from 2.8.0 to 2.9.0

#1146 code4craft closed 6 months ago
0
希望作者可以将doCycleRetry改成protect访问级别

#1145 hackeryutu opened 6 months ago
2
循环点击下一页，并设置循环结束条件

#1144 adminjohn opened 6 months ago
0
使用RedisScheduler会报找不到rpush方法,使用默认的Schduler正常。

#1143 hackeryutu closed 6 months ago
1
[Snyk] Security upgrade org.mapdb:mapdb from 3.0.10 to 3.1.0

#1142 code4craft closed 6 months ago
0
希望作者支持一下动态重试？

#1141 sparrow-ez opened 6 months ago
3
多久支持playwright

#1140 694475668 opened 7 months ago
0
Refactored code for increased optimization.

#1139 Parthgajera056 closed 3 months ago
0
Refactor addTargetRequests method to eliminate redundant code.

#1138 harikrishna553 closed 7 months ago
0
Refactored to remove multiple calls of getSourceTexts() api

#1137 harikrishna553 closed 8 months ago
0
Refactor compareLong method using Long.compare, corrected the local v…

#1136 harikrishna553 closed 8 months ago
0
如何在PageProcessor.process中,将page.getHtml()的内容传入至Xpath2Selector

#1135 Admin0x002 closed 8 months ago
1
[Snyk] Security upgrade org.seleniumhq.selenium:selenium-java from 3.141.59 to 4.14.1

#1134 code4craft closed 8 months ago
1
启动时，自定义参数放在Request后面接收不到

#1133 yangjinde opened 8 months ago
2
文档中的图片都变破图了，请修复，谢谢！

#1132 szRyu666 opened 9 months ago
1
Fix typos

#1131 maciejwalkowiak closed 9 months ago
0
如何保持cookie不过期？

#1130 getideas opened 9 months ago
0
请教怎样控制爬虫延时或者暂停？

#1129 Mr-LiuDC opened 10 months ago
3
请教怎样实现一个基于数据库的Scheduler？

#1128 Mr-LiuDC opened 10 months ago
3
fix(sec): upgrade net.sourceforge.htmlcleaner:htmlcleaner to

#1127 dack-su closed 10 months ago
0
[Snyk] Security upgrade net.sourceforge.htmlcleaner:htmlcleaner from 2.26 to 2.29

#1126 code4craft closed 11 months ago
0
WebMagic Extension version 0.9.0 has CVE-2023-2976 vulnerability

#1125 JackLinkai opened 11 months ago
0
HttpClientDownloader download error

#1124 mayn7z closed 11 months ago
1
java.lang.OutOfMemoryError: Java heap space

#1123 meikey opened 12 months ago
1