-
**Describe the bug**
The worker and crawlab master can keep restarting after the upgrade.
**To Reproduce**
Steps to reproduce the behavior:
1. Using 0.5.1 crawlab setup, put the dockers down:
`…
-
I was deploy the k8s of crawlab and after a random range of time, my k8s pod auto terminating and created new one.
The only error log I found is in the following image:
-
想请教下作者，开发这个crawlab爬虫管理平台需要用到哪些技术栈或知识储备呢。来自一名学生
-
Hi,
This is a question, not a bug report.
[url-frontier](https://github.com/crawler-commons/url-frontier) is an API to define a [crawl frontier](https://en.wikipedia.org/wiki/Crawl_frontier). …
-
**Bug 描述**
当任务日志超过100页或者更多页时 自动跳转第一页，无法实时显示最新的日志
**复现步骤**
该 Bug 复现步骤如下
1.运行爬虫打印日志使其超过100页或更多页
2. 它会短暂的跳转最新页(但是输出的内容是第一页)，然后强行跳转第一页
3. 周而复始
**期望结果**
希望他能正常的显示日志
-
**Describe the bug**
I successfully implemented the Crawlab software utilizing the power of Kubernetes, consisting of a single Node Master and three Worker nodes. On a daily basis, I undertake the ri…
-
Request URL: http://10.0.0.101:8000/nodes/6039c407dea1b3001b875b47/langs
Request Method: OPTIONS
Status Code: 204 No Content
打开节点
一直有请求发送端口：8000/nodes/6039c407dea1b3001b875b47/langs 然后有状态204,200…
-
Crawlab can't download media files and large files (e.g. jpg files, mp3 files, gif files, zip files and so on.)
A viable approach is to add one type of node. Let's say, MediaWorker. The links of th…
-
**Bug 描述**
源码编译，当seaweedfs和master程序不在同一服务器部署时候，修改default_config.go里的 seaweedfs 的路径，在上传文件时候正常，但是查看 /logs/{id} 对应的日志时候发现接口还是走 localhost:8888 ，导致无法正常读取。
**复现步骤**
该 Bug 复现步骤如下
1. crawlab主程序部署在 A服务器，…
-
**Bug 描述**
例如，当采集器任务完成时数据库连接没有回收，数据库连接一直属于Sleep 状态，入库通过pyhon 的crawlab库的save_item 单采集器任务连接占用上千，通过save_items 批量写入也会占用几百的数据库连接，多采集器任务采集时，数据库连接很快就会被crawlab使用完，导致影响正常业务❌❌❌。
严重影响正常业务，希望马上解决，mongo和mysql都…