-
**Bug 描述**
例如,当 Crawlab 使用容器化部署方案是,会出现偶发性的节点异常重启事件,导致长任务异常结束。
日志里最大嫌疑人为:
panic: runtime error: invalid memory address or nil pointer dereference
signal SIGSEGV: segmentation violation code=0x1 a…
-
**Bug 描述**
crawlab和seaweedfs不在同一台机器上时,配置了CRAWLAB_FS_FILER_PROXY路径,还是会出现
Get "http://localhost:8888/logs/63183f8c1daf1288cdc1a0b8/00000000": dial tcp [::1]:8888: connect: connection refused
**配置…
-
I would like to support Crawlab by implementing [Uffizzi](https://github.com/UffizziCloud/uffizzi) preview environments.
Disclaimer: I work on [Uffizzi](https://github.com/UffizziCloud/uffizzi).
Uf…
-
**Bug 描述**
例如,当通过Web界面新建一个Scrapy类型的爬虫时, 点击Scrapy 标签,返回错误信息:
请求错误 500: http://127.0.0.1:8000/spiders/********/scrapy/settings.
2020/09/11 21:46:03 error init config error:Config File "config" Not F…
-
旧版本的节点角色管理 CRAWLAB_SERVER_MASTER 默认是 N,相对比较合理。
不知道从哪个版本开始变成默认为 Y 了(导致与说明文档不一致) ,建议退回到默认是 N,以避免不可以预期的其它麻烦。
另外并行任务线程工作方式有待优化,从开发组了解到节点会根据 CRAWLAB_TASK_WORKERS 的设置值创建相应的线程数,在每一秒同时进行轮询扫描,导致过高的 CPU 占…
-
**Bug 描述**
例如,当爬虫类型为scrapy时,在爬虫类中定义成员,执行execjs方法,出现bug。
**复现步骤**
该 Bug 复现步骤如下
1. 编写一个爬虫类, 在类中写一个类级成员属性,其为用execjs执行的结果,哪怕执行一个空字符串;
2. 节点安装execjs库:pip install pyexecjs;
3. crawlab检查代码时会报异常:请检查您的…
-
**Bug 描述**
重启crawlab后会初始化配置
**复现步骤**
1、重启docker容器
2、在登陆进去就会看到恢复了默认配置
**期望结果**
我需要如何设置才能关掉这个选项。
-
Hi guys ı have watched the autors video he says you can host more then 100 spiders on crawlab but how can we import them automatically? It's very hard to import one by one.
-
**Describe the bug**
![image](https://user-images.githubusercontent.com/2762704/114363678-3a9cce80-9bab-11eb-8b53-b397ddf88836.png)
You know what it means, it's clearly by design and cannot be d…
-
I was deploy the k8s of crawlab and after a random range of time, my k8s pod auto terminating and created new one.
The only error log I found is in the following image: