search

crawlab-team / crawlab

Distributed web crawler admin platform for spiders management regardless of languages and frameworks. 分布式爬虫管理平台，支持任何语言和框架
https://www.crawlab.cn
BSD 3-Clause "New" or "Revised" License
11.38k stars 1.8k forks source link

CRAWLAB_TASK_HANDLER_MAXRUNNERS is not effect #1288

Closed thachnv92 closed 1 year ago

thachnv92 commented 1 year ago

Describe the bug CRAWLAB_TASK_HANDLER_MAXRUNNERS | Max number of task runners (concurrent spider tasks) that a node can run | 16 | 8

I want to set CRAWLAB_TASK_HANDLER_MAXRUNNERS = 0 for Crawlab master, due to I don't want to run task on Crawlab master

To Reproduce Steps to reproduce the behavior: Update CRAWLAB_TASK_HANDLER_MAXRUNNERS = 0

Expected behavior Max runner on Crawlab master is 0

tikazyq commented 1 year ago

Have you tried disabling the master node?

image
thachnv92 commented 1 year ago

I try it. But when master node die, other master node replace it. It stills enable with 8 runners.

tikazyq commented 1 year ago

Can you put your config file docker-compose.yaml without sensitive info?

thachnv92 commented 1 year ago

I use manifest K8s

tikazyq commented 1 year ago

You can send the YAML config as well