crawlab-team / crawlab

Distributed web crawler admin platform for spiders management regardless of languages and frameworks. 分布式爬虫管理平台,支持任何语言和框架
https://www.crawlab.cn
BSD 3-Clause "New" or "Revised" License
11.38k stars 1.8k forks source link

settings 引入 datetime后系统json报错 #1004

Closed zhangqiang603zq closed 2 years ago

zhangqiang603zq commented 3 years ago

import datetime

BOT_NAME = 'spidertest'

SPIDER_MODULES = ['spidertest.spiders'] NEWSPIDER_MODULE = 'spidertest.spiders'

to_day = datetime.datetime.now() log_file_path = BOTNAME + "/logs/spidertest{}{}_{}.log".format(to_day.year, to_day.month, to_day.day) LOG_LEVEL = "INFO" LOG_FILE = log_file_path

上传包到平台系统报错:

http://xxx.com:8077/spiders/61404bf8c6435042811f56a7/scrapy/settings. Traceback (most recent call last): File "/usr/local/bin/crawlab", line 11, in sys.exit(main()) File "/usr/local/lib/python3.6/site-packages/crawlab/cli/init.py", line 150, in main cli() File "/usr/local/lib/python3.6/site-packages/click/core.py", line 764, in call return self.main(args, kwargs) File "/usr/local/lib/python3.6/site-packages/click/core.py", line 717, in main rv = self.invoke(ctx) File "/usr/local/lib/python3.6/site-packages/click/core.py", line 1137, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "/usr/local/lib/python3.6/site-packages/click/core.py", line 956, in invoke return ctx.invoke(self.callback, ctx.params) File "/usr/local/lib/python3.6/site-packages/click/core.py", line 555, in invoke return callback(args, **kwargs) File "/usr/local/lib/python3.6/site-packages/crawlab/cli/init.py", line 109, in settings client.settings(directory) File "/usr/local/lib/python3.6/site-packages/crawlab/core/client.py", line 195, in settings print(json.dumps(data)) File "/usr/lib64/python3.6/json/init.py", line 231, in dumps return _default_encoder.encode(obj) File "/usr/lib64/python3.6/json/encoder.py", line 199, in encode chunks = self.iterencode(o, _one_shot=True) File "/usr/lib64/python3.6/json/encoder.py", line 257, in iterencode return _iterencode(o, 0) File "/usr/lib64/python3.6/json/encoder.py", line 180, in default o.class.name) TypeError: Object of type 'module' is not JSON serializable

tikazyq commented 3 years ago

不在settings.py里写动态内容就可以,去掉import这些