中文 | English
Lite version of Crawlab, golang-based web crawler management platform, supporting crawlers in any language.
Compared with Crawlab, this lite version focuses on crawler management on a single machine, it runs independent of any external database and removes a lot of non-essential features.
:warning: This version is still in early development and some features may be unstable.
docker-compose.yml
in any directory as follows:version: '3'
services:
master:
image: zkqiang/crawlab-lite:latest
container_name: master
ports:
- "8080:8080"
volumes:
- "./data:/app/data" # persistent volume
docker-compose up -d
http://localhost:8080
git clone https://github.com/crawlab-team/crawlab-lite
cd crawlab-lite
cd backend
go run main.go
cd ../frontend
npm i && npm run serve
http://localhost:8080
Crawlab Lite | Crawlab | |
---|---|---|
Cross-language spider | ✅ | ✅ |
Multi node | ❌ | ✅ |
Cron | ✅ | ✅ |
Log output | ✅ | ✅ |
Spider version manage | ✅ | ❌ |
Analytics | ❌ | ✅ |
Notification | ❌ | ✅ |
Online code editor | ❌ | ✅ |
Configurable spider | ❌ | ✅ |
SDK | ❌ | ✅ |