Web app for Scrapyd cluster management, Scrapy log analysis & visualization, Auto packaging, Timer tasks, Monitor & Alert, and Mobile UI. DEMO :point_right:
scrapydweb_1 | Traceback (most recent call last):
scrapydweb_1 | File "/usr/local/bin/scrapydweb", line 5, in <module>
scrapydweb_1 | from scrapydweb.run import main
scrapydweb_1 | File "/usr/local/lib/python3.7/site-packages/scrapydweb/__init__.py", line 10, in <module>
scrapydweb_1 | from flask_compress import Compress
scrapydweb_1 | File "/usr/local/lib/python3.7/site-packages/flask_compress/__init__.py", line 1, in <module>
scrapydweb_1 | from .flask_compress import Compress
scrapydweb_1 | File "/usr/local/lib/python3.7/site-packages/flask_compress/flask_compress.py", line 14, in <module>
scrapydweb_1 | import brotli
scrapydweb_1 | File "/usr/local/lib/python3.7/site-packages/brotli.py", line 8, in <module>
scrapydweb_1 | import _brotli
scrapydweb_1 | ImportError: Error loading shared library libstdc++.so.6: No such file or directory (needed by /usr/local/lib/python3.7/site-packages/_brotli.cpython-37m-x86_64-linux-gnu.so)
All the other services are working fine. And i'm using default configuration for scrapydweb.
I'm dockerizing the whole application runing scrapyd, splash and scrapydweb.
enable terminal output
ENV PYTHONUNBUFFERED 1
copy requirements file inside the docker container
copy [source] [destination]
COPY ./requirements.txt /requirements.txt
RUN apk update && apk upgrade
Add the os level dependency
RUN apk add --update --no-cache --virtual .tmp-build-deps \ g++ gcc libc-dev linux-headers musl-dev zlib \ zlib-dev openssl-dev cargo libffi-dev python3-dev \ libressl-dev libcurl libstdc++
RUN apk add --no-cache libstdc++
RUN pip install --upgrade pip RUN pip install --no-cache-dir -r /requirements.txt
Remove temporary build dependency
RUN apk del .tmp-build-deps
RUN mkdir /xyz COPY ./xyz /xyz WORKDIR /xyz
user permissions
RUN adduser -D appUser USER appUser
All the other services are working fine. And i'm using default configuration for scrapydweb.