my8100 / scrapydweb

Web app for Scrapyd cluster management, Scrapy log analysis & visualization, Auto packaging, Timer tasks, Monitor & Alert, and Mobile UI. DEMO :point_right:
https://github.com/my8100/files
GNU General Public License v3.0
3.16k stars 565 forks source link

ImportError: Error loading shared library libstdc++.so.6 #195

Open sbhusal123 opened 2 years ago

sbhusal123 commented 2 years ago

I'm dockerizing the whole application runing scrapyd, splash and scrapydweb.

Dockerfile


FROM python:3.7-alpine

enable terminal output

ENV PYTHONUNBUFFERED 1

copy requirements file inside the docker container

copy [source] [destination]

COPY ./requirements.txt /requirements.txt

RUN apk update && apk upgrade

Add the os level dependency

RUN apk add --update --no-cache --virtual .tmp-build-deps \ g++ gcc libc-dev linux-headers musl-dev zlib \ zlib-dev openssl-dev cargo libffi-dev python3-dev \ libressl-dev libcurl libstdc++

RUN apk add --no-cache libstdc++

RUN pip install --upgrade pip RUN pip install --no-cache-dir -r /requirements.txt

Remove temporary build dependency

RUN apk del .tmp-build-deps

RUN mkdir /xyz COPY ./xyz /xyz WORKDIR /xyz

user permissions

RUN adduser -D appUser USER appUser


> requirements.txt
```txt
scrapy-splash==0.8.0
scrapyd==1.2.1
scrapyd-client==1.2.0
scrapydweb==1.4.0

docker-compose.yml

version: "3"

services:
    scrapyd:
        build:
            context: .
        depends_on:
            - splash
        volumes:
          - ./xyz:/xyz
        command: sh -c "scrapyd"
        ports:
          - "6800:6800"

    splash:
        image: scrapinghub/splash
        ports:
            - "8050:8050"

    scrapydweb:
        build:
            context: .
        depends_on:
            - scrapyd
        volumes:
          - ./xyz:/xyz
        command: sh -c "scrapydweb"
        ports:
          - "5000:5000"

Error:

scrapydweb_1  | Traceback (most recent call last):
scrapydweb_1  |   File "/usr/local/bin/scrapydweb", line 5, in <module>
scrapydweb_1  |     from scrapydweb.run import main
scrapydweb_1  |   File "/usr/local/lib/python3.7/site-packages/scrapydweb/__init__.py", line 10, in <module>
scrapydweb_1  |     from flask_compress import Compress
scrapydweb_1  |   File "/usr/local/lib/python3.7/site-packages/flask_compress/__init__.py", line 1, in <module>
scrapydweb_1  |     from .flask_compress import Compress
scrapydweb_1  |   File "/usr/local/lib/python3.7/site-packages/flask_compress/flask_compress.py", line 14, in <module>
scrapydweb_1  |     import brotli
scrapydweb_1  |   File "/usr/local/lib/python3.7/site-packages/brotli.py", line 8, in <module>
scrapydweb_1  |     import _brotli
scrapydweb_1  | ImportError: Error loading shared library libstdc++.so.6: No such file or directory (needed by /usr/local/lib/python3.7/site-packages/_brotli.cpython-37m-x86_64-linux-gnu.so)

All the other services are working fine. And i'm using default configuration for scrapydweb.