scrapinghub / scrapinghub-stack-scrapy

Software stack with latest Scrapy and updated deps
https://support.scrapinghub.com/support/solutions/articles/22000200402
BSD 3-Clause "New" or "Revised" License
60 stars 20 forks source link

Please create images based on Debian 11 Bullseye #184

Open lsli8888 opened 10 months ago

lsli8888 commented 10 months ago

Your images are based on Debian 10 Buster. I want to use your image for running the Scrapy Playwright plugin. Unfortunately, Playwright currently only supports Debian 11 Bullseye.

The issue is Debian 10 Buster doesn't have some libraries that are needed by Playwright dependencies. Here's my Dockerfile:

FROM scrapinghub/scrapinghub-stack-scrapy:2.10-latest

ENV TERM xterm
ENV SCRAPY_SETTINGS_MODULE myscrapyproject.settings

RUN mkdir -p /app
WORKDIR /app
COPY ./requirements.txt /app/requirements.txt

RUN pip install --no-cache-dir -r requirements.txt
COPY . /app
RUN python setup.py install

RUN playwright install
RUN playwright install-deps

I get the following error:

> [10/9] RUN playwright install-deps:
0.873 BEWARE: your OS is not officially supported by Playwright; installing dependencies for ubuntu20.04-x64 as a fallback.
0.873 Installing dependencies...
0.924 Get:1 http://deb.debian.org/debian buster InRelease [122 kB]
0.943 Get:2 http://deb.debian.org/debian-security buster/updates InRelease [34.8 kB]
0.944 Get:3 http://deb.debian.org/debian buster-updates InRelease [56.6 kB]
1.109 Get:4 http://deb.debian.org/debian buster/main amd64 Packages [7909 kB]
1.843 Get:5 http://deb.debian.org/debian-security buster/updates/main amd64 Packages [571 kB]
1.899 Get:6 http://deb.debian.org/debian buster-updates/main amd64 Packages [8788 B]
2.926 Fetched 8702 kB in 2s (4346 kB/s)
2.926 Reading package lists...
3.540 Reading package lists...
4.186 Building dependency tree...
4.301 Reading state information...
4.375 Package ttf-ubuntu-font-family is not available, but is referred to by another package.
4.375 This may mean that the package is missing, has been obsoleted, or
4.375 is only available from another source
4.375 
4.375 Package libjpeg-turbo8 is not available, but is referred to by another package.
4.375 This may mean that the package is missing, has been obsoleted, or
4.375 is only available from another source
4.375 
4.426 E: Package 'ttf-ubuntu-font-family' has no installation candidate
4.426 E: Unable to locate package libenchant-2-2
4.426 E: Unable to locate package libicu66
4.426 E: Package 'libjpeg-turbo8' has no installation candidate
4.426 E: Unable to locate package libvpx6
4.426 E: Unable to locate package libevent-2.1-7
4.426 E: Couldn't find any package by glob 'libevent-2.1-7'
4.426 E: Couldn't find any package by regex 'libevent-2.1-7'
4.428 Failed to install browser dependencies
4.428 Error: Installation process exited with code: 100
------
Dockerfile:26
--------------------
  24 |     # RUN apt-get update -y
  25 |     RUN playwright install
  26 | >>> RUN playwright install-deps
  27 |     
--------------------
ERROR: failed to solve: process "/bin/sh -c playwright install-deps" did not complete successfully: exit code: 1
revilzs commented 5 months ago

I was able to workaround the playwright installation with:

RUN . /etc/os-release && \
    echo "deb http://ftp.us.debian.org/debian $VERSION_CODENAME main non-free" >> /etc/apt/sources.list.d/fonts.list