hmaarrfk
commented
3 months ago How do i run a line of python code that triggers jpeg12_write_raw_data? That is still unclear to me.
Closed kevinrue closed 1 week ago
hmaarrfk
commented
3 months ago How do i run a line of python code that triggers jpeg12_write_raw_data? That is still unclear to me.
kevinrue
commented
3 months ago Thanks for the reply. Happy to provide any information I can.
I'm afraid our latest working conda environment is quite old and many dependencies have incremented many versions in the meantime, so it's hard to pinpoint what changed. I don't want to distract you with unnecessary or irrelevant details, but please ask anything that I can provide to help!
hmaarrfk
commented
3 months ago ok i'll wait for your reproducer to take any further action.
peterch405
commented
1 week ago I have been having the same issue.
! ImportError: /opt/conda/envs/scil3extfig/lib/python3.10/site-packages/PIL/../../../libtiff.so.6: undefined symbol: jpeg12_write_raw_data, version LIBJPEG_8.0My conda env with the error:
python=3.10.14 libtiff=4.6.0 pillow=10.0.1 matplotlib=3.8.4 numpy=1.26.4 cooltools=0.5.2 pairtools=1.0.3 cooler=0.9.3 bioframe=0.6.4 pygenometracks=3.3 tqdm seaborn=0.13.2 pandas=1.5.3Downgrading tolibtiff=4.5.1 and pillow=10.0.0 fixed the issue.
kevinrue
commented
1 week ago @peterch405 you sir are a star ⭐
Here's my adaptation of your fix: https://github.com/kevinrue/jurassicpark/commit/0ca8eecb909e2224546cd7e655a4baf89e17601f
Which passed github action here https://github.com/kevinrue/velociraptor/actions/runs/11051037022/job/30700081490?pr=91
Thanks a million for solving something I've struggled with for months!
hmaarrfk
commented
1 week ago @peterch405 it seems you are using bioconda, i can't really help with that unfortunately. it is likely best if we try to narrow down the problem on conda-forge first, then if it can't be reproduced, raise an issue with bioconda.
Narrowing down your recreator to things available on conda-forge:
mamba create --name test python=3.10.14 libtiff=4.6.0 pillow=10.0.1 --channel conda-forge --override-channels
mamba activate test
python -c "from PIL import Image"
mamba install python=3.10.14 libtiff=4.6.0 pillow=10.0.1 matplotlib=3.8.4 numpy=1.26.4 tqdm seaborn=0.13.2 pandas=1.5.3 --channel conda-forge --override-channels
python -c "from PIL import Image"Both seem to work on my linux machine.
If you can create a similar reproducer:
mamba env create --name test --channel conda-forge --override-channel THE_PACKAGESmamba activatepython -c "some import statement" we can resume the issue.
Otherwise I'm tempted to keep this closed :(
kevinrue
commented
1 week ago @hmaarrfk
He may have been using bioconda but thanks to his suggestion, I've built a pure conda-forge environment that solves my issue, see https://github.com/kevinrue/jurassicpark/actions/runs/11051833674/job/30702589157#step:8:38
I hope that helps!
peterch405
commented
6 days ago libtiff' andpillow' both came from conda-forge. Bioconda is only used for the other packages that aren't available from conda-forge.
The bug is probably not in conda-forge and probably not even in libtiff.
For completeness, the bug for me arises from a very specific use case and could be in reticulate, rmarkdown, or a number of other packages, I honestly don't know. To reproduce for any other poor soul to get this error:
test.Rmd (add code blocks, not sure how to include them here):
---
title: "test"
output: html_document
---
{r setup, include=FALSE}
knitr::opts_chunk$set(echo = TRUE)
library(reticulate)
use_condaenv("libtiff_test")
{python}
import seaborn as snstest_2.Rmd (add code blocks):
---
title: "test"
output: html_document
---
{r setup, include=FALSE}
knitr::opts_chunk$set(echo = TRUE)
library(reticulate)
use_condaenv("libtiff_test_2")
{python}
import seaborn as snsDockerfile:
FROM rocker/tidyverse:4.3.0
RUN apt-get update
RUN apt-get install -y \
libpng-dev
ARG MINIFORGE_NAME=Miniforge3
ARG MINIFORGE_VERSION=24.7.1-0
ARG TARGETPLATFORM
ENV CONDA_DIR=/opt/conda
ENV LANG=C.UTF-8 LC_ALL=C.UTF-8
ENV PATH=${CONDA_DIR}/bin:${PATH}
RUN apt-get update > /dev/null && \
apt-get install --no-install-recommends --yes \
wget bzip2 ca-certificates \
git \
tini \
> /dev/null && \
apt-get clean && \
rm -rf /var/lib/apt/lists/* && \
wget --no-hsts --quiet https://github.com/conda-forge/miniforge/releases/download/${MINIFORGE_VERSION}/${MINIFORGE_NAME}-${MINIFORGE_VERSION}-Linux-$(uname -m).sh -O /tmp/miniforge.sh && \
/bin/bash /tmp/miniforge.sh -b -p ${CONDA_DIR} && \
rm /tmp/miniforge.sh && \
conda clean --tarballs --index-cache --packages --yes && \
find ${CONDA_DIR} -follow -type f -name '*.a' -delete && \
find ${CONDA_DIR} -follow -type f -name '*.pyc' -delete && \
conda clean --force-pkgs-dirs --all --yes && \
echo ". ${CONDA_DIR}/etc/profile.d/conda.sh && conda activate base" >> /etc/skel/.bashrc && \
echo ". ${CONDA_DIR}/etc/profile.d/conda.sh && conda activate base" >> ~/.bashrc
RUN mamba create -n libtiff_test -y python=3.10.14 libtiff=4.5.1 pillow=10.0.0 matplotlib=3.8.4 seaborn=0.13.2 && \
mamba clean --all -y
RUN mamba create -n libtiff_test_2 -y python=3.10.14 libtiff=4.6.0 pillow=10.0.1 matplotlib=3.8.4 seaborn=0.13.2 && \
mamba clean --all -y
RUN Rscript -e "install.packages('remotes', repos = 'https://cran.rstudio.com')"
RUN Rscript -e "remotes::install_version('reticulate', version = '1.35.0', repos = 'https://cran.rstudio.com')"
COPY ./test.Rmd /test.Rmd
COPY ./test_2.Rmd /test_2.RmdWorks:
docker exec <container_name> mamba run -n libtiff_test python -c 'import seaborn'
docker exec <container_name> mamba run -n libtiff_test_2 python -c 'import seaborn'Even this works:
docker exec <container_name> Rscript -e "library(reticulate); use_condaenv('libtiff_test'); import('seaborn')"
docker exec <container_name> Rscript -e "library(reticulate); use_condaenv('libtiff_test_2'); import('seaborn')"test_2.Rmd fails:
docker exec <container_name> Rscript -e "rmarkdown::render('test.Rmd')"
docker exec <container_name> Rscript -e "rmarkdown::render('test_2.Rmd')"Full error:
Error in `py_call_impl()`:
! ImportError: /opt/conda/envs/libtiff_test_2/lib/python3.10/site-packages/PIL/../../../libtiff.so.6: undefined symbol: jpeg12_write_raw_data, version LIBJPEG_8.0
Run `reticulate::py_last_error()` for details.
Backtrace:
1. rmarkdown::render("test_2.Rmd")
2. knitr::knit(knit_input, knit_output, envir = envir, quiet = quiet)
3. knitr:::process_file(text, output)
7. knitr:::process_group.block(group)
8. knitr:::call_block(x)
...
12. knitr (local) engine(options)
13. reticulate::eng_python(options)
14. reticulate:::py_compile_eval(snippet, compile_mode)
17. builtins$eval(compiled, globals, locals)
18. reticulate:::py_call_impl(callable, call_args$unnamed, call_args$named)
Execution haltedAdditional note, I get the same error with quarto=1.5.57.
hmaarrfk
commented
6 days ago hmm i'm sorry, this is beyond me. I think you should open an other issue (just copy paste this) so that an other person can come chime in and help.
The OP closed this issue, so I think it would better help you if you opened your own....
kevinrue
commented
4 days ago Hi.
Happy to re-open this issue if it saves any duplication of effort / information.
I only closed this issue because I finally got to a combination of conda package versions that avoids the problem, and I personally gave up on trying to understand where the issue originates.
Solution to issue cannot be found in the documentation.
Issue
May I redirect you to https://github.com/python-pillow/Pillow/issues/8148 to give you all the details and some additional context without copy pasting here?
I am happy to clarify anything you need to investigate further!
Installed packages
Environment info