search

adithya-s-k / marker-api

Easily deployable 🚀 API to convert PDF to markdown quickly with high accuracy.
GNU General Public License v3.0
744 stars 72 forks source link

Docker GPU install Error #20

Open ediaz-biamp opened 1 day ago

ediaz-biamp commented 1 day ago

Hey I'm having trouble installing the docker GPU image. I tried it yesterday and it was fine, but I had to re-install due to some compatibility errors with Obsidian. Today, it won't install at all... Here's the error info, please let me know how to fix. Thanks!

[+] Building 152.1s (12/12) FINISHED docker:desktop-linux => [internal] load build definition from Dockerfile.gpu.server 0.0s => => transferring dockerfile: 1.92kB 0.0s => [internal] load metadata for docker.io/nvidia/cuda:11.8.0-cudnn8-devel-ubuntu22.04 0.4s => [auth] nvidia/cuda:pull token for registry-1.docker.io 0.0s => [internal] load .dockerignore 0.0s => => transferring context: 2B 0.0s => [1/7] FROM docker.io/nvidia/cuda:11.8.0-cudnn8-devel-ubuntu22.04@sha256:8f9dd0d09d3ad3900357a1cf7f887888b5b74056636cd6ef03c160c3cd4b1d95 0.0s => => resolve docker.io/nvidia/cuda:11.8.0-cudnn8-devel-ubuntu22.04@sha256:8f9dd0d09d3ad3900357a1cf7f887888b5b74056636cd6ef03c160c3cd4b1d95 0.0s => [internal] load build context 1.3s => => transferring context: 79.93MB 1.3s => CACHED [2/7] RUN apt-get update && apt-get install -y --no-install-recommends wget curl unzip git python3 python3-pip libgl1 libglib2.0-0 0.0s => [3/7] RUN pip3 install --no-cache-dir torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118 89.6s => [4/7] WORKDIR /app 0.1s => [5/7] COPY ../ ./ 0.1s => [6/7] RUN pip3 install --no-cache-dir -e . 35.0s => ERROR [7/7] RUN python -c 'from marker.models import load_all_models; load_all_models()' 25.6s

[7/7] RUN python -c 'from marker.models import load_all_models; load_all_models()': 25.02 Traceback (most recent call last): 25.02 File "", line 1, in 25.02 File "/usr/local/lib/python3.10/dist-packages/marker/models.py", line 74, in load_all_models 25.02 order = setup_order_model(device, dtype) 25.02 File "/usr/local/lib/python3.10/dist-packages/marker/models.py", line 61, in setup_order_model 25.02 model = load_order_model() 25.02 File "/usr/local/lib/python3.10/dist-packages/surya/model/ordering/model.py", line 27, in load_model 25.02 model = OrderVisionEncoderDecoderModel.from_pretrained(checkpoint, config=config, torch_dtype=dtype) 25.02 File "/usr/local/lib/python3.10/dist-packages/transformers/models/vision_encoder_decoder/modeling_vision_encoder_decoder.py", line 376, in from_pretrained 25.02 return super().from_pretrained(pretrained_model_name_or_path, *model_args, *kwargs) 25.02 File "/usr/local/lib/python3.10/dist-packages/transformers/modeling_utils.py", line 4096, in from_pretrained 25.02 model = cls(config, model_args, model_kwargs) 25.02 File "/usr/local/lib/python3.10/dist-packages/transformers/models/vision_encoder_decoder/modeling_vision_encoder_decoder.py", line 199, in init 25.02 decoder = AutoModelForCausalLM.from_config(config.decoder) 25.02 File "/usr/local/lib/python3.10/dist-packages/transformers/models/auto/auto_factory.py", line 440, in from_config 25.02 return model_class._from_config(config, kwargs) 25.02 File "/usr/local/lib/python3.10/dist-packages/transformers/modeling_utils.py", line 1543, in _fromconfig 25.02 model = cls(config, **kwargs) 25.02 File "/usr/local/lib/python3.10/dist-packages/surya/model/ordering/decoder.py", line 495, in init 25.02 self.model = MBartOrderDecoderWrapper(config) 25.02 File "/usr/local/lib/python3.10/dist-packages/surya/model/ordering/decoder.py", line 480, in init 25.02 self.decoder = MBartOrderDecoder(config) 25.02 File "/usr/local/lib/python3.10/dist-packages/surya/model/ordering/decoder.py", line 294, in init 25.02 self.layers = nn.ModuleList([MBartOrderDecoderLayer(config) for in range(config.decoderlayers)]) 25.02 File "/usr/local/lib/python3.10/dist-packages/surya/model/ordering/decoder.py", line 294, in 25.02 self.layers = nn.ModuleList([MBartOrderDecoderLayer(config) for in range(config.decoder_layers)]) 25.02 File "/usr/local/lib/python3.10/dist-packages/surya/model/ordering/decoder.py", line 209, in init 25.02 self.self_attn = MBART_ATTENTION_CLASSES[config._attn_implementation]( 25.02 KeyError: 'sdpa' 25.02 Loaded detection model vikp/surya_det3 on device cpu with dtype torch.float32 25.02 Loaded detection model vikp/surya_layout3 on device cpu with dtype torch.float32

Dockerfile.gpu.server:61

59 | RUN pip3 install --no-cache-dir -e . 60 | 61 | >>> RUN python -c 'from marker.models import load_all_models; load_all_models()' 62 | 63 | EXPOSE 8080

ERROR: failed to solve: process "/bin/sh -c python -c 'from marker.models import load_all_models; load_all_models()'" did not complete successfully: exit code: 1

View build details: docker-desktop://dashboard/build/desktop-linux/desktop-linux/rsc4z5o5u43zcoc69dk21y0b5 Unable to find image 'marker-api-gpu:latest' locally docker: Error response from daemon: pull access denied for marker-api-gpu, repository does not exist or may require 'docker login'. See 'docker run --help'.

cahya-wirawan commented 1 day ago

Try to replace "apt install python3-packaging" to "apt-get install -y --no-install-recommends python3-packaging" in Dockerfile.gpu.server. or use following simpler Dockerfile that I got from https://github.com/adithya-s-k/marker-api/blob/980bc9396e6791c283f5dac24e631bcfb9469d6f/Dockerfile:

FROM pytorch/pytorch:2.3.1-cuda11.8-cudnn8-runtime

# Update package lists and install necessary packages
RUN apt-get update \
    && apt-get install -y ffmpeg libsm6 libxext6

# Assuming you have a setup.py or requirements.txt in your project directory
COPY . /app
WORKDIR /app

# Install your Python package (assuming it has a setup.py)
RUN pip3 install --no-cache-dir -e .

# Download models used by convert
#RUN python -c 'from marker.models import load_all_models; load_all_models()'

# Expose the desired port
EXPOSE 8000

# Run the server
CMD ["uvicorn", "server:app", "--host", "0.0.0.0" ,"--port", "8000", "--workers", "4"]
ediaz-biamp commented 17 hours ago

Thanks for the quick reply. So I tried replacing that line in the Dockerfile, and I still obtained an error on install during the load_models part... I will attempt to install with the other Dockerfile now. +] Building 278.8s (12/12) FINISHED docker:desktop-linux => [internal] load build definition from Dockerfile.gpu.server 0.0s => => transferring dockerfile: 1.95kB 0.0s => [internal] load metadata for docker.io/nvidia/cuda:11.8.0-cudnn8-devel-ubuntu22.04 1.5s => [auth] nvidia/cuda:pull token for registry-1.docker.io 0.0s => [internal] load .dockerignore 0.0s => => transferring context: 2B 0.0s => CACHED [1/7] FROM docker.io/nvidia/cuda:11.8.0-cudnn8-devel-ubuntu22.04@sha256:8f9dd0d09d3ad3900357a1cf7f887888b5b74056636cd6ef03c160c3cd4b1d95 0.0s => => resolve docker.io/nvidia/cuda:11.8.0-cudnn8-devel-ubuntu22.04@sha256:8f9dd0d09d3ad3900357a1cf7f887888b5b74056636cd6ef03c160c3cd4b1d95 0.0s => [internal] load build context 1.4s => => transferring context: 79.93MB 1.4s => [2/7] RUN apt-get update && apt-get install -y --no-install-recommends wget curl unzip git python3 python3-pip libgl1 libglib2.0-0 cu 124.8s => [3/7] RUN pip3 install --no-cache-dir torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118 90.5s => [4/7] WORKDIR /app 0.1s => [5/7] COPY ../ ./ 0.2s => [6/7] RUN pip3 install --no-cache-dir -e . 36.0s => ERROR [7/7] RUN python -c 'from marker.models import load_all_models; load_all_models()' 25.7s

[7/7] RUN python -c 'from marker.models import load_all_models; load_all_models()': 24.96 Traceback (most recent call last): 24.96 File "", line 1, in 24.96 File "/usr/local/lib/python3.10/dist-packages/marker/models.py", line 74, in load_all_models 24.97 order = setup_order_model(device, dtype) 24.97 File "/usr/local/lib/python3.10/dist-packages/marker/models.py", line 61, in setup_order_model 24.97 model = load_order_model() 24.97 File "/usr/local/lib/python3.10/dist-packages/surya/model/ordering/model.py", line 27, in load_model 24.97 model = OrderVisionEncoderDecoderModel.from_pretrained(checkpoint, config=config, torch_dtype=dtype) 24.97 File "/usr/local/lib/python3.10/dist-packages/transformers/models/vision_encoder_decoder/modeling_vision_encoder_decoder.py", line 376, in from_pretrained 24.97 return super().from_pretrained(pretrained_model_name_or_path, *model_args, *kwargs) 24.97 File "/usr/local/lib/python3.10/dist-packages/transformers/modeling_utils.py", line 4096, in from_pretrained 24.97 model = cls(config, model_args, model_kwargs) 24.97 File "/usr/local/lib/python3.10/dist-packages/transformers/models/vision_encoder_decoder/modeling_vision_encoder_decoder.py", line 199, in init 24.97 decoder = AutoModelForCausalLM.from_config(config.decoder) 24.97 File "/usr/local/lib/python3.10/dist-packages/transformers/models/auto/auto_factory.py", line 440, in from_config 24.97 return model_class._from_config(config, kwargs) 24.97 File "/usr/local/lib/python3.10/dist-packages/transformers/modeling_utils.py", line 1543, in _fromconfig 24.97 model = cls(config, **kwargs) 24.97 File "/usr/local/lib/python3.10/dist-packages/surya/model/ordering/decoder.py", line 495, in init 24.97 self.model = MBartOrderDecoderWrapper(config) 24.97 File "/usr/local/lib/python3.10/dist-packages/surya/model/ordering/decoder.py", line 480, in init 24.97 self.decoder = MBartOrderDecoder(config) 24.97 File "/usr/local/lib/python3.10/dist-packages/surya/model/ordering/decoder.py", line 294, in init 24.97 self.layers = nn.ModuleList([MBartOrderDecoderLayer(config) for in range(config.decoderlayers)]) 24.97 File "/usr/local/lib/python3.10/dist-packages/surya/model/ordering/decoder.py", line 294, in 24.97 self.layers = nn.ModuleList([MBartOrderDecoderLayer(config) for in range(config.decoder_layers)]) 24.97 File "/usr/local/lib/python3.10/dist-packages/surya/model/ordering/decoder.py", line 209, in init 24.97 self.self_attn = MBART_ATTENTION_CLASSES[config._attn_implementation]( 24.97 KeyError: 'sdpa' 24.97 Loaded detection model vikp/surya_det3 on device cpu with dtype torch.float32 24.97 Loaded detection model vikp/surya_layout3 on device cpu with dtype torch.float32

Dockerfile.gpu.server:61

59 | RUN pip3 install --no-cache-dir -e . 60 | 61 | >>> RUN python -c 'from marker.models import load_all_models; load_all_models()' 62 | 63 | EXPOSE 8080

ERROR: failed to solve: process "/bin/sh -c python -c 'from marker.models import load_all_models; load_all_models()'" did not complete successfully: exit code: 1

View build details: docker-desktop://dashboard/build/desktop-linux/desktop-linux/qvja33iou5cm6lkmu3oyig3iy Unable to find image 'marker-api-gpu:latest' locally docker: Error response from daemon: pull access denied for marker-api-gpu, repository does not exist or may require 'docker login'. See 'docker run --help'.

cahya-wirawan commented 16 hours ago

Forget this Dockerfile.gpu.server, it has many bugs. Just use the Dockerfile I mentioned above.

ediaz-biamp commented 15 hours ago

So I fixed my computer's permission issues, and I just tried the new Dockerfile. It seems the problem is persisting... Any tips on solving? ERROR: failed to solve: process "/bin/sh -c python -c 'from marker.models import load_all_models; load_all_models()'" did not complete successfully: exit code: 1

cahya-wirawan commented 10 hours ago

Probably you have the issue with transformers version as mentioned in https://github.com/VikParuchuri/marker/issues/318 So, try to add transformers = "4.45.2" in pyproject.toml under [tool.poetry.dependencies] and build again.