Unexpected keyword argument 'interpolate_pos_encoding'

[mdoughty-tagleaf]

After detecting bounding boxes for a few pages, the program throws the following:

TypeError: VariableDonutSwinEmbeddings.forward() got an unexpected keyword argument 'interpolate_pos_encoding'

My environment is a Debian 12 Docker container. This is my Dockerfile and compose YAML

FROM bitnami/pytorch

USER root

# Update container
RUN apt-get update
RUN apt-get upgrade -y

# Open GL
RUN apt-get install -y \
    libgl1-mesa-glx \
    libglib2.0-0
RUN rm -rf /var/lib/apt/lists/*

# Marker
RUN pip install marker-pdf

USER 1001

services:
  pdf-service:
    build:
      context: .
      dockerfile: build/PdfService.dockerfile
    tty: true
    ports:
      - "80:8484"
    volumes:
      - <dir>/cache:/app/cache
      - <dir>/src:/app/src
      - <dir>/out:/app/out
      - ${HOME}/Downloads:/app/storage
    environment:
      - HF_HOME=/app/cache
      - HOME=/app/cache
    deploy:
      resources:
        limits:
          memory: 12G
          cpus: '6'

Full console output upon testing with a single PDF

> marker_single "./storage/<file>" ./out

Loaded detection model vikp/surya_det2 on device cpu with dtype torch.float32
Loaded detection model vikp/surya_layout2 on device cpu with dtype torch.float32
Loaded reading order model vikp/surya_order on device cpu with dtype torch.float32
Loaded recognition model vikp/surya_rec on device cpu with dtype torch.float32
Loaded texify model to cpu with torch.float32 dtype
Detecting bboxes: 100%|████████████████████████████████████████████████████████████████████| 1/1 [02:29<00:00, 149.28s/it]
Detecting bboxes: 100%|█████████████████████████████████████████████████████████████████████| 1/1 [01:21<00:00, 81.39s/it]
Finding reading order:   0%|                                                                        | 0/1 [00:00<?, ?it/s]
Traceback (most recent call last):
  File "/opt/bitnami/python/bin/marker_single", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/opt/bitnami/python/lib/python3.11/site-packages/convert_single.py", line 28, in main
    full_text, images, out_meta = convert_single_pdf(fname, model_lst, max_pages=args.max_pages, langs=langs, batch_multiplier=args.batch_multiplier, start_page=args.start_page)
                                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/bitnami/python/lib/python3.11/site-packages/marker/convert.py", line 113, in convert_single_pdf
    surya_order(doc, pages, order_model, batch_multiplier=batch_multiplier)
  File "/opt/bitnami/python/lib/python3.11/site-packages/marker/layout/order.py", line 33, in surya_order
    order_results = batch_ordering(images, bboxes, order_model, processor, batch_size=int(get_batch_size() * batch_multiplier))
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/bitnami/python/lib/python3.11/site-packages/surya/ordering.py", line 68, in batch_ordering
    return_dict = model(
                  ^^^^^^
  File "/opt/bitnami/python/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1532, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/bitnami/python/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1541, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/bitnami/python/lib/python3.11/site-packages/surya/model/ordering/encoderdecoder.py", line 38, in forward
    encoder_outputs = self.encoder(
                      ^^^^^^^^^^^^^
  File "/opt/bitnami/python/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1532, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/bitnami/python/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1541, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/bitnami/python/lib/python3.11/site-packages/transformers/models/donut/modeling_donut_swin.py", line 965, in forward
    embedding_output, input_dimensions = self.embeddings(
                                         ^^^^^^^^^^^^^^^^
  File "/opt/bitnami/python/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1532, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/bitnami/python/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1541, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: VariableDonutSwinEmbeddings.forward() got an unexpected keyword argument 'interpolate_pos_encoding'

[blmdxiao]

I’ve encountered the same error on my MacBook too.

[andreasgranhoylieng]

I have the same error on windows

[hexiao0275]

I have the same error on linux

[xujialin03]

一样的错误 windows下

[kiberdruzhinnik]

same on windows

workaround

pip install transformers==4.42.1

edited correct version is 4.41.2

pip install transformers==4.41.2

[jamesmcroft]

same on windows

workaround

pip install transformers==4.42.1

Attempted this workaround but still no luck unfortunately on Linux.

[kiberdruzhinnik]

same on windows workaround pip install transformers==4.42.1

Attempted this workaround but still no luck unfortunately on Linux.

sorry, correct version is 4.41.2

pip install transformers==4.41.2

[jamesmcroft]

same on windows workaround pip install transformers==4.42.1

Attempted this workaround but still no luck unfortunately on Linux.

sorry, correct version is 4.41.2

pip install transformers==4.41.2

As yes, this works perfectly. Thank you for the correction

[mdoughty-tagleaf]

You're a beautiful person, @kiberdruzhinnik 🙏

[kiberdruzhinnik]

actually this is not completed as pip install marker-pdf installs transformers version of >=4.41.0,<5.0.0 which is now incompatible

[VikParuchuri]

I will patch this in all repos so it's compatible with the newer transformers versions

It's due to this transformers commit - https://github.com/huggingface/transformers/commit/481a95781404e48b1c80940be17e8279dec82fe8#diff-00107184180b3f2cd8d4d2ef397c7aa93bf605e1bf7530d19fd9d83ce92eec23

[VikParuchuri]

Fixed by https://github.com/VikParuchuri/marker/pull/213