opendatalab / MinerU

A one-stop, open-source, high-quality data extraction tool, supports PDF/webpage/e-book extraction.一站式开源高质量数据提取工具,支持PDF/网页/多格式电子书提取。
https://opendatalab.com/OpenSourceTools
GNU Affero General Public License v3.0
13.04k stars 972 forks source link

使用mps后无法运行 #194

Closed Raclez closed 2 months ago

Raclez commented 2 months ago

Description of the bug | 错误描述

NotImplementedError: The operator 'aten::upsample_bicubic2d.out' is not currently implemented for the MPS device. If you want this op to be added in priority during the prototype phase of this feature, please comment on https://github.com/pytorch/pytorch/issues/77764. As a temporary fix, you can set the environment variable PYTORCH_ENABLE_MPS_FALLBACK=1 to use the CPU as a fallback for this op. WARNING: this will be slower than running natively on MPS.

How to reproduce the bug | 如何复现

怎么解决

Operating system | 操作系统

MacOS

Python version | Python 版本

3.10

Software version | 软件版本 (magic-pdf --version)

0.6.x

Device mode | 设备模式

mps

myhloli commented 2 months ago

mps的支持依赖pytorch官方,遇到这个报错请参考官方修复方案

As a temporary fix, you can set the environment variable PYTORCH_ENABLE_MPS_FALLBACK=1 to use the CPU as a fallback for this op.

Raclez commented 2 months ago

export PYTORCH_ENABLE_MPS_FALLBACK=1后就报错这个所以我才问 @Error: command buffer exited with error status. The Metal Performance Shaders operations encoded on it may not have completed. Error: (null) Internal Error (0000000e:Internal Error) <AGXG13GFamilyCommandBuffer: 0x10952cd10> label = device = <AGXG13GDevice: 0x31af29600> name = Apple M1 commandQueue = <AGXG13GFamilyCommandQueue: 0x31af73a00> label = device = <AGXG13GDevice: 0x31af29600> name = Apple M1 retainedReferences = 1

myhloli commented 2 months ago

那只能全局退回cpu了 修改下magic-pdf.json中

{
"device-mode":"cpu"
}
myhloli commented 2 months ago

MPS is not stable,we will remove mps support from main-line.maybe cuda is a nice way to use this project.

868618 commented 2 months ago

MPS is not stable,we will remove mps support from main-line.maybe cuda is a nice way to use this project.

大哥,要是mps not stable,我们这些个Mac用户咋办啊,用CPU跑很慢的,我这边有几万个PDF队列,请官方予以考虑

myhloli commented 2 months ago

MPS is not stable,we will remove mps support from main-line.maybe cuda is a nice way to use this project.

大哥,要是mps not stable,我们这些个Mac用户咋办啊,用CPU跑很慢的,我这边有几万个PDF队列,请官方予以考虑

目前确实是mps兼容性问题比较多,受限于开发人员精力有限,后续将不再支持mps。如果只有一个批次的几万本pdf,可以考虑租用支持cuda的云服务,单卡4090大概一天可以处理10w页左右,一周时间差不多可以处理完几万本。