内存泄漏导致进程被杀死

[Cokejia]

Description of the bug | 错误描述

在处理一个PDF页数为379页的文件时，模型会不断占用内存，达到一定量后被oom杀死（本地机子上也会不断占用内存，最后出现MemoryError）。 magic-pdf --version 0.10.0 运行时内存占用如下： 显存占用如下：

求大佬赐教，非常着急，感谢感谢！！！

How to reproduce the bug | 如何复现

以下是日志文件内容： WARNING: OMP_NUM_THREADS set to 12, not 1. The computation speed will not be optimized if you use data parallel. It will fail if this PaddlePaddle binary is compiled with OpenBlas since OpenBlas does not support multi-threads. PLEASE USE OMP_NUM_THREADS WISELY. import tensorrt_llm failed, if do not use tensorrt, ignore this message import lmdeploy failed, if do not use lmdeploy, ignore this message 2024-11-23 20:56:07.879	INFO	magic_pdf.libs.pdf_check:detect_invalid_chars:57 - cid_count: 0, text_len: 10, cid_chars_radio: 0.0 2024-11-23 20:56:07.880	WARNING	magic_pdf.filter.pdf_classify_by_type:classify:334 - pdf is not classified by area and text_len, by_image_area: False, by_text: False, by_avg_words: False, by_img_num: True, by_text_layout: False, by_img_narrow_strips: True, by_invalid_chars: True 2024-11-23 20:56:07.882	INFO	magic_pdf.model.pdf_extract_kit:init:78 - DocAnalysis init, this may take some times, layout_model: layoutlmv3, apply_formula: True, apply_ocr: True, apply_table: False, table_model: rapid_table, lang: None 2024-11-23 20:56:07.882	INFO	magic_pdf.model.pdf_extract_kit:init:91 - using device: cuda 2024-11-23 20:56:07.882	INFO	magic_pdf.model.pdf_extract_kit:init:95 - using models_dir: /root/.cache/modelscope/hub/opendatalab/PDF-Extract-Kit-1___0/models CustomVisionEncoderDecoderModel init VariableUnimerNetModel init VariableUnimerNetPatchEmbeddings init VariableUnimerNetModel init VariableUnimerNetPatchEmbeddings init CustomMBartForCausalLM init CustomMBartDecoder init [11/23 20:56:18 detectron2]: Rank of current process: 0. World size: 1 [11/23 20:56:18 detectron2]: Environment info:

sys.platform linux Python 3.10.15 (main, Oct 3 2024, 07:27:34) [GCC 11.2.0] numpy 1.26.4 detectron2 0.6 @/root/miniconda3/envs/MinerU/lib/python3.10/site-packages/detectron2 Compiler GCC 11.4 CUDA compiler not available DETECTRON2_ENV_MODULE PyTorch 2.3.1+cu121 @/root/miniconda3/envs/MinerU/lib/python3.10/site-packages/torch PyTorch debug build False torch._C._GLIBCXX_USE_CXX11_ABI False GPU available Yes GPU 0 NVIDIA GeForce RTX 3080 (arch=8.6) Driver version 550.120 CUDA_HOME /usr/local/cuda Pillow 11.0.0 torchvision 0.18.1+cu121 @/root/miniconda3/envs/MinerU/lib/python3.10/site-packages/torchvision torchvision arch flags 5.0, 6.0, 7.0, 7.5, 8.0, 8.6, 9.0 fvcore 0.1.5.post20221221 iopath 0.1.9 cv2 4.6.0

---

PyTorch built with:

• GCC 9.3
• C++ Version: 201703
• Intel(R) oneAPI Math Kernel Library Version 2022.2-Product Build 20220804 for Intel(R) 64 architecture applications
• Intel(R) MKL-DNN v3.3.6 (Git Hash 86e6af5974177e513fd3fee58425e1063e7f1361)
• OpenMP 201511 (a.k.a. OpenMP 4.5)
• LAPACK is enabled (usually provided by MKL)
• NNPACK is enabled
• CPU capability usage: AVX512
• CUDA Runtime 12.1
• NVCC architecture flags: -gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_80,code=sm_80;-gencode;arch=compute_86,code=sm_86;-gencode;arch=compute_90,code=sm_90
• CuDNN 8.7 (built against CUDA 11.8)
  • Built with CuDNN 8.9.2
• Magma 2.6.1
• Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=12.1, CUDNN_VERSION=8.9.2, CXX_COMPILER=/opt/rh/devtoolset-9/root/usr/bin/c++, CXX_FLAGS= -D_GLIBCXX_USE_CXX11_ABI=0 -fabi-version=11 -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -DNDEBUG -DUSE_KINETO -DLIBKINETO_NOROCTRACER -DUSE_FBGEMM -DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -O2 -fPIC -Wall -Wextra -Werror=return-type -Werror=non-virtual-dtor -Werror=bool-operation -Wnarrowing -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-unused-parameter -Wno-unused-function -Wno-unused-result -Wno-strict-overflow -Wno-strict-aliasing -Wno-stringop-overflow -Wsuggest-override -Wno-psabi -Wno-error=pedantic -Wno-error=old-style-cast -Wno-missing-braces -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Wno-stringop-overflow, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_VERSION=2.3.1, USE_CUDA=ON, USE_CUDNN=ON, USE_CUSPARSELT=1, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_GLOO=ON, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=OFF, USE_NCCL=1, USE_NNPACK=ON, USE_OPENMP=ON, USE_ROCM=OFF, USE_ROCM_KERNEL_ASSERT=OFF,

[11/23 20:56:18 detectron2]: Command line arguments: {'config_file': '/root/miniconda3/envs/MinerU/lib/python3.10/site-packages/magic_pdf/resources/model_config/layoutlmv3/layoutlmv3_base_inference.yaml', 'resume': False, 'eval_only': False, 'num_gpus': 1, 'num_machines': 1, 'machine_rank': 0, 'dist_url': 'tcp://127.0.0.1:57823', 'opts': ['MODEL.WEIGHTS', '/root/.cache/modelscope/hub/opendatalab/PDF-Extract-Kit-1___0/models/Layout/LayoutLMv3/model_final.pth']} [11/23 20:56:18 detectron2]: Contents of args.config_file=/root/miniconda3/envs/MinerU/lib/python3.10/site-packages/magic_pdf/resources/model_config/layoutlmv3/layoutlmv3_base_inference.yaml: AUG: DETR: true CACHE_DIR: ~/cache/huggingface CUDNN_BENCHMARK: false DATALOADER: ASPECT_RATIO_GROUPING: true FILTER_EMPTY_ANNOTATIONS: false NUM_WORKERS: 4 REPEAT_THRESHOLD: 0.0 SAMPLER_TRAIN: TrainingSampler DATASETS: PRECOMPUTED_PROPOSAL_TOPK_TEST: 1000 PRECOMPUTED_PROPOSAL_TOPK_TRAIN: 2000 PROPOSAL_FILES_TEST: [] PROPOSAL_FILES_TRAIN: [] TEST:

• scihub_train TRAIN:
• scihub_train GLOBAL: HACK: 1.0 ICDAR_DATA_DIR_TEST: '' ICDAR_DATA_DIR_TRAIN: '' INPUT: CROP: ENABLED: true SIZE:
  • 384
  • 600 TYPE: absolute_range FORMAT: RGB MASK_FORMAT: polygon MAX_SIZE_TEST: 1333 MAX_SIZE_TRAIN: 1333 MIN_SIZE_TEST: 800 MIN_SIZE_TRAIN:
• 480
• 512
• 544
• 576
• 608
• 640
• 672
• 704
• 736
• 768
• 800 MIN_SIZE_TRAIN_SAMPLING: choice RANDOM_FLIP: horizontal MODEL: ANCHOR_GENERATOR: ANGLES:
  • • -90
    • 0
    • 90 ASPECT_RATIOS:
  • • 0.5
    • 1.0
    • 2.0 NAME: DefaultAnchorGenerator OFFSET: 0.0 SIZES:
  • • 32
  • • 64
  • • 128
  • • 256
  • • 512 BACKBONE: FREEZE_AT: 2 NAME: build_vit_fpn_backbone CONFIG_PATH: '' DEVICE: cuda FPN: FUSE_TYPE: sum IN_FEATURES:
  • layer3
  • layer5
  • layer7
  • layer11 NORM: '' OUT_CHANNELS: 256 IMAGE_ONLY: true KEYPOINT_ON: false LOAD_PROPOSALS: false MASK_ON: true META_ARCHITECTURE: VLGeneralizedRCNN PANOPTIC_FPN: COMBINE: ENABLED: true INSTANCES_CONFIDENCE_THRESH: 0.5 OVERLAP_THRESH: 0.5 STUFF_AREA_LIMIT: 4096 INSTANCE_LOSS_WEIGHT: 1.0 PIXEL_MEAN:
• 127.5
• 127.5
• 127.5 PIXEL_STD:
• 127.5
• 127.5
• 127.5 PROPOSAL_GENERATOR: MIN_SIZE: 0 NAME: RPN RESNETS: DEFORM_MODULATED: false DEFORM_NUM_GROUPS: 1 DEFORM_ON_PER_STAGE:
  • false
  • false
  • false
  • false DEPTH: 50 NORM: FrozenBN NUM_GROUPS: 1 OUT_FEATURES:
  • res4 RES2_OUT_CHANNELS: 256 RES5_DILATION: 1 STEM_OUT_CHANNELS: 64 STRIDE_IN_1X1: true WIDTH_PER_GROUP: 64 RETINANET: BBOX_REG_LOSS_TYPE: smooth_l1 BBOX_REG_WEIGHTS:
  • 1.0
  • 1.0
  • 1.0
  • 1.0 FOCAL_LOSS_ALPHA: 0.25 FOCAL_LOSS_GAMMA: 2.0 IN_FEATURES:
  • p3
  • p4
  • p5
  • p6
  • p7 IOU_LABELS:
  • 0
  • -1
  • 1 IOU_THRESHOLDS:
  • 0.4
  • 0.5 NMS_THRESH_TEST: 0.5 NORM: '' NUM_CLASSES: 10 NUM_CONVS: 4 PRIOR_PROB: 0.01 SCORE_THRESH_TEST: 0.05 SMOOTH_L1_LOSS_BETA: 0.1 TOPK_CANDIDATES_TEST: 1000 ROI_BOX_CASCADE_HEAD: BBOX_REG_WEIGHTS:
  • • 10.0
    • 10.0
    • 5.0
    • 5.0
  • • 20.0
    • 20.0
    • 10.0
    • 10.0
  • • 30.0
    • 30.0
    • 15.0
    • 15.0 IOUS:
  • 0.5
  • 0.6
  • 0.7 ROI_BOX_HEAD: BBOX_REG_LOSS_TYPE: smooth_l1 BBOX_REG_LOSS_WEIGHT: 1.0 BBOX_REG_WEIGHTS:
  • 10.0
  • 10.0
  • 5.0
  • 5.0 CLS_AGNOSTIC_BBOX_REG: true CONV_DIM: 256 FC_DIM: 1024 NAME: FastRCNNConvFCHead NORM: '' NUM_CONV: 0 NUM_FC: 2 POOLER_RESOLUTION: 7 POOLER_SAMPLING_RATIO: 0 POOLER_TYPE: ROIAlignV2 SMOOTH_L1_BETA: 0.0 TRAIN_ON_PRED_BOXES: false ROI_HEADS: BATCH_SIZE_PER_IMAGE: 512 IN_FEATURES:
  • p2
  • p3
  • p4
  • p5 IOU_LABELS:
  • 0
  • 1 IOU_THRESHOLDS:
  • 0.5 NAME: CascadeROIHeads NMS_THRESH_TEST: 0.5 NUM_CLASSES: 10 POSITIVE_FRACTION: 0.25 PROPOSAL_APPEND_GT: true SCORE_THRESH_TEST: 0.05 ROI_KEYPOINT_HEAD: CONV_DIMS:
  • 512
  • 512
  • 512
  • 512
  • 512
  • 512
  • 512
  • 512 LOSS_WEIGHT: 1.0 MIN_KEYPOINTS_PER_IMAGE: 1 NAME: KRCNNConvDeconvUpsampleHead NORMALIZE_LOSS_BY_VISIBLE_KEYPOINTS: true NUM_KEYPOINTS: 17 POOLER_RESOLUTION: 14 POOLER_SAMPLING_RATIO: 0 POOLER_TYPE: ROIAlignV2 ROI_MASK_HEAD: CLS_AGNOSTIC_MASK: false CONV_DIM: 256 NAME: MaskRCNNConvUpsampleHead NORM: '' NUM_CONV: 4 POOLER_RESOLUTION: 14 POOLER_SAMPLING_RATIO: 0 POOLER_TYPE: ROIAlignV2 RPN: BATCH_SIZE_PER_IMAGE: 256 BBOX_REG_LOSS_TYPE: smooth_l1 BBOX_REG_LOSS_WEIGHT: 1.0 BBOX_REG_WEIGHTS:
  • 1.0
  • 1.0
  • 1.0
  • 1.0 BOUNDARY_THRESH: -1 CONV_DIMS:
  • -1 HEAD_NAME: StandardRPNHead IN_FEATURES:
  • p2
  • p3
  • p4
  • p5
  • p6 IOU_LABELS:
  • 0
  • -1
  • 1 IOU_THRESHOLDS:
  • 0.3
  • 0.7 LOSS_WEIGHT: 1.0 NMS_THRESH: 0.7 POSITIVE_FRACTION: 0.5 POST_NMS_TOPK_TEST: 1000 POST_NMS_TOPK_TRAIN: 2000 PRE_NMS_TOPK_TEST: 1000 PRE_NMS_TOPK_TRAIN: 2000 SMOOTH_L1_BETA: 0.0 SEM_SEG_HEAD: COMMON_STRIDE: 4 CONVS_DIM: 128 IGNORE_VALUE: 255 IN_FEATURES:
  • p2
  • p3
  • p4
  • p5 LOSS_WEIGHT: 1.0 NAME: SemSegFPNHead NORM: GN NUM_CLASSES: 10 VIT: DROP_PATH: 0.1 IMG_SIZE:
  • 224
  • 224 NAME: layoutlmv3_base OUT_FEATURES:
  • layer3
  • layer5
  • layer7
  • layer11 POS_TYPE: abs WEIGHTS: OUTPUT_DIR: SCIHUB_DATA_DIR_TRAIN: ~/publaynet/layout_scihub/train SEED: 42 SOLVER: AMP: ENABLED: true BACKBONE_MULTIPLIER: 1.0 BASE_LR: 0.0002 BIAS_LR_FACTOR: 1.0 CHECKPOINT_PERIOD: 2000 CLIP_GRADIENTS: CLIP_TYPE: full_model CLIP_VALUE: 1.0 ENABLED: true NORM_TYPE: 2.0 GAMMA: 0.1 GRADIENT_ACCUMULATION_STEPS: 1 IMS_PER_BATCH: 32 LR_SCHEDULER_NAME: WarmupCosineLR MAX_ITER: 20000 MOMENTUM: 0.9 NESTEROV: false OPTIMIZER: ADAMW REFERENCE_WORLD_SIZE: 0 STEPS:
• 10000 WARMUP_FACTOR: 0.01 WARMUP_ITERS: 333 WARMUP_METHOD: linear WEIGHT_DECAY: 0.05 WEIGHT_DECAY_BIAS: null WEIGHT_DECAY_NORM: 0.0 TEST: AUG: ENABLED: false FLIP: true MAX_SIZE: 4000 MIN_SIZES:
  • 400
  • 500
  • 600
  • 700
  • 800
  • 900
  • 1000
  • 1100
  • 1200 DETECTIONS_PER_IMAGE: 100 EVAL_PERIOD: 1000 EXPECTED_RESULTS: [] KEYPOINT_OKS_SIGMAS: [] PRECISE_BN: ENABLED: false NUM_ITER: 200 VERSION: 2 VIS_PERIOD: 0

[11/23 20:56:21 d2.checkpoint.detection_checkpoint]: [DetectionCheckpointer] Loading from /root/.cache/modelscope/hub/opendatalab/PDF-Extract-Kit-1_0/models/Layout/LayoutLMv3/modelfinal.pth ... [11/23 20:56:21 fvcore.common.checkpoint]: [Checkpointer] Loading from /root/.cache/modelscope/hub/opendatalab/PDF-Extract-Kit-10/models/Layout/LayoutLMv3/model_final.pth ... 2024-11-23 20:56:23.160 | INFO | magic_pdf.model.pdf_extract_kit:init:170 - DocAnalysis init done! 2024-11-23 20:56:23.161 | INFO | magic_pdf.model.doc_analyze_by_custom_model:custom_model_init:131 - model init cost: 15.280268430709839 Killed

Operating system | 操作系统

Linux

Python version | Python 版本

3.10

Software version | 软件版本 (magic-pdf --version)

0.9.x

Device mode | 设备模式

cuda

[Cokejia]

内存泄露，占用到一定量后，进程被杀死： 运行上述的服务器配置： 镜像 PyTorch 2.1.2 Python 3.10(ubuntu22.04) Cuda 11.8 GPU RTX 3090(24GB) * 1 CPU 12 vCPU Intel(R) Xeon(R) Platinum 8375C CPU @ 2.90GHz 内存 72GB

[myhloli]

装paddlegpu了吗？

[Cokejia]

装paddlegpu了吗？

装了

[myhloli]

出问题的文件可以上传一份给我们复测吗

[Cokejia]

出问题的文件可以上传一份给我们复测吗

已发送至您的邮箱moe@myhloli.com，请注意查收。 请您在运行过程中，重点关注一下内存的使用量。

[myhloli]

试了下你这个扫描版的pdf分辨率太高了，截图处理的时候把内存爆掉了，后面我们调整下逻辑对分辨率过大的pdf不做缩放处理就可以了

[Cokejia]

试了下你这个扫描版的pdf分辨率太高了，截图处理的时候把内存爆掉了，后面我们调整下逻辑对分辨率过大的pdf不做缩放处理就可以了

好的，大佬。请问这个不做缩放处理具体怎么操作呢？如何在代码里更改呢？

[myhloli]

https://github.com/opendatalab/MinerU/pull/1106/files

[Cokejia]

https://github.com/opendatalab/MinerU/pull/1106/files

好的，谢谢大佬。但是我是直接通过命令行调用的模型，请问源代码存储在本地的哪个文件目录下呢？

[myhloli]

在你的conda安装目录里

[Cokejia]

在你的conda安装目录里

好的，已找到，非常感谢！