[Question]: 集成测试代码出现ModuleNotFoundError. 安装失败，如何解决？

[WhuanY]

请提出你的问题

• 背景： 目前正在开发LoKrModel的中后期阶段。 当前，在进行测试时出现了如下问题：

  tests/llm/test_lokr.py:80: 
  _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
  tests/llm/testing_utils.py:73: in run_predictor
  predict()
  llm/predict/predictor.py:1625: in predict
  predictor = create_predictor(predictor_args, model_args)
  llm/predict/predictor.py:1354: in create_predictor
  from paddlenlp.experimental.transformers import (
  paddlenlp/experimental/transformers/__init__.py:15: in <module>
  from .bloom import *
  paddlenlp/experimental/transformers/bloom/__init__.py:15: in <module>
  from .modeling import *
  paddlenlp/experimental/transformers/bloom/modeling.py:23: in <module>
  from paddlenlp.experimental.transformers.fused_transformer_layers import (
  _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
  
  from __future__ import annotations
  
  import os
  from dataclasses import dataclass
  from typing import List, Optional
  
  import numpy as np
  import paddle
  import paddle.distributed as dist
  from paddle.framework import LayerHelper, core, in_dynamic_mode, in_dynamic_or_pir_mode
  from paddle.incubate.nn.functional import (
      fused_layer_norm,
      fused_moe,
      fused_rms_norm,
      masked_multihead_attention,
      variable_length_memory_efficient_attention,
  )
  from paddle.nn import Layer
  from paddle.nn.initializer import Constant
  from paddle.nn.quant import weight_only_linear
  
  from paddlenlp.utils.import_utils import is_paddlenlp_ops_available
  from paddlenlp.utils.log import logger
  
  if not is_paddlenlp_ops_available():
      logger.warning(
          "The paddlenlp_ops package is not installed. you can read the docs and install it by hand, "
          "you can refer to: https://github.com/PaddlePaddle/PaddleNLP/blob/develop/csrc/README.md"
      )
  >   from paddlenlp_ops import rebuild_padding_v2
  E   ModuleNotFoundError: No module named 'paddlenlp_ops'

  我按照错误日志的要求，手动安装paddlenlp_ops包裹。 但是出现如下问题： 执行到 python setup_cuda.py install一行时，出现这个包裹没有找到的问题：

  
  (PaddleLoKr) (base) wuyuhuan@zuchongzhi:~/PaddleNLP/csrc$ python setup_cuda.py install
  WARNING: OMP_NUM_THREADS set to 20, not 1. The computation speed will not be optimized if you use data parallel. It will fail if this PaddlePaddle binary is compiled with OpenBlas since OpenBlas does not support multi-threads.
  PLEASE USE OMP_NUM_THREADS WISELY.
  Cloning into 'third_party/nlohmann_json'...
  remote: Enumerating objects: 35301, done.
  remote: Counting objects: 100% (6620/6620), done.
  remote: Compressing objects: 100% (778/778), done.
  error: RPC failed; curl 56 Recv failure: Connection reset by peer
  error: 665 bytes of body are still expected
  fetch-pack: unexpected disconnect while reading sideband packet
  fatal: early EOF
  fatal: fetch-pack: invalid index-pack output
  Git clone https://github.com/nlohmann/json.git operation failed with the following error: Command '['git', 'clone', '-b', 'v3.11.3', '--single-branch', 'https://github.com/nlohmann/json.git', 'third_party/nlohmann_json']' returned non-zero exit status 128.
  Please check your network connection or access rights to the repository.
  If the problem persists, please refer to the README file for instructions on how to manually download and install the necessary components.
  running install
  /home/wuyuhuan/anaconda3/envs/PaddleLoKr/lib/python3.8/site-packages/setuptools/_distutils/cmd.py:66: SetuptoolsDeprecationWarning: setup.py install is deprecated.
  !!
  
      ********************************************************************************
      Please avoid running ``setup.py`` directly.
      Instead, use pypa/build, pypa/installer or other
      standards-based tools.
  
      See https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html for details.
      ********************************************************************************

!! self.initialize_options() /home/wuyuhuan/anaconda3/envs/PaddleLoKr/lib/python3.8/site-packages/setuptools/_distutils/cmd.py:66: EasyInstallDeprecationWarning: easy_install command is deprecated. !!

    ********************************************************************************
    Please avoid running ``setup.py`` and ``easy_install``.
    Instead, use pypa/build, pypa/installer or other
    standards-based tools.

    See https://github.com/pypa/setuptools/issues/917 for details.
    ********************************************************************************

!! self.initialize_options() running bdist_egg running egg_info writing paddlenlp_ops.egg-info/PKG-INFO writing dependency_links to paddlenlp_ops.egg-info/dependency_links.txt writing top-level names to paddlenlp_ops.egg-info/top_level.txt reading manifest file 'paddlenlp_ops.egg-info/SOURCES.txt' writing manifest file 'paddlenlp_ops.egg-info/SOURCES.txt' installing library code to build/paddlenlp_ops/bdist.linux-x86_64/egg running install_lib running build_ext Compiling user custom op, it will cost a few seconds..... building 'paddlenlp_ops' extension /usr/local/cuda/bin/nvcc -I/home/wuyuhuan/anaconda3/envs/PaddleLoKr/lib/python3.8/site-packages/paddle/include -I/home/wuyuhuan/anaconda3/envs/PaddleLoKr/lib/python3.8/site-packages/paddle/include/third_party -I/usr/local/cuda/include -I/home/wuyuhuan/anaconda3/envs/PaddleLoKr/include/python3.8 -I/home/wuyuhuan/anaconda3/envs/PaddleLoKr/include/python3.8 -c /home/wuyuhuan/PaddleNLP/csrc/gpu/dequant_int8.cu -o /home/wuyuhuan/PaddleNLP/csrc/build/paddlenlp_ops/lib.linux-x86_64-cpython-38/dequant_int8.cu.o -DPADDLE_WITH_CUDA -DEIGEN_USE_GPU -ccbin cc -Xcompiler -fPIC --expt-relaxed-constexpr -DNVCC -gencode arch=compute_80,code=sm_80 -O3 -UCUDA_NO_HALF_OPERATORS -UCUDA_NO_HALF_CONVERSIONS -UCUDA_NO_BFLOAT16_OPERATORS -UCUDA_NO_BFLOAT16_CONVERSIONS -UCUDA_NO_BFLOAT162_OPERATORS -UCUDA_NO_BFLOAT162_CONVERSIONS -Igpu -Igpu/cutlass_kernels -Igpu/fp8_gemm_with_cutlass -Igpu/cutlass_kernels/fp8_gemm_fused/autogen -Ithird_party/cutlass/include -Ithird_party/nlohmann_json/single_include -Igpu/sample_kernels -w -DPADDLE_WITH_CUSTOM_KERNEL -D_GLIBCXX_USE_CXX11_ABI=1 -std=c++17 In file included from /home/wuyuhuan/PaddleNLP/csrc/gpu/dequant_int8.cu:15: /home/wuyuhuan/PaddleNLP/csrc/gpu/helper.h:40:10: fatal error: nlohmann/json.hpp: No such file or directory 40 | #include "nlohmann/json.hpp"

的问题。
既然没有fetch成功，我猜想可能是版本问题，于是自己到对应的nlohmann仓库去fetch，放到指定工作目录下:
![image](https://github.com/user-attachments/assets/ad0cccbe-3ff4-4008-9547-61f4442134da)
再次运行：

python setup_cuda.py install

出现了我不知道如何解决的问题：

/usr/lib/gcc/x86_64-linux-gnu/13/include/amxtileintrin.h(42): error: identifier "builtin_ia32_ldtilecfg" is undefined builtin_ia32_ldtilecfg (__config); ^

/usr/lib/gcc/x86_64-linux-gnu/13/include/amxtileintrin.h(49): error: identifier "builtin_ia32_sttilecfg" is undefined builtin_ia32_sttilecfg (__config); ^

2 errors detected in the compilation of "/home/wuyuhuan/PaddleNLP/csrc/gpu/dequant_int8.cu". error: command '/usr/local/cuda/bin/nvcc' failed with exit code 2

有没有其他办法解决该算子问题？或者说原本的.md文档需要更新？多谢😊

[ZHUI]

你好

1. 如果你的测试，不需要跑预测。可以直接注释掉相关推理测试的运行代码
2. 编译问题可能是版本问题，你本地的cuda 和 gcc 是什么版本？

[WhuanY]

不需要跑预测