search

HIT-SCIR / pyltp

pyltp: the python extension for LTP
1.53k stars 352 forks source link

请问如何在arm上安装pyltp #218

Closed jxfruit closed 4 years ago

jxfruit commented 4 years ago

在提问之前，请确认以下几点:

问题类型

使用apt-get安装libboost，然后从官网clone源码，./configure，make时出错 from /ddhome/pyltp/ltp/thirdparty/boost/libs/regex/src/cregex.cpp:22: /ddhome/pyltp/ltp/thirdparty/boost/include/boost/regex/v4/instances.hpp: At global scope: /ddhome/pyltp/ltp/thirdparty/boost/include/boost/regex/v4/instances.hpp:194:71: error: template argument 1 is invalid match_results<std::basic_string::const_iterator>::maybe_assign( ^ /ddhome/pyltp/ltp/thirdparty/boost/include/boost/regex/v4/instances.hpp:194:71: error: template argument 2 is invalid /ddhome/pyltp/ltp/thirdparty/boost/include/boost/regex/v4/instances.hpp:195:80: error: template argument 1 is invalid const match_results<std::basic_string::const_iterator>& m); ^ /ddhome/pyltp/ltp/thirdparty/boost/include/boost/regex/v4/instances.hpp:195:80: error: template argument 2 is invalid /ddhome/pyltp/ltp/thirdparty/boost/include/boost/regex/v4/instances.hpp:195:80: error: template argument 1 is invalid /ddhome/pyltp/ltp/thirdparty/boost/include/boost/regex/v4/instances.hpp:195:80: error: template argument 2 is invalid /ddhome/pyltp/ltp/thirdparty/boost/include/boost/regex/v4/instances.hpp:195:80: error: template argument 1 is invalid /ddhome/pyltp/ltp/thirdparty/boost/include/boost/regex/v4/instances.hpp:195:80: error: template argument 2 is invalid /ddhome/pyltp/ltp/thirdparty/boost/include/boost/regex/v4/instances.hpp:195:80: error: template argument 1 is invalid /ddhome/pyltp/ltp/thirdparty/boost/include/boost/regex/v4/instances.hpp:195:80: error: template argument 2 is invalid /ddhome/pyltp/ltp/thirdparty/boost/include/boost/regex/v4/instances.hpp:195:80: error: template argument 1 is invalid /ddhome/pyltp/ltp/thirdparty/boost/include/boost/regex/v4/instances.hpp:195:80: error: template argument 2 is invalid /ddhome/pyltp/ltp/thirdparty/boost/include/boost/regex/v4/instances.hpp:195:13: error: invalid use of template-name 'boost::match_results' without an argument list const match_results<std::basic_string::const_iterator>& m); 。。。。 [ 26%] Built target otpos make[2]: Leaving directory '/ddhome/pyltp/build' Makefile:83: recipe for target 'all' failed make[1]: [all] Error 2 make[1]: Leaving directory '/ddhome/pyltp/build' Makefile:9: recipe for target 'all' failed make: [all] Error 2 我怀疑是boost缺少regex的lib库，然后又下载boost1.5.3源码包编译regex，也成功的生成相关lib包/usr/local/lib/libboost_regex.so.1.53.0，然后再次进入pyltp进行编译，但是仍然出现上述错误，请问应该如何解决，或者官方能不能提供一个支持arm的包，十分感谢

出错场景

代码片段

如何复现这一错误

运行环境

arm，ubuntu，python3.7.6

期望结果

其他

Please ensure your issue adheres to the following guidelines:

What is affected by this bug?

When does this occur?

Where on the code does it happen?

How do we replicate the issue?

Your environment information

Expected behavior (i.e. solution)

Other Comments

jxfruit commented 4 years ago

使用python setup.py install安装和进入ltp目录单独编译ltp时出现都出现/ddhome/pyltp-setup/ltp/thirdparty/dynet/dynet/mem.cc:13:10: fatal error: mm_malloc.h: No such file or directory

include 

      ^~~~~~~~~~~~~

compilation terminated. thirdparty/dynet/dynet/CMakeFiles/dynet.dir/build.make:283: recipe for target 'thirdparty/dynet/dynet/CMakeFiles/dynet.dir/mem.cc.o' failed make[2]: [thirdparty/dynet/dynet/CMakeFiles/dynet.dir/mem.cc.o] Error 1 make[2]: Waiting for unfinished jobs.... [ 87%] Linking CXX executable ../../../tools/train/otcws [ 87%] Built target otcws CMakeFiles/Makefile2:825: recipe for target 'thirdparty/dynet/dynet/CMakeFiles/dynet.dir/all' failed make[1]: [thirdparty/dynet/dynet/CMakeFiles/dynet.dir/all] Error 2 Makefile:129: recipe for target 'all' failed make: [all] Error 2 然后我查看路径/ddhome/pyltp-setup/ltp/thirdparty/dynet/dynet/mem.cc下甚至整个工程都找了，确实没有找到这个叫mm_malloc.h的头文件，请问大佬该如何是好啊

jxfruit commented 4 years ago

参考https://github.com/clab/dynet/issues/1042解决

excited-tiger commented 4 years ago

参考https://github.com/clab/dynet/issues/1042解决

方便详细讲一下吗？

excited-tiger commented 4 years ago

参考https://github.com/clab/dynet/issues/1042解决

已解决，测试可用，多谢群主， 解决办法：修改 ltp/thirdparty/dynet/dynet/mem.cc文件

直接替换为：


#include "dynet/mem.h"

#include <cstdlib>
#include <cstring>
#include <iostream>
#if !_WINDOWS
#include <sys/shm.h>
#include <sys/mman.h>
#endif

#include <fcntl.h>
#if !_WINDOWS
//#include <mm_malloc.h>
#endif
#include "dynet/except.h"
#if HAVE_CUDA
#include "dynet/cuda.h"
#include <cuda.h>
#include <cuda_runtime.h>
#endif

using namespace std;

namespace dynet {

MemAllocator::~MemAllocator() {}

void* CPUAllocator::malloc(size_t n) {
  void* ptr = aligned_alloc(n, align);
  if (!ptr) {
    cerr << "CPU memory allocation failed n=" << n << " align=" << align << endl;
    throw dynet::out_of_memory("CPU memory allocation failed");
  }
  return ptr;
}

void CPUAllocator::free(void* mem) {
//  _mm_free(mem);
}

void CPUAllocator::zero(void* p, size_t n) {
  memset(p, 0, n);
}

void* SharedAllocator::malloc(size_t n) {
#if _WINDOWS
  cerr << "Shared memory not supported in Windows" << endl;
  throw dynet::out_of_memory("Shared memory allocation failed");
#else
  void* ptr = mmap(NULL, n, PROT_READ|PROT_WRITE, MAP_ANON|MAP_SHARED, -1, 0);
  if (!ptr) {
    cerr << "Shared memory allocation failed n=" << n << endl;
    throw dynet::out_of_memory("Shared memory allocation failed");
  }
  return ptr;
#endif
}

void SharedAllocator::free(void* mem) {
//  munmap(mem, n);
}

void SharedAllocator::zero(void* p, size_t n) {
  memset(p, 0, n);
}

#if HAVE_CUDA
void* GPUAllocator::malloc(size_t n) {
  void* ptr = nullptr;
  CUDA_CHECK(cudaSetDevice(devid));
  CUDA_CHECK(cudaMalloc(&ptr, n));
  if (!ptr) {
    cerr << "GPU memory allocation failed n=" << n << endl;
    throw dynet::out_of_memory("GPU memory allocation failed");
  }
  return ptr;
}

void GPUAllocator::free(void* mem) {
  CUDA_CHECK(cudaFree(mem));
}

void GPUAllocator::zero(void* p, size_t n) {
  CUDA_CHECK(cudaSetDevice(devid));
  CUDA_CHECK(cudaMemsetAsync(p, 0, n));
}

#endif

} // namespace dynet```
wuzhongya1 commented 2 years ago

void CPUAllocator::free(void* mem) { // _mm_free(mem); }

请问这里是否会引起内存泄漏。aligned_alloc()对应的是free(), 是否应该改成free(mem);