search

ugermann / ssplit-cpp

Approximate reimplementation of the sentence splitter from the Moses toolkit.
Other
4 stars 9 forks source link

Buildability on CentOS-7 and Windows using PCRE2 sources #30

Open jerinphilip opened 2 years ago

jerinphilip commented 2 years ago

Experimentation in several repositories lead me to believe we have to take a look at the ExternalProject build of PCRE2 (from sources).

I'm trying to tinker with a GitHub action that allows me to release py3.6, py3.7 manylinux wheels eventually to get batteries included bergamot package into pypa. Here's a teaser.

However, the CentOS-7 based image which generates manylinux wheels seems to be not working well with ssplit-cpp. Please find the build full log at:

https://github.com/jerinphilip/lemonade/runs/4912697746?check_suite_focus=true

PCRE2 from source build log: relevant bits (click to expand) ``` [ 75%] Building C object CMakeFiles/pcre2-8-static.dir/src/pcre2_xclass.c.o [ 77%] Linking C static library libpcre2-8.a [ 77%] Built target pcre2-8-static [ 80%] Building C object CMakeFiles/pcre2-posix-static.dir/src/pcre2posix.c.o [ 83%] Linking C static library libpcre2-posix.a [ 83%] Built target pcre2-posix-static [ 86%] Building C object CMakeFiles/pcre2grep.dir/src/pcre2grep.c.o [ 88%] Linking C executable pcre2grep [ 88%] Built target pcre2grep [ 91%] Building C object CMakeFiles/pcre2test.dir/src/pcre2test.c.o [ 94%] Linking C executable pcre2test [ 94%] Built target pcre2test [ 97%] Building C object CMakeFiles/pcre2_jit_test.dir/src/pcre2_jit_test.c.o [100%] Linking C executable pcre2_jit_test [100%] Built target pcre2_jit_test [ 2%] Performing install step for 'pcre2' Consolidate compiler generated dependencies of target pcre2-8-static [ 77%] Built target pcre2-8-static Consolidate compiler generated dependencies of target pcre2-posix-static [ 83%] Built target pcre2-posix-static Consolidate compiler generated dependencies of target pcre2grep [ 88%] Built target pcre2grep Consolidate compiler generated dependencies of target pcre2test [ 94%] Built target pcre2test Consolidate compiler generated dependencies of target pcre2_jit_test [100%] Built target pcre2_jit_test ```

Weirdly in the last the build is failing with the following error message.

  [ 98%] Linking CXX static library libbergamot-translator.a
  [ 98%] Built target bergamot-translator
  [100%] Building CXX object bindings/python/CMakeFiles/_bergamot.dir/bergamot.cpp.o
  gmake[3]: *** No rule to make target `lib/libpcre2-8.a', needed by `../lib.linux-x86_64-3.6/bergamot/_bergamot.cpython-36m-x86_64-linux-gnu.so'.  Stop.
  gmake[2]: *** [bindings/python/CMakeFiles/_bergamot.dir/all] Error 2
  gmake[1]: *** [bindings/python/CMakeFiles/_bergamot.dir/rule] Error 2
  gmake: *** [_bergamot] Error 2

Note that the existing build using ExternalProject compiles with Ubuntu and MacOS and fails with CentOS. We are also aware of issues with Windows when attempting to switch to compile PCRE2 from sources (we're working around this by using a vcpkg provision of pcre2 and using the not from source ExternalProject right now).

Since there are other issues affecting Windows compilation as well, I was wondering if it's worthwhile to pool the efforts and fix the problem here so as to provide a neat means to get the targets (when loaded from ExternalProject built from source) all the way up via CMake.

Thanks in advance,

ugermann commented 2 years ago

Hmm... https://pythonspeed.com/articles/centos-8-is-dead/ Let me dig around for a Docker image to test compilation that I don't have to perform CPR on to get it working.

ugermann commented 2 years ago

I think I fixed the issue. @jerinphilip Can you try if it works now?

ugermann commented 2 years ago

Hold that: regression test failed, but I think I'm onto the right thing ...

jerinphilip commented 2 years ago

@ugermann I have https://github.com/jerinphilip/bergamot-translator/pull/60 ready to provide feedback.

ugermann commented 2 years ago

It's a mess https://gitlab.kitware.com/cmake/cmake/-/issues/23365

ugermann commented 2 years ago

@jerinphilip I'm not able to run the workflows in https://github.com/jerinphilip/bergamot-translator/pull/60.

[Build/Ubuntu 20.04 / py3.8-3   ] ⭐  Run Set up Python
[Build/Ubuntu 20.04 / py3.8-3   ]   ☁  git clone 'https://github.com/actions/setup-python' # ref=v2
[Build/Ubuntu 20.04 / py3.8-3   ]   cloning https://github.com/actions/setup-python to /home/germann/.cache/act/actions-setup-python@v2
[Build/Ubuntu 20.04 / py3.8-3   ] Unable to resolve v2: reference not found
DEBU[0010] reference not found                          
DEBU[0010] reference not found                          
[Build/Ubuntu 20.04 / py3.8-3   ]   ❌  Failure - Set up Python
jerinphilip commented 2 years ago

I'm not sure where the error logs are coming from in your message, but looks like manylinux build passed after I point to current main: https://github.com/jerinphilip/bergamot-translator/runs/5991984231?check_suite_focus=true

ugermann commented 2 years ago

The error log comes from running

act -v -W .github/workflows/build.yml

locally. That's nectos/act.