Add build Action for windows/linux

[syheliel]

github provides github action to verify whether the repo can be built. It helps you to figure out what's the true dependencies you need to run on a new machine.

[alexhiggins732]

I would love to see a github build pipeline, or even azure. Yafu is now in a state where I can no longer build it because it has external dependencies for which the source code is no longer availabe. For example, MSieve's repository is down and the link to the site is a deadlink. Meanwhile Yafu requires it to build.

Ben, could you perhaps refactor your repository to include the external dependencies you use to build? Ideally, git hub action or not, someone should be able to clone your repository and build. Find tuning for things like x32 vs x64 or for an OS, such as linux, or windows, or mac is one thing, but having external dependencies on repositories that are no longer openly available is entirely another topic.

[tdulcet]

Msieve is on SourceForge and the link on the README works fine for me. I believe you are referring to GMP-ECM, which it looks like was moved here: https://gitlab.inria.fr/zimmerma/ecm, although it is also available to install with some package mangers.

I cannot help with building on Windows, but on Linux (and the WSL) with the apt package manager, including on GitHub Actions, this should work to build everything with the maximum resulting performance and minimal build time:

export CC=gcc
# Or: export CC=clang

git clone https://github.com/bbuhrow/ytools.git
cd ytools
# Add below if supported: USE_AVX2=1 USE_BMI2=1
make -j "$(nproc)" CC="$CC" OPT_FLAGS='-O3 -flto -march=native'
cd ..

git clone https://github.com/bbuhrow/ysieve.git
cd ysieve
# Add below if supported: USE_AVX2=1 USE_BMI2=1
make -j "$(nproc)" CC="$CC" OPT_FLAGS='-O3 -flto -march=native'
cd ..

sudo apt install -y subversion libgmp-dev libecm1-dev

svn checkout https://svn.code.sf.net/p/msieve/code/trunk msieve
cd msieve
make all -j "$(nproc)" CC="$CC" NO_ZLIB=1 ECM=1 VBITS=128
cd ..

git clone https://github.com/bbuhrow/yafu.git
cd yafu
# Add below if supported: USE_AVX2=1 USE_BMI2=1
make -j "$(nproc)" CC="$CC" OPT_FLAGS='-O3 -flto -march=native' NFS=1

Of course, one could also build both GMP and GMP-ECM from scratch as well if desired...

[alexhiggins732]

There is a list of repo dependencies, some of which are no longer being maintained so you have to take a gamble on some random repo fork.

Then among those repos are additional dependencies, which require manually modifying the build files for your environment (Cuda Compute, GPU SDK props, Yasm targets/props, cuda targets props, include paths, include header names, etc..), installing their dependencies (Cuda SDK, Cuda Driver, Yasm, Compiler build tools, Zlib, pthreads), changing project include directories/header file names and trail and error to figure out which build configurations work (stage1_core_smXX.ptx, Cuda sm and compute among the various libraries using cuda)

Yes, it can be done. It took about 2 weeks for me to figure out all the build errors for each library, then figure out what Cuda configurations where needed for each that worked with the other libraries, which external binaries where needed where for each libary, and how to restructure the pulled repos so dependencies could be found. But I now have it building for windows and linux on my end, along with the required libraries in their correct location but what a pain.

• Yafu:
  • Ytools
  • Ysieve
  • Msieve
  • PThreads
  • Zlib
  • Gmp/MPIR
  • Gmp-Ecm

[tdulcet]

It likely would be very helpful for setting up GitHub Actions if you could provide the list of commands you used on Windows and Linux to successfully build everything. The GitHub Actions VMs of course do not have a GPU, so one cannot run CUDA code, but one could still build CUDA code if needed...

[bbuhrow]

Yafu is not really set up to take advantage of GPUs so trying to integrate it into the build is creating unnecessary problems for yourself. The exceptions are msieve gpu polyselect or msieve gpu linear algebra but I would argue that if you are running large enough problems to need those things then you probably shouldn't be using yafu.

I don't know anything about github build actions and I apologize that yafu is difficult to build but that's unlikely to change for the forseeable future. I recommend WSL2 for windows. The linux build path is far easier, IMO, and also produces much faster executables.