search

spacemeshos / SMIPS

Spacemesh Improvement Proposals
https://spacemesh.io
Creative Commons Zero v1.0 Universal
7 stars 1 forks source link

Using the POST go lib in go-spacemesh (WIP) #48

Open avive opened 3 years ago

avive commented 3 years ago

TL;DR - go-spacemesh dev env setup

Follow these steps to properly link go-spacemesh with the gpu-post platform libs:

  1. Copy all artifacts for your dev platform to the go-spacemesh exe path from a compatible gpu-post github release.
  2. Execute go-spacemesh only from its own directory, or set working dir for go-spacemesh exe to be its own directory, or export an VK_ICD_FILENAMES env var with the value set to the go-spacemesh directory.
  3. On windows - use MinGW-W64-builds-4.3.5 or newer version for build toolchain (make, gcc, etc...).
  4. On all platforms: change the permissions on all libs in the artifacts, so your user is allowed to execute them.
  5. On macOS: You MUST right-click-open via Finder on each of the 4 dylibs you downloaded from the internet to enable dynamic linking of apps with these libs because they are not notarized.

Quick test to verify linking works: use the SmesherService::GetProviders() method - it is implemented in the gpu-post c-lib.

macOS gpu-post artifacts

Q: Why are there so many files? A: Apple doesn't include Vulkan drivers in macOS so they must be installed manually. The Vulkan runtime installer is much larger, so we provide these files to get Vulkan working. In Linux and Windows, full Vulkan support is included in the gpu driver if the gpu supports Vulkan.

Linux gpu-post artifacts

Windows gpu-post artifacts

Note that on unix based systems lib is prefixed to the lib name but not in Windows. This is why the Windows libs names are different than Linux and macOS libs.

Tasks

Verify dev env setup on the following:

Overview

go-spacemesh uses several dynamic libraries which are platform-specific. These libs needs to be linked by go-spacemesh in releases, testnets and ci.

Scenarios

  1. Developer - Running a single go-spacemesh node built locally from source code
  2. Developer - Running local-net via terminal
  3. Developer - Running go-spacemesh tests on local machine from cloned repo
  4. Automation - Running go-spacemesh tests in automation via GHA
  5. Automation - Building go-spacemesh releases
  6. Automation - Building Smapp releases
  7. Developer - Running go-spacemesh managed nodes in a testnet.
  8. User: Running go-spacemesh via terminal
  9. User: Running go-spacemesh via Smapp (or any other client which manages a node)

Platforms

  1. Windows 10
  2. macOS
  3. Linux (Ubuntu or other)

Scenario 1: Developer: running a single go-spacemesh built locally from source code

Solution: see working instructions at top of this doc.

Scenario 2: Developer - Running local-net via terminal

Localnet nodes run in a linux docker container without access to the host system gpu drivers apis. The task here is to configure each node to link with the linux .so lib so they can generate post using cpu provider.

Scenario 3: Developer - Running go-spacemesh tests on local machine from cloned repo

Task: we need to find a setup that will enable running gpu-post integration tests in go-spacemesh.

Scenario 4: Developer - Running go-spacemesh tests in automation via GHA

Task: configure go-sm integration tests to link with libgpu-post.so so cpu provider can be used.

Scenario 5: Automation - Building go-spacemesh releases

  1. Modify the build process to add the artifacts from a gpu-post release (configured to be used in the go-spacemesh repo) so each release for one of the 3 platforms is a zip file which contains the go-sm binary for that platform as well the gpu-post artifacts for that platform.
  2. Use the modified make file for go-spacemesh which properly set the rpath to loader path

Scenario 6: 6. Automation - Building Smapp releases

Task: modify smapp release code to add all the gpu-post artifacts for each of the 3 platforms to the go-spacemesh directory on user's machine after install.

Scenario 7: Developer - Running go-spacemesh managed nodes in a testnet

Task: list here required changes to Spacecraft to support libs linking.

Scenario 8. User: Running go-spacemesh via terminal

  1. Download zip file from a go-spacemesh github release for one of the supported platforms and extract all to a directory.
  2. Run go-spacemesh via terminal from its directory (not from another directory).

The directory will have the artifacts for the user's platform so user doesn't need to do anything to get gpu-post to work.

Scenario 9. User: Running go-spacemesh via Smapp (or any other client which manages a node)

  1. Install Smapp as usual - no changes needed.

Smapp changes:

Working Directory

Go-spacemesh build changes

  1. Modify all go-spacemesh build script in MAKE files (in repo and in automation) to add the the executable directory to rpath embedded in the go-spacemesh executable. This enables go-spacemesh to dynamically link with libraries at execution time.

macOS

install_name_tool -add_rpath @loader_path/ go-spacemesh

Linux

Use patchelf

/patchelf --set-rpath . go-spacemesh

Windows

Q: is this needed? what's the cli to do this?

About MoltenVK_icd.json

avive commented 3 years ago

Progress on dev setup using the instructions at top of this smip:

sudachen commented 3 years ago

For me it works on Linux/Nvidia, Linux-Docker/Nvida, Windows/Nvidia/UHD.

avive commented 3 years ago

For me it works on Linux/Nvidia, Linux-Docker/Nvida, Windows/Nvidia/UHD.

if you share your tests flow i can try on all platforms for amd/vulkan - we have the hardware to test.