faasm development environment setting

[lyuxiaosu]

Hi, I want to build faasm and deploy a local cluster with it. If I use the released version to deploy, everything is fine, but if I used the local build version, inv func demo hello doesn't work. Here is the step I did.

1. git submodule update --init --recursive
2. source ./bin/workon.sh
3. faasmctl deploy.compose --mount-source .
4. docker ps to check all containers are running
5. export FAASM_INI_FILE=./faasm.ini
6. faasmctl cli.faasm
7. inv dev.cmake
8. inv dev.tools
9. exit container
10. docker logs to check all containers are running in normal
11. faasmctl cli.cpp
12. inv func demo hello

Then it gets some errors:

(faasm) xiaosuGW@node0:/my_mount/faasm$ faasmctl cli.cpp
[+] Running 1/1
 ✔ Container faasm-191668-cpp-1  Started                                                                                                                                       0.3s
[+] Copying 1/0
 ✔ faasm-191668-cpp-1 copy /my_mount/faasm/faasm.ini to faasm-191668-cpp-1:/tmp/faasm.ini Copied                                                                               0.0s

----------------------------------
CPP CLI
Version: 0.3.1
Project root: /code/cpp/bin/..
Mode: container
----------------------------------

(cpp) root@a2ef3a5ae440:/code/cpp# inv func demo hello
cmake -GNinja -DCMAKE_TOOLCHAIN_FILE=/usr/local/faasm/toolchain/tools/WasiToolchain.cmake -DCMAKE_BUILD_TYPE=Release /code/cpp/func
CMake Error at /usr/share/cmake-3.25/Modules/CMakeDetermineSystem.cmake:130 (message):
  Could not find toolchain file:
  /usr/local/faasm/toolchain/tools/WasiToolchain.cmake
Call Stack (most recent call first):
  CMakeLists.txt:2 (project)

CMake Error: CMake was unable to find a build program corresponding to "Ninja".  CMAKE_MAKE_PROGRAM is not set.  You probably need to select a different build tool.
CMake Error: CMAKE_C_COMPILER not set, after EnableLanguage
CMake Error: CMAKE_CXX_COMPILER not set, after EnableLanguage
-- Configuring incomplete, errors occurred!
Traceback (most recent call last):
  File "/code/cpp/venv/bin/inv", line 8, in <module>
    sys.exit(program.run())
  File "/code/cpp/venv/lib/python3.10/site-packages/invoke/program.py", line 398, in run
    self.execute()
  File "/code/cpp/venv/lib/python3.10/site-packages/invoke/program.py", line 583, in execute
    executor.execute(*self.tasks)
  File "/code/cpp/venv/lib/python3.10/site-packages/invoke/executor.py", line 140, in execute
    result = call.task(*args, **call.kwargs)
  File "/code/cpp/venv/lib/python3.10/site-packages/invoke/tasks.py", line 138, in __call__
    result = self.body(*args, **kwargs)
  File "/code/cpp/tasks/func.py", line 67, in compile
    wasm_cmake(FUNC_DIR, FUNC_BUILD_DIR, func, clean, debug)
  File "/code/cpp/faasmtools/compile_util.py", line 35, in wasm_cmake
    raise RuntimeError("Failed on cmake for {}".format(target))
RuntimeError: Failed on cmake for hello

This can be reproduced. Did I miss something? Thank you for your help.

[csegarragonz]

Hi,

This may happen if the cluster deployment failed at some point in the past, and you are running it again.

In fact, if you were to run the instructions you pasted on a completely fresh directory, the first time you run docker ps (with, btw, you are better off running faasmctl status), you should see that most containers failed to start.

This is because when you deploy a development cluster with --mount-source, faasmctl will mount your local ./dev/faasm/build into /build/faasm and share this directory with all the containers. As a consequence, the first time you run the code (and ./dev does not exist) /build/faasm will be empty in all directories (and in particular the binaries won't be in /build/faasm/bin).

Similarly, when deploying with --mount-source, we will mount ./dev/faasm-local into /usr/local/faasm for all containers in the cluster. /usr/local/faasm is a very important directory for faasm as it stores all the cross-compilation toolchain in /usr/local/faasm/tools and the LLVM sysroot to cross-compile to WASM in /usr/local/faasm/llvm-sysroot.

Your error is complaining that /usr/local/faasm/toolchain is an empty directory. This is because faasmctl deploy.compose did not populate the sysroot mount. This population happens here: https://github.com/faasm/faasmctl/blob/main/faasmctl/util/compose.py#L241-L268. In summary, if ./dev/faasm-local already existed for some reason, the population is skipped. This can happen after a failed deployment.

In short, I suggest doing the following steps:

faasmctl delete
sudo rm -rf ./dev
faasmctl deploy.compose --mount-source .
# Now this should report some failed containers
faasmctl status
faasmctl cli.faasm
inv dev.tools
exit
faasmctl restart -s worker -s upload -s planner
# All containers should be happy now
faasmctl status
faasmctl cli.cpp
inv func demo hello

Note that what I just described is exactly what the e2e tests do: https://github.com/faasm/faasm/blob/main/tests/e2e/run.sh

[lyuxiaosu]

Thanks for the detailed reply @csegarragonz , this solved the problem, now I can build and run happily:) Really appreciate!

[csegarragonz]

Glad I could help!