Please mention that this only supports x86_64

[qlyoung]

I wanted to use this on my ARM cluster, spent some 3 hours working through various issues only to discover there's only hardcoded support for x86_64 here. I beg of you, please call this out in your docs or readme or something to save others the trouble.

[networkop]

That's true, it was never tested with ARM. I don't think that x86_64 is hard-coded anywhere apart from the Dockerfiles, so the code should work on both platforms. I am happy to accept PRs if you manage to get it working.

[qlyoung]

Yeah, should have opened a PR, was just losing my mind when I opened this ^^.

To provide some color, most of the time was spent attempting to build meshnet-cni, which doesn't work on k8s 1.18 (shall I open a separate issue for that?). Then I tried with networkservicemesh (I want to use k8s-topo which looks fricking awesome) which ostensibly supports ARM if you build it yourself. Indeed, most of it built - but one component depends on vpp-agent, which doesn't. My ill fated journey ended here:

root@clusterpi-69:/home/pi/vpp-agent# make prod-image
# building prod image
IMAGE_TAG= ./docker/prod/build.sh
Current architecture (armv7l) is not supported.

I'd love to use meshnet-cni instead, but I'm on k8s 1.18 which introduced a lot of incompatible API changes, plus the ARM build issues.

[networkop]

yep, a separate issue for 1.18 would be appreciated. as for the NSM, I've never tested it with VPP as the dataplane, only with kernel-forwarder. Try looking around their makefiles, e.g. this one, to find how to build it for arm.

[qlyoung]

I got NSM working, sort of, but ran into issues with the NSC sidecars. Looped back around here and built this project for ARM. To do this I had to:

• build and install kustomize from source and kill the .mk/kustomize lines that download amd64 binaries
• download an ARM binary for protoc instead of amd64 - unfortunately k8s-sigs doesn't publish those, and it takes several hours to build, but I found a random one lying around
• patch the Dockerfile to properly install protoc .so's, by default libprotoc and libprotobuf so's end up in /go/lib where the linker can't find them
• upload my docker images to my personal registry, patch all the k8s resource files to use those
• applied this patch

  
  --- a/manifests/base/meshnet.yml
  +++ b/manifests/base/meshnet.yml
  @@ -121,8 +121,8 @@ items:
       hostPID: true
       hostIPC: true
       serviceAccountName: meshnet

• nodeSelector:
• beta.kubernetes.io/arch: amd64

After that I got this built and installed.

Would you be willing to take a look at potentially using buildx to make multiarch images? This is way easier to set up than NSM

[networkop]

of course, I'm happy to accept any contributions. Feel free to open a PR.

[networkop]

fixed in #25