search

KatharaFramework / Kathara

A lightweight container-based network emulation system.
https://www.kathara.org/
GNU General Public License v3.0
452 stars 63 forks source link

Apple M1 port tracking issue #131

Closed nopid closed 2 years ago

nopid commented 3 years ago

Porting Kathará to ARM64 architecture to support it both under linux/arm64 and macOS requires several simple steps. A proof of concept universal kathara binary is currently running on my system (see screenshot below).

This issue aims to act as a tracker to the steps to be fulfilled:

katharaM1
Skazza94 commented 3 years ago

Hi @nopid, first of all thank you for the amazing contribution to Kathará!

We were also working on ARM64 and M1 support lately. We really appreciate your PRs and right now we're working to obtain an Apple M1 Mac for testing and releasing Kathará. You PRs seems good, we'll merge them in the next release as soon as we have the hardware to test it.

Thanks and sorry for the delay, Mariano.

nopid commented 3 years ago

Hi @Skazza94,

Thanks for your feedback. We are migrating our networking teaching labs in Orléans from Netkit to Kathará. During the process, I will try to propose contributions that we find valuable in our teaching experience.

When you have a M1, here is what I do to compile a universal (multi-arch) kathara binary. It requires a universal python3 interpreter (I am using a python39 +universal port from Macports):

cd src
sed 's/upx=True,/upx=True,\n          target_arch="universal2",/' < ../scripts/OSX/kathara.spec > kathara.spec
python3.9 -m venv /tmp/foobar
source /tmp/foobar/bin/activate
export ARCHFLAGS='-arch arm64 -arch x86_64'
pip install https://github.com/pyinstaller/pyinstaller/archive/refs/heads/develop.zip
pip install --force-reinstall --no-binary :all: appscript
pip install -r requirements.txt 
pyinstaller --distpath=./kathara.dist --workpath=./kathara.build kathara.spec

Thanks to the whole team making Kathará, Nicolas

Skazza94 commented 3 years ago

Hi @nopid, as a little update, we'll probably have an M1 machine in September.

I'll keep you updated!

Mariano.

Skazza94 commented 3 years ago

Hi @nopid, about the last task (Modify the macOS release script to build a universal2 binary...) we installed on an M1 Mac the same Kathará 3.2.0 package used on x86 and it works (manually patching the NetworkPlugin tag to latest). Package for v3.2.0 was built using pyinstaller on develop branch.

We also checked that it was not run by Rosetta and according to the file command it is built for x86_64.

Any idea why it works? Do you think this task is still needed?

nopid commented 3 years ago

Hi @Skazza94,

Running the x86_64 version on M1 might work but it should add a layer of translation. It is strange that "it was not run by Rosetta". I would expect at least part of it to be (for instance library dependencies).

According to the documentation pyinstaller builds single-arch for the current machine default arch. Notice that in order to produce a multi-arch binary, it requires not only a multi-arch driver (already provided by pyinstaller) but also multi-arch versions of every dependency.

A multi-arch binary is certainly better here. What my script does is simply:

At the end of the process, the kathara binary is not that much bigger a file. The only downside is that it requires macOS 11.

PS. Also the architecture=any Debian version was running without flaw under Ubuntu (using https://github.com/utmapp/UTM/ to virtualize Ubuntu).

Skazza94 commented 3 years ago

Hi @nopid, your new script to build an universal2 binary seems good. However, we still need to keep compatibility with macOS 10 in some way. So, the OSX Makefile should build both binaries. If you already have a modified Makefile, please add it into the PR #139. Otherwise we'll write it ourselves.

About Debian, we can't change the architecture flag since we also use the same build files to deploy the binary on Launchpad. Since Launchpad machines are not connected to the Internet, we download all the Python dependencies before pushing the source package to Launchpad (here). We need to find a way to download the arm64 dependencies even inside an x86 container and put them into the source package. Maybe the ARCHFLAGS you suggested works also for this purpose? Additionally, we also need to build the arm64 binary on Arch and RHEL.

In the meanwhile, building a local unsigned .deb surely works!

Thanks for your contribution and support. Mariano.

Skazza94 commented 2 years ago

Hi @nopid, finally we are working on the M1 release. You can check the status in the develop branch. I decided to compile two separate binaries instead of the universal2 in order to avoid compatibility problems with OS 10.X.

A binary package for Apple M1 will be available soon!

About Debian, we'll also work on supporting ARM64, including Launchpad.

nopid commented 2 years ago

Hi @Skazza94,

That is good news for all the arm64 users 👍.

Simple to use binaries are perfect for most users. A question though for people using the python API: running Kathará commands through the binary wrappers induces a noticeable lag contrary to a pure Python direct usage. Have you considered having the command-line part of the code inside the python package so that: (1) pip would install the kathara command when installing the library ; (2) or that python -m kathara would behave like the kathara command.

Skazza94 commented 2 years ago

Hi @nopid, we officially released the ARM64 version of Kathara (v3.3.0) on both macOS and Launchpad. I don't fully undestand your problem with the Python API. It is better if you open another issue with some code examples or screenshots, so we can discuss better and be on topic. I'll close this issue (related to ARM64 😄)

nopid commented 2 years ago

Hi @Skazza94, Will do!