Multiple network interfaces

[mbentley]

I would like to be able to create a VM with multiple network interfaces. It doesn't seem to be possible to do this with tart today from what I can see and from what I've tried. I tried to specify multiple --net-bridged options and it only created one interface and looking at the config, it doesn't seem like the config.json has any obvious settings to be able to create multiple interfaces.

[fkorotkov]

It seems it's possible to provide several interfaces to VMs. We need to test things and we'll get back to you next week.

[fkorotkov]

@mbentley what is your use case?

I did a quick try and you can check this custom build of Tart. I tried to run it like this after tar -xzvf tart.tar.gz:

./tart.app/Contents/MacOS/tart run --net-bridged=Ethernet --net-bridged="Wi-Fi" ventura-base

Which resulted in the VM to have two en0 and en1 ports reported by networksetup -listallhardwareports.

[mbentley]

Thanks for the build! I will have to test it this evening. I have what is probably a bit of an edge case - I'm looking to be able to create Linux virtual machines (in my case Debian) on my Mac Mini (m2), install Docker, and then I want my containers to use macvlan. The difficulty with macvlan is that In order to access the container from the host running the container, I need to add another network interface and typically what I do is to add a second NIC (reference to the limitation) to either a physical or virtual machine when I need to use macvlan instead of a virtual interface. This allows me to utilize a completely separate physical network link as well, if I chose to do so but in most instances, I would probably use the same mac host interface multiple times, so something like:

tart run --net-bridged=en0 --net-bridged=en0 debtest

Where I could repeat the same --net-bridged parameter, multiple times, even with the same interface multiple times.

Overall, my use case is to move from using lima which I think might be a bit too opinionated in terms of how's used as more of just a simple method for running and controlling VMs from the CLI. I realize that my use case seems to be a bit different than what seems to be the primary focus of the project - running VMs for CI so if that's against the direction you all would want to take the project, I understand.

[mbentley]

At least from my tests I did last night of the provided build, I am not seeing multiple network interfaces, at least not on linux. I did try both my en0 and Wi-Fi networks but I still only get one interface.

Using https://dl-cdn.alpinelinux.org/alpine/v3.18/releases/aarch64/alpine-standard-3.18.2-aarch64.iso:

/Applications/tart.app/Contents/MacOS/tart create --linux alpine
/Applications/tart.app/Contents/MacOS/tart run --net-bridged="en0" --net-bridged="Wi-Fi" --disk ~/Downloads/alpine-standard-3.18.2-aarch64.iso alpine

[fkorotkov]

@mbentley could you please try this build. It contains a change that allow multiple identical bridged interfaces passed via --net-bridged. Before only a unique set of them could be specified.

I've tested with both stardard variant and virt variant of alpine *.iso and can see eth0 and eth1 interfaces now.

As per goals of the Tart Project, it indeed started as tool to run CI tasks in isolated macOS virtual machines but since then we extended the support for Linux and many other things to make Tart a more generic tool for running any virtualized workloads.

[mbentley]

Awesome, that works - I see multiple network interfaces in my linux VM. I do see that they both have the same mac address but there are two interfaces that I can reach from other hosts on my network.

[fkorotkov]

Opened #578. Would love to learn more about your setup when it will work for you. Sounds like a very interesting thing I'd like to learn more about.

[hepivax]

hey @fkorotkov, I've tried this with Tart 2.4.2 on Sonoma image with mapping of two vlan interfaces: tart run sonoma --net-bridged=en0.20 --net-bridged=en0.30 The first interface is working properly on the guest and appear in System Settings as a network service, the second is not, but I do see that en1 is listed. if I swap the interface order it will work as well, meaning there's no problem on the host interfaces

[fkorotkov]

@hepivax what does the following command output for you?

ssh admin@$(tart ip --resolver=arp sonoma) < "networksetup -listallhardwareports"

For me it outputs 3 devices: 1 Tart version one and two network devices.

[hepivax]

@fkorotkov - same as in my machine:

Hardware Port: tart-version-2.4.2
Device: tart-version-2.4.2
Ethernet Address: N/A

Hardware Port: Ethernet
Device: en0
Ethernet Address: e6:fe:17:76:4d:f6

Hardware Port: Ethernet Adapter (en2)
Device: en2
Ethernet Address: de:04:6c:26:13:0b

VLAN Configurations
===================

The OS booted with en0 configured on the Ethernet network service. Thanks to your suggestion I noticed the second interface - en2 which didn't have a network service. I ran:

networksetup -createnetworkservice  Ethernet2 en2
networksetup -setnetworkserviceenabled Ethernet2 on

Unfortunately, it didn't cause the interface to function, it appears in ifconfig with status: inactive and it isn't listed in the output of networksetup -listallnetworkservices

[fkorotkov]

Not sure what's going on there. The initial issue was around Linux VMs. What is your use case for multiple networks for macOS?

[hepivax]

It needs to be connected to two networks, one for control (ssh, appium) and another for Internet access

[MacTroy]

I have the same need as hepivax: in my case, a Mac Studio with three separate NICs, connected to three different network segments, and it would be very useful to have a Sonoma VM that would accept two or three --net-bridged="en0" --net-bridged="en2" arguments that produce usable interfaces within the VM. As of now, it accepts/creates an interface for whichever --net-bridged= argument comes first and ignores any subsequent ones.

[hepivax]

I have the same need as hepivax: in my case, a Mac Studio with three separate NICs, connected to three different network segments, and it would be very useful to have a Sonoma VM that would accept two or three --net-bridged="en0" --net-bridged="en2" arguments that produce usable interfaces within the VM. As of now, it accepts/creates an interface for whichever --net-bridged= argument comes first and ignores any subsequent ones.

@MacTroy this seems to be a limitation of Apple Virtualization Framework. I made an attempt with Parallels and got the same result. I made a workaround using Linux as a router using Iptables with a few NAT rules.

[MacTroy]

@hepivax Interesting. Given that it's happening in Parallels as well, I agree that it's likely Apple's issue, but based on their documentation, it does not seem to be working as designed. Phrasing in multiple places here: https://developer.apple.com/documentation/virtualization/vzbridgednetworkinterface really makes it sound like multiple physical interfaces should be available to a single VM, especially:

the system creates one object for each physical interface of the host computer and stores those objects in the networkInterfaces property. Iterate over the objects in that property to retrieve the network interfaces you need.

@fkorotkov is this worth reporting to Apple as a bug?

Also, if you've got sample configs somewhere for that iptables solution, @hepivax , I'd be interested. Not being able to access at least two NICs in a VM is really hamstringing two of my projects at the moment.