search

aleph-im / aleph-vm

Aleph.im VM execution engine
MIT License
43 stars 18 forks source link

Problem: IPv6 forwarding on v0.3.0 makes node unresponsive to ping #441

Open odesenfans opened 11 months ago

odesenfans commented 11 months ago

Describe the bug A node operator (TacoMatte) updated their CRN to v0.3.0-rc1 and reported that the version stopped appearing on the account page. Upon investigation, it turns out that their node was not responding to IPv6 ping requests. IPv4 ping requests work fine.

Reverting to 0.2.8 fixes the issue. We tried the following solutions:

The two first attempts did not help, but disabling IPv6 forwarding fixes the issue. This is a good workaround for now but limits the features available on such nodes.

Server configuration

Additional context The node operator had to customize their network configuration to enable IPv6:

msg515813166-26246

I will update this issue with additional details on the cloud provider once I receive them.

odesenfans commented 11 months ago

The issue still occurs on v0.3.0. Another node operator (@aiz) reported the same issue on Scaleway Dedibox servers. It is unclear why IPv6 forwarding breaks IPv6 networking there.

hoh commented 11 months ago

I can reproduce this issue on our VM lab server 2, scaleway Bare Metal server, Debian 12.

odesenfans commented 11 months ago

The right process for node operators for now:

  1. Upgrade normally to 0.3.0
  2. If it does not work, disable IPv6 forwarding by setting ALEPH_VM_IPV6_FORWARDING_ENABLED=False in /etc/aleph-vm/supervisor.env
  3. Optionally, check if forwarding is still enabled (ex: the supervisor was not switched off properly) and disable it manually:
cat /proc/sys/net/ipv6/conf/all/forwarding   # If this returns 1, run the following command
echo 0 > /proc/sys/net/ipv6/conf/all/forwarding
  1. Contact your cloud provider and ask them "I tried to enable IPv6 forwarding on my server. This makes my machine unreachable over IPv6. Why?".