restrict outgoing traffic

[xmonader]

restricted outgoing network, only traffic to router is allowed (we get router info over DHCP, auto config) default whitelisted outgoing web traffic (can be turned off by farmer), which means VM's can only go to whitelisted services (https) and web domains (can be with filter e.g. *.ubuntu.com), this to make sure people cannot use our VM's for e.g. hacking the whitelists for outgoing traffic come from a github repo which is signed by us, this gets reloaded every hour

check with @delandtj

[ashraffouda]

blocked on specs from @delandtj

[iwanbk]

Thinking about this,

• the restriction doesn't apply on the host node:
  • otherwise zos housekeeping can be broken
• the restriction only applied on the container/VM:
  • the rules can be applied to the main interface(eth0?) if the packet come from the bridge (zos bridge?)

only traffic to router is allowed

• we don'need to do anything for this because it is the default behavior

he whitelists for outgoing traffic come from a github repo which is signed by us,

• not sure about the signing method
• if we have http whitelist, does it means all other outgoing packets getting dropped?:
  • if yes, we still need to allow dns to make http works
    • mycelium is not restricted

[delandtj]

Now a question : What use is a vm if it doesn't have Internet (because a few whitelisted hosts can hardly be called 'Internet')? In essence, we are defining 'Something' but still have no clue of WHAT we want to obtain. Saying that a VM can not be abused for 'hacking' and have a feature request for 'restrict outgoing traffic' doesn't cover the slightest bit of what is really necessary. Most of it is more policy than implementation. Policy :

• a vm is allowed to access the internet : yes/no Point is that restricting outgoing always cuts short legitimate use too (e.g. smtp)
  • Policy is then enforced by us? By Farmer? A combination?
• a vm is accessible from the internet : what are the ways?
  • services can be run only over mycelium IP ?
    • access is proxied ? What is policy here? What is allowed? Static permissions? Farmer-based permissions?
    • Side-cars? Transparent proxies? eBPF maagik?
    • if IPv6: portforwarding allowed? What permissions?
  • services can only be run on vms with a public IP.
    • what is allowed, what/who sets policy (see above)?
• a VM can't use the farmer's local network.
  • Of course not. Definitely not.
  • But ZOS nodes do, even it it's only for mycelium neighbors.

I'm sure I'm already missing a whole slew of cases, but we need to find them all, define policy and then code something, not the other way around.

[Mik-TF]

Good points @delandtj.Also if all users go through KYC maybe this outgoing traffic restriction is not as necessary for the farmers+grid's security.

[iwanbk]

Well, i thought that the brief specs in the issue description was a result of some discussions, but looks like it was not.

I have some questions:

1. why we need to make restrictions?

   • i rented quite a lot VPS in the past, and i don't remember to had usage limitation, except for SMTP to avoid email spam
   • there was indeed rate limiting

2. Also if all users go through KYC maybe this outgoing traffic restriction is not as necessary for the farmers+grid's security.

Yes, it certainly helps

Rather than creating policy about what to restricts, this is my thought:

1. Rate limiting the bandwidth, optional
2. metrics + alerts to detect unusual activity
3. if blocking SMTP port still considered as industry standard, then build feature to do it.
   • also check for other standards
4. If TF grid is special case that we need to have restriction, maybe we could wrap netfilter, and start with important functionalities.

[Mik-TF]

As I understand, we won't go forward with this as KYC fixes lots of this issue.

@sabrinasadik or @xmonader If you could please confirm and close the issue if possible. Thanks.

[xmonader]

status quo: no need to do any sort of allowed / denied lists given that KYC is enforced, the only concern is local lan security

[Mik-TF]

Great. Any link to the local lan security so I can track on gitea? Thanks.

[xmonader]

https://github.com/threefoldtech/zos/issues/2455