ASMap v2

[willcl-ark]

Another go at using ASMap.

This time do not rely on copying in asmap.dat when we build the container, instead (in Dockerland) mount it as a volume into each container.

Missing:

• K8s equivalent action to mount generated ASMap into pods
• Reloading tank AS from asmap.txt (can be done, not sure if necessary for server to ever know of this after initialisation?)
• Get rid of hardcoded variables in compose_backend.py volume mount
• Check that using AS numbers in range 1 - 64496 is correct
• Check that mapping an ip address /32 to an AS is correct

[vercel[bot]]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
warnet	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Dec 22, 2023 3:19am

[willcl-ark]

Thanks, I'll try address this this evening. Agree it's not ready yet.

It's very annoying (with asmap) the workflow requires us to start the pods, wait for ip addresses, and then pass those ip addresses back to bitcoind before it starts :S

[willcl-ark]

OK I updated those nits, but it's still not going to be any good to merge... Need to brainstorm how it would be possible to build an ASMap on k8s.

The only "sane" way I can think of doing it, which might work on both systems, is to override the default command (bitcoind) with cat /dev/null, wait for the containers to start, fetch the ip addresses, build the asmap, load it in, then use exec to start bitcoind.

This might have it's own problems though, for example if bitcoind crashes the container won't notify as the "main" process is now cat :(

Any other less insane ideas?

[m3dwards]

In k8s there is a concept of init containers which I think might be worth considering for this. Init containers are part of the same pod, they run to completion before the main workload container starts.

How about:

1. Have a single volume for the ASMap files that the RPC pod and all tank pods can mount
2. After creating tank pods, the RPC container generates the required ASMap files and writes them to that volume, named by tank index. It can query the kubernetes service host to get IP addresses for each tank.
3. Tank pod has init container that simply blocks until the required ASMap file is present for it's tank
4. bitcoind container has ASMap file preconfigured in flag, by the time it starts the file will be present on the mount

The reason I think we will need a single volume that all pods mount is that I don't think it's possible to mount the volume after pod creation. As the RPC pod is long running it would have started before any tanks and their respective volumes were created.

[willcl-ark]

Going to close this for now. As we are looking like we are going to fallback to orchestrator-assigned ip addresses, either with @pinheadmz nftables approach, or patched images a-la #261, ASMap will not work easily in either case.

We can revisit if needed.