Mikrotik / RouterOS support impossible?

[robdejonge]

Given the fact that Mikrotik users would typically probably be interested in a service like Control D, the fact that more recent versions of RouterOS support running ARM-based Docker containers, and the built-in resolver of RouterOS handles DoH quite poorly, it would seem to me this is a prime candidate to have support running ctrld.

Q1: This is why I am wondering if there are any reasons it is not yet listed here. Are there reasons why it would not work? Or is this more a matter of priorities?

Q2: If the latter, I might have a go at it myself to see if I can make it work. But before I do, I need to make sure the Docker container can be configured to never touch the filesystem. My particular router (RB4011) does have some storage, but it really is not ideally suited for frequent reads & writes. Obviously I could configure a RAM disk, remove logging to a file from my configuration file, etc. But even then, some containers are just not suited for this type of operation.

[yegors]

Hi,

We have an official docker container, you can find it here: https://hub.docker.com/r/controldns/ctrld

This was not tested on Mikrotik, so feel free to give it a shot.

[robdejonge]

Thanks, @yegors. I'm getting familiar with the product at the moment and have it running on a Linux container (LXC) now. First impression is excellent. Next is to use the Docker container, but when I tried that I got endless streams of errors. I'll try that on a clean host next to see if I can get it to work there. Once that is set up properly, I'll see if I can get it to work on my router.

What can you tell me about the second question I asked? My router only has flash storage that is not user-serviceable, so if that dies my router dies. Needless to say, I don't want that to happen.

So, what can you tell me about disk access by ctrld? Obviously I should not configure logging. What other functions will trigger frequent disk access (read or write) and is there anything internal to the program that will cause frequent disk access that I can't disable?

[yegors]

Can you clarify what the "endless stream of errors" is? You should be able to see it in the Docker console.

The ctrld binary will write a config file to "disk" when running in standard mode. If you cannot write ANYTHING to disk, you should use the ephemeral mode: https://github.com/Control-D-Inc/ctrld/blob/main/docs/ephemeral_mode.md

However this was not tested inside a container. It's better to give it a disk, even if its a RAM disk.

[robdejonge]

I've been able to get it running in Docker. Turns out the cause of those errors was an incorrect [listener] definition. Once I realized ip was not a required setting, it all worked just fine with an assumed '0.0.0.0' value. Using an external configuration file, I used :

docker run -d --name=ctrld -p 127.0.0.1:53:53/tcp -p 127.0.0.1:53:53/udp \
-v /opt/controld/ctrld.toml:/etc/controld/ctrld.toml --network="host" controldns/ctrld:latest -vv 

No need for ephemeral mode in my case. My Mikrotik router does allow disk writes. My question was mainly if things like the cache or other normal functions of ctrld would pound the disk a lot and kill my flash storage or if it's all kept in RAM. I'd appreciate a clarification of that.

Then, to get it working on Mikrotik (and once this is working, I'll share the config so you can add it to your docs) I've so far not been able to. I can get the container up and running, but due to limitations of the RouterOS implementation of Docker, I have access to the Docker console. The fact that I want to continue using the resolver built into RouterOS, of course does not help either. But I'll see if I can make any further progress.

[yegors]

At runtime, there is nothing writing to disk unless you have logging enabled to write to an external file. On some platforms -vv output would be captured by journalctl / logread and written to disk, but I don't think that's a factor for Docker. That being said, you may want to get rid of the -vv flag to prevent verbose logging into stdout.

Lastly, and at a higher level: You can certainly run a container for DNS, but in all honestly it sounds like a huge overkill just to send DNS queries to an upstream. Something like Rpi can be better suited for the task, without any containers. You also probably lose some of the capabilities of ctrld like LAN awareness / client discovery while running in a container.

[robdejonge]

The Docker implementation on my Mikrotik router is very challenging to use. I can use your Docker Hub image, mount a configuration file into it and it gets reported as running. But due to constraints in RouterOS there is no way for me to see container console output, due to the image there is no shell into the container possible, and for whatever reason I see nothing appear in a logfile even if configured to debug level. I'd really like to get this working and publish steps, as I think there is a significant overlap between Mikrotik and Control D user profiles. But I'm basically groping around in the dark. And not in a fun way!

Regarding the decision to run it inside Docker: well, this is the only way to get external software running on Mikrotik / RouterOS. While it is based on Linux, there is no traditional shell access to the device. So why not run it on a separate device? It's running on a Proxmox vm right now, but I really want to have it run on the router as that is the most reliable and always-on hardware I run in my home. Even with my Proxmox host turned off entirely, my home network remains online.

[robdejonge]

Some progress! I finally figured out how to get access to the output of the container.

From that I learned that, for whatever reason, the Docker image on my Linux box loads /etc/controld/ctrld.toml as the default config file while the one on my Mikrotik router seems to want to load /ctrld.toml. As a result, it kept not finding a config file, which meant a default config was used, which explains why none of the lookups showed up in my Activity Log on the Control D website.

So I adjusted my volume mapping to use the external configuration file at the right location and now it is reporting a permission error ("failed to write config file") and then stops the container. I am not sure why it needs to be able to write to a config file, but this gives me something to chew on at least.

I shall return.

[yegors]

If you start the container with your resolver ID (--cd= flag), the config will be fetched from API and re-written every time. To avoid this, and use the mapped config you have, don't supply the resolver ID on container run.

[robdejonge]

Thanks. I prefer using a config file, rather than the command line, for the configuration.

Can you clarify why ctrld needs write access to the config file? Seems like something you should at least be able to overrule.

[yegors]

Hi,

I provided you with a solution to do exactly that. Make your own config, and enforce it without the --cd flag. Check the docs for what --cd flag does in the main README.

[robdejonge]

I understand, but why does ctrld need write access to the config file? It’s not unheard of, but also not exactly common for software to do anything but read a config file.

[yegors]

The --cd mode is special, as the config is rendered server side, or the server delivers a custom config you crafted in the web GUI. In this case, it has to write it to disk. This also serves as a "last known working" configuration, if there is an API outage, the service will still work.

If you don't use this mode (which is optional and not default), ctrld behaves like you would expect - enforce the config on disk, as is.

[robdejonge]

I do not believe that is correct. Without using the --cd mode, I believe that ctrld overwrites a custom config file if its not happy with something. I've seen it add things to my config file. Disturbing behavior if you ask me.

BUT ... progress! It seems that if write access to a file is required, the Mikrotik Docker environment needs me to mount a directory containing that file rather than just the file. For read-only access, I can just mount the file.

I now have ctrld running on my Mikrotik router, using my custom config file. And, yes, it overwrote the file! :-)

[cuonglm]

I do not believe that is correct. Without using the --cd mode, I believe that ctrld overwrites a custom config file if its not happy with something. I've seen it add things to my config file. Disturbing behavior if you ask me.

BUT ... progress! It seems that if write access to a file is required, the Mikrotik Docker environment needs me to mount a directory containing that file rather than just the file. For read-only access, I can just mount the file.

I now have ctrld running on my Mikrotik router, using my custom config file.

And, yes, it overwrote the file! :-)

I think we missed the commit to avoid un-necessary writing to config file with custom config when releasing v1.3.0 🤔

This would be fixed in next release. In general, ctrld will only write to config file when using "--cd" flag, or using custom config with undefined ip or port.

[robdejonge]

@cuonglm Thanks for the comment. I had previously noticed an undefined listener→ip being added to my config. Just now, although for the first time, it added dhcp_lease_file_path = '' and dhcp_lease_file_format='' as well.

Personally, even if things are missing and have a default value, I'd expect a program to write something to the log like "No IP configured, using default of 0.0.0.0" ... but not for it to overwrite my config file.

[cuonglm]

I had previously noticed an undefined listener→ip being added to my config.

Just now, it added dhcp_lease_file_path = '' and dhcp_lease_file_format='' as well.

If you don't specify IP or Port in your config, ctrld will pick a working one, writing to config file that picked IP or Port. Next time you start with the same config, nothing will be written.

Just now, it added dhcp_lease_file_path = '' and dhcp_lease_file_format='' as well.

Yes, it's the default value when ctrld writing config to file.

This would be fixed in next release. In general, ctrld will only write to config file when using "--cd" flag, or using custom config with undefined ip or port.

We only missed this when cherry-pick commit from our internal VCS to Github. The actual binary does have the fix to not writing config file if not necessary.

[cuonglm]

I think we missed the commit to avoid un-necessary writing to config file with custom config when releasing v1.3.0 🤔

I added them in #78

[robdejonge]

It took me a little more tinkering before it was working well for my (admittedly non-compliant) setup. This is how I got ctrld up and running on a Mikrotik device running RouterOS 7.11. This should work from version 7.4 and above.

If this is your first container, (source)

1. Install the 'container-*.npk' package from the Extra packages zip (download)
2. Enable container mode /system/device-mode/update container=yes
3. Reference Docker Hub /container/config/set registry-url=https://registry-1.docker.io/ tmpdir=disk1/pull
4. Create a bridge for container /interface/bridge/add name=containers
5. Add an address to the bridge /ip/address/add address=172.17.0.1/24 interface=containers
6. Set up NAT for outgoing traffic /ip/firewall/nat/add chain=srcnat action=masquerade src-address=172.17.0.0/24

To add the basic container,

7. Create a virtual Ethernet interface /interface/veth/add name=veth1 address=172.17.0.2/24 gateway=172.17.0.1
8. Add the virtual interface to the bridge /interface/bridge/port add bridge=containers interface=veth1
9. Set up NAT for incoming traffic /ip/firewall/nat add action=dst-nat chain=dstnat dst-address=192.168.1.1 dst-port=53 protocol=udp to-addresses=172.17.0.2 to-ports=53
10. Add the container image /container/add remote-image=controldns/ctrld interface=veth1 root-dir=disk1/ctrld
11. Find the container index (/container/print), which will be 0 if this is your first container
12. Start the container /container/start 0

This has you up and running with a standard out of the box configuration. Obviously adjust some of the details according to you own setup.

Some extra options:

• To see container console output in your Mikrotik log, add logging=yes to command 10
• To see verbose debug output on the console output, add cmd=-vv to command 10
• To store the containers file system in memory rather than on (flash) storage, /disk/add type=tmpfs temps-max-size=100M slot=RAM and change command 10 with root-dir=RAM/ctrld
• To automatically start the container when you reboot your router, add start-on-boot=yes to command 10
• To run with your personal Control-D account/device, add cmd=--cd abcd1234 to command 10
• You can add more complicated configuration options by passing more settings in the cmd= setting of command 9, detailed here

If you prefer using an actual configuration file, save your ctrld.toml to a folder on your router and

1. Add a container mount /container/mounts/add name=ctrld_config src=disk1/ctrld dst=/etc/controld
2. Add the mount to command 10 above by adding mounts=ctrld_config
3. You can edit the config file with /file/edit disk1/ctrld/ctrld.toml

If on top of this, like me, you still want to be able to reference the INTERNAL resolver from the container (for example because you create static DNS entries for DHCP leases), add src-address=!172.17.0.2 to command 9 and configure an upstream option in your config accordingly.