search

opnsense / plugins

OPNsense plugin collection
https://opnsense.org/
BSD 2-Clause "Simplified" License
849 stars 644 forks source link

WIP: Fix 4352 #4353

Open Jeremy-Boyle opened 1 week ago

Jeremy-Boyle commented 1 week ago

This issue fixes #4352

image image image image image

Jeremy-Boyle commented 1 week ago

One issue to note here this does remove

general.rndcalgo and general.rndsecret,

A migration will have to be made to migrate this key to the new keys. Primary domains will also have to migrate if domain.allowrndctransfer or domain.allowrndcupdate was enabled,

Flow for migration.

I'm new to how migrations work and will need help or at least some pointers on how to test migrations, I have created a migration .php file, which has not been pushed yet because I am unsure how that works can someone please assist with the migrations ?

Jeremy-Boyle commented 1 week ago

@NHellFire @sestary Any idea how we can sync from the database file back into opnsense for transfers ? I have solved the issue with the journal restarts, running rndc sync, freeze and thaw poplulates the db file now. But is still removed on config save.

Looks like we would have to parse the db file and update the model, I'm not sure what the best option to do this would be ?

Jeremy-Boyle commented 1 week ago

https://github.com/opnsense/plugins/blob/master/www/OPNProxy/src/opnsense/scripts/OPNProxy/redis_sync_users.py

This is one option that i see for example, i would like to prevent opening the config file if possible. But this is a option, that potentially could have a cron or something that sits and watches for when it needs to update.

Jeremy-Boyle commented 1 week ago

I did see this

https://github.com/opnsense/plugins/pull/3258

@sestary what was the reasoning behind moving this to the general field ?

With the functionality provided inside of https://github.com/opnsense/plugins/pull/4177 does this now make sense, also, this would also allow in a future request to migrate secondary zones to also use the keys as well.

NHellFire commented 1 week ago

@NHellFire @sestary Any idea how we can sync from the database file back into opnsense for transfers ? I have solved the issue with the journal restarts, running rndc sync, freeze and thaw poplulates the db file now. But is still removed on config save.

Looks like we would have to parse the db file and update the model, I'm not sure what the best option to do this would be ?

An ideal solution would be to parse the journal with named-journalprint and sync those updates with the opnsense config, then delete the journal. That way the updated records would be visible in the UI and not be lost when manually editing the zone.

root@OPNsense:~ # named-journalprint /usr/local/etc/namedb/primary/dev.k0s.home.db.jnl
del dev.k0s.home.       86400   IN  SOA ns.dev.k0s.home. mail.opnsense.localdomain. 2411151601 21600 3600 3542400 3600
add dev.k0s.home.       86400   IN  SOA ns.dev.k0s.home. mail.opnsense.localdomain. 2411151602 21600 3600 3542400 3600
add dashboard.dev.k0s.home. 0   IN  A   198.18.28.1
add external-dnsdashboard.dev.k0s.home. 0 IN TXT    "heritage=external-dns,external-dns/owner=dev,external-dns/resource=ingress/kubernetes-dashboard/kubernetes-dashboard"
add external-dnsa-dashboard.dev.k0s.home. 0 IN TXT  "heritage=external-dns,external-dns/owner=dev,external-dns/resource=ingress/kubernetes-dashboard/kubernetes-dashboard"

Alternatively, rather than processing all the individual updates manually, we could just process the updated zone and delete any records from the opnsense config that are no longer in the zone:

root@OPNsense:~ # named-checkzone -Djs full dev.k0s.home /usr/local/etc/namedb/primary/dev.k0s.home.db
dev.k0s.home.                     86400 IN SOA  ns.dev.k0s.home. mail.opnsense.localdomain. 2411151639 21600 3600 3542400 3600
dev.k0s.home.                     86400 IN NS   ns.dev.k0s.home.
dashboard.dev.k0s.home.               0 IN A        198.18.28.1
external-dnsa-dashboard.dev.k0s.home.         0 IN TXT      "heritage=external-dns,external-dns/owner=dev,external-dns/resource=ingress/kubernetes-dashboard/kubernetes-dashboard"
external-dnsdashboard.dev.k0s.home.       0 IN TXT      "heritage=external-dns,external-dns/owner=dev,external-dns/resource=ingress/kubernetes-dashboard/kubernetes-dashboard"

We'd need to ensure that the journal is has been applied to the opnsense config any time we're listing or modifying the zone to avoid conflicts.

Both formats are easy enough to parse, I'm just not sure how to go about integrating that

Jeremy-Boyle commented 1 week ago

Hi @NHellFire , yes i thought of that same example and started writing some code to do that.

One option i was thinking was to hook it with the PHP, to open the file and create the record and handle the relations, However, this would have a issue with that you would have to have a button to sync it that the user would have to push or you could hook it to do the check every time someone goes to look at it. Both didn't really seem like good options. I have not looked around for any examples, but would it be possible to add a php based file cron that would run periodically that would allow this ? That way we can avoid manually editing the config ?

What do you think about this ?

When user goes to page -> function will run manually when looking at any records for that zone -> adds them before grabbing the records -> records are populated in the ui. This is a potential example https://github.com/opnsense/plugins/blob/598cf8ccd321bf6980d84f3aef63c312ac606130/dns/bind/src/opnsense/mvc/app/controllers/OPNsense/Bind/Api/RecordController.php#L60 of how the search works. Would have to add something in general I think to kick it off ?

Cron -> syncs at a configurable time interval that can be set by the user?

Alternatively, rather than processing all the individual updates manually, we could just process the updated zone and delete any records from the opnsense config that are no longer in the zone

This could be a good way forward, however once we sync it with the config, it will move that record into the primary db. With the fix that i have here it will add it to the primary with syncing / freezing, that removes it from the journal . So always reading the journal as the might not be 100% i would have to check how that works.