Hello, I've created an dns operator to manage dns instances and records and it works well when create/update/delete resources one by one, but when I create multiple resources using a yaml file only some of them are processed by kopf completely and they are executed in a order different than created (in my use case the dnsrecord is created before the dnsserver is fully processed so it fails).
Kopf has some protection measures against quickly arriving events — the "batch window" (settings.batching.batch_window, defaults to 0.1 seconds). All events that arrived within this time window are reduced only to the latest event, which means the latest object state. All intermediate events/states are ignored.
I don't know if that could be related.
Kopf version
1.35.6
Kubernetes version
1.22
Python version
No response
Code
https://github.com/davidp1404/dns-operator)
Logs
$ kopf run --priority=10 --all-namespaces src/dns-operator.py
[2022-09-09 18:08:48,282] kopf.activities.star [INFO ] Activity 'configure' succeeded.
[2022-09-09 18:08:48,283] kopf._core.engines.a [INFO ] Initial authentication has been initiated.
[2022-09-09 18:08:48,323] kopf.activities.auth [INFO ] Activity 'login_via_client' succeeded.
[2022-09-09 18:08:48,324] kopf._core.engines.a [INFO ] Initial authentication has finished.
[2022-09-09 18:09:07,450] kopf.objects [INFO ] [default/record1] Adding Record "record1 IN A 10.0.0.2
[2022-09-09 18:09:07,485] kopf.objects [INFO ] [default/record3] Adding Record "3 IN PTR record3.sample.org.
[2022-09-09 18:09:07,488] kopf.objects [INFO ] [default/record4] Adding Record "4 IN PTR record4.sample.org.
[2022-09-09 18:09:07,489] kopf.objects [INFO ] [default/record5] Adding Record "5 IN PTR record5.sample.org.
[2022-09-09 18:09:07,505] kopf.objects [INFO ] [default/record2] Adding Record "record2 IN A 10.0.0.1
[2022-09-09 18:09:07,502] kopf.objects [INFO ] [default/record6] Adding Record "record6 IN CNAME record1.sample.org.
[2022-09-09 18:09:07,539] kopf.objects [INFO ] [default/record2] Adding Record "record2 IN A 10.0.0.3
[2022-09-09 18:09:17,655] kopf.objects [INFO ] [default/record1] Rollout deployment finish without errors
[2022-09-09 18:09:17,660] kopf.objects [INFO ] [default/record1] Handler 'create_dnsrecords' succeeded.
[2022-09-09 18:09:17,662] kopf.objects [INFO ] [default/record1] Creation is processed: 1 succeeded; 0 failed.
[2022-09-09 18:09:17,697] kopf.objects [INFO ] [default/record3] Rollout deployment finish without errors
[2022-09-09 18:09:17,702] kopf.objects [INFO ] [default/record3] Handler 'create_dnsrecords' succeeded.
[2022-09-09 18:09:17,703] kopf.objects [INFO ] [default/record3] Creation is processed: 1 succeeded; 0 failed.
[2022-09-09 18:09:17,770] kopf.objects [INFO ] [default/record5] Rollout deployment finish without errors
[2022-09-09 18:09:17,773] kopf.objects [INFO ] [default/record5] Handler 'create_dnsrecords' succeeded.
[2022-09-09 18:09:17,774] kopf.objects [INFO ] [default/record5] Creation is processed: 1 succeeded; 0 failed.
[2022-09-09 18:09:17,865] kopf.objects [INFO ] [default/record2] Rollout deployment finish without errors
[2022-09-09 18:09:17,869] kopf.objects [INFO ] [default/record2] Handler 'create_dnsrecords' succeeded.
[2022-09-09 18:09:17,870] kopf.objects [INFO ] [default/record2] Creation is processed: 1 succeeded; 0 failed.
[2022-09-09 18:09:17,886] kopf.objects [INFO ] [default/record6] Rollout deployment finish without errors
[2022-09-09 18:09:17,895] kopf.objects [INFO ] [default/record6] Handler 'create_dnsrecords' succeeded.
[2022-09-09 18:09:17,895] kopf.objects [INFO ] [default/record6] Creation is processed: 1 succeeded; 0 failed.
[2022-09-09 18:09:17,941] kopf.objects [INFO ] [default/record4] Rollout deployment finish without errors
[2022-09-09 18:09:17,944] kopf.objects [INFO ] [default/record4] Handler 'create_dnsrecords' succeeded.
[2022-09-09 18:09:17,945] kopf.objects [INFO ] [default/record4] Creation is processed: 1 succeeded; 0 failed.
Additional information
In previous logs, all the records seem to be processed but the changes were not shown in the configmap, something that works fine if I create/delete/update records one by one.
In this example record 3 an 4 are not in configmap and kopf doesn't show any error
`$ k get cm dns-operator-sample-org -o yaml
apiVersion: v1
data:
Corefile: |
.:1053 {
root /etc/coredns
errors
health
ready
loadbalance round_robin
prometheus :9153
reload 2s 1s
log
file db.sample.org sample.org
file db.0.0.10.in-addr.arpa 0.0.10.in-addr.arpa
file db.example.org example.org
file db.0.1.10.in-addr.arpa 0.1.10.in-addr.arpa
}
db.0.0.10.in-addr.arpa: |
$ORIGIN 0.0.10.in-addr.arpa.
@ 3600 IN SOA sns.dns.icann.org. noc.dns.icann.org. (
2017042761 ; serial
7200 ; refresh (2 hours)
3600 ; retry (1 hour)
1209600 ; expire (2 weeks)
3600 ; minimum (1 hour)
)
3600 IN NS a.iana-servers.net.
3600 IN NS b.iana-servers.net.
5 IN PTR record5.sample.org.
db.0.1.10.in-addr.arpa: |
$ORIGIN 0.1.10.in-addr.arpa.
@ 3600 IN SOA sns.dns.icann.org. noc.dns.icann.org. (
2017042761 ; serial
7200 ; refresh (2 hours)
3600 ; retry (1 hour)
1209600 ; expire (2 weeks)
3600 ; minimum (1 hour)
)
3600 IN NS a.iana-servers.net.
3600 IN NS b.iana-servers.net.
db.example.org: |
$ORIGIN example.org.
@ 3600 IN SOA sns.dns.icann.org. noc.dns.icann.org. (
2017042761 ; serial
7200 ; refresh (2 hours)
3600 ; retry (1 hour)
1209600 ; expire (2 weeks)
3600 ; minimum (1 hour)
)
3600 IN NS a.iana-servers.net.
3600 IN NS b.iana-servers.net.
db.sample.org: |
$ORIGIN sample.org.
@ 3600 IN SOA sns.dns.icann.org. noc.dns.icann.org. (
2017042761 ; serial
7200 ; refresh (2 hours)
3600 ; retry (1 hour)
1209600 ; expire (2 weeks)
3600 ; minimum (1 hour)
)
3600 IN NS a.iana-servers.net.
3600 IN NS b.iana-servers.net.
record1 IN A 10.0.0.2
record2 IN A 10.0.0.1
record2 IN A 10.0.0.3
kind: ConfigMap
metadata:
creationTimestamp: "2022-09-09T15:39:59Z"
labels:
app: dns-operator-sample-org
app.kubernetes.io/managed-by: dns-operator
name: dns-operator-sample-org
namespace: default
ownerReferences:
Long story short
Hello, I've created an dns operator to manage dns instances and records and it works well when create/update/delete resources one by one, but when I create multiple resources using a yaml file only some of them are processed by kopf completely and they are executed in a order different than created (in my use case the dnsrecord is created before the dnsserver is fully processed so it fails).
I've read in issue 729 this:
I don't know if that could be related.
Kopf version
1.35.6
Kubernetes version
1.22
Python version
No response
Code
Logs
Additional information
In previous logs, all the records seem to be processed but the changes were not shown in the configmap, something that works fine if I create/delete/update records one by one. In this example record 3 an 4 are not in configmap and kopf doesn't show any error
`$ k get cm dns-operator-sample-org -o yaml apiVersion: v1 data: Corefile: | .:1053 { root /etc/coredns errors health ready loadbalance round_robin prometheus :9153 reload 2s 1s log file db.sample.org sample.org file db.0.0.10.in-addr.arpa 0.0.10.in-addr.arpa file db.example.org example.org file db.0.1.10.in-addr.arpa 0.1.10.in-addr.arpa } db.0.0.10.in-addr.arpa: | $ORIGIN 0.0.10.in-addr.arpa. @ 3600 IN SOA sns.dns.icann.org. noc.dns.icann.org. ( 2017042761 ; serial 7200 ; refresh (2 hours) 3600 ; retry (1 hour) 1209600 ; expire (2 weeks) 3600 ; minimum (1 hour) ) 3600 IN NS a.iana-servers.net. 3600 IN NS b.iana-servers.net. 5 IN PTR record5.sample.org. db.0.1.10.in-addr.arpa: | $ORIGIN 0.1.10.in-addr.arpa. @ 3600 IN SOA sns.dns.icann.org. noc.dns.icann.org. ( 2017042761 ; serial 7200 ; refresh (2 hours) 3600 ; retry (1 hour) 1209600 ; expire (2 weeks) 3600 ; minimum (1 hour) ) 3600 IN NS a.iana-servers.net. 3600 IN NS b.iana-servers.net. db.example.org: | $ORIGIN example.org. @ 3600 IN SOA sns.dns.icann.org. noc.dns.icann.org. ( 2017042761 ; serial 7200 ; refresh (2 hours) 3600 ; retry (1 hour) 1209600 ; expire (2 weeks) 3600 ; minimum (1 hour) ) 3600 IN NS a.iana-servers.net. 3600 IN NS b.iana-servers.net. db.sample.org: | $ORIGIN sample.org. @ 3600 IN SOA sns.dns.icann.org. noc.dns.icann.org. ( 2017042761 ; serial 7200 ; refresh (2 hours) 3600 ; retry (1 hour) 1209600 ; expire (2 weeks) 3600 ; minimum (1 hour) ) 3600 IN NS a.iana-servers.net. 3600 IN NS b.iana-servers.net. record1 IN A 10.0.0.2 record2 IN A 10.0.0.1 record2 IN A 10.0.0.3 kind: ConfigMap metadata: creationTimestamp: "2022-09-09T15:39:59Z" labels: app: dns-operator-sample-org app.kubernetes.io/managed-by: dns-operator name: dns-operator-sample-org namespace: default ownerReferences: