AxfrProvider unable to dump when Host is a DNS name

jbertozzi commented 1 year ago

Hello,

I cannot dump my current Microsoft AD zones to octodns configuration file.

cat config/ad.yaml
providers:                           
  ad:                              
      class: octodns_bind.AxfrSource 
      host: ad.example.org       
      port: 53
  config:
       class: octodns.provider.yaml.YamlProvider
       directory: ./config/zones                  
       default_ttl: 3600                        
       enforce_order: True                       
zones:
  example.org.:
    sources:
      - ad
    target:
      - config

Dumping the zone:

octodns-dump --debug --config-file=config/lady.yaml --output-dir=config/zones/ example.org. ad
2023-04-27T13:33:36  [139897537410880] INFO  Manager __init__: config_file=config/ad.yaml (octoDNS 0.9.21)
2023-04-27T13:33:36  [139897537410880] INFO  Manager _config_executor: max_workers=1
2023-04-27T13:33:36  [139897537410880] INFO  Manager _config_include_meta: include_meta=False                                                                                                                     2023-04-27T13:33:36  [139897537410880] INFO  Manager __init__: global_processors=[]
2023-04-27T13:33:36  [139897537410880] DEBUG Manager _config_providers: configuring providers
2023-04-27T13:33:36  [139897537410880] DEBUG AxfrSource[ad] __init__: id=ad, host=ad.example.org., port=53, key_name=None, key_secret=False, key_algorithm=False
2023-04-27T13:33:36  [139897537410880] INFO  Manager __init__: provider=ad (octodns_bind 0.0.2)
2023-04-27T13:33:36  [139897537410880] DEBUG YamlProvider[config] __init__: id=config, directory=./config, default_ttl=3600, enforce_order=1, populate_should_replace=0
2023-04-27T13:33:36  [139897537410880] DEBUG YamlProvider[config] __init__: id=config, apply_disabled=False, update_pcent_threshold=0.30, delete_pcent_threshold=0.30
2023-04-27T13:33:36  [139897537410880] INFO  Manager __init__: provider=config (octodns.provider.yaml 0.9.21)
2023-04-27T13:33:36  [139897537410880] INFO  Manager dump: zone=example.org., output_dir=zones/, output_provider=None, lenient=False, split=False, sources=['ad']
2023-04-27T13:33:36  [139897537410880] INFO  Manager dump: using custom YamlProvider
2023-04-27T13:33:36  [139897537410880] DEBUG YamlProvider[dump] __init__: id=dump, directory=zones/, default_ttl=3600, enforce_order=1, populate_should_replace=0
2023-04-27T13:33:36  [139897537410880] DEBUG YamlProvider[dump] __init__: id=dump, apply_disabled=False, update_pcent_threshold=0.30, delete_pcent_threshold=0.30
2023-04-27T13:33:36  [139897537410880] DEBUG Zone __init__: zone=Zone<example.org.>, sub_zones=set()
2023-04-27T13:33:36  [139897537410880] DEBUG AxfrSource[ad] populate: name=example.org., target=False, lenient=False
Traceback (most recent call last):
  File "/var/tmp/env/lib64/python3.11/site-packages/dns/inet.py", line 90, in af_for_address
    dns.ipv4.inet_aton(text)
  File "/var/tmp/env/lib64/python3.11/site-packages/dns/ipv4.py", line 57, in inet_aton
    raise dns.exception.SyntaxError
dns.exception.SyntaxError: Text input is malformed.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/var/tmp/env/lib64/python3.11/site-packages/dns/inet.py", line 95, in af_for_address
    dns.ipv6.inet_aton(text, True)
  File "/var/tmp/env/lib64/python3.11/site-packages/dns/ipv6.py", line 181, in inet_aton
    raise dns.exception.SyntaxError
dns.exception.SyntaxError: Text input is malformed.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/var/tmp/env/bin/octodns-dump", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/var/tmp/env/lib64/python3.11/site-packages/octodns/cmds/dump.py", line 48, in main
    manager.dump(
  File "/var/tmp/env/lib64/python3.11/site-packages/octodns/manager.py", line 755, in dump
    source.populate(zone, lenient=lenient)
  File "/var/tmp/env/lib64/python3.11/site-packages/octodns_bind/__init__.py", line 53, in populate
    rrs = self.zone_records(zone)
          ^^^^^^^^^^^^^^^^^^^^^^^
  File "/var/tmp/env/lib64/python3.11/site-packages/octodns_bind/__init__.py", line 184, in zone_records
    z = dns.zone.from_xfr(
        ^^^^^^^^^^^^^^^^^^
  File "/var/tmp/env/lib64/python3.11/site-packages/dns/zone.py", line 1369, in from_xfr
    for r in xfr:
  File "/var/tmp/env/lib64/python3.11/site-packages/dns/query.py", line 1221, in xfr
   File "/var/tmp/env/lib64/python3.11/site-packages/dns/query.py", line 1221, in xfr
    (af, destination, source) = _destination_and_source(
                                ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/var/tmp/env/lib64/python3.11/site-packages/dns/query.py", line 221, in _destination_and_source
    af = dns.inet.af_for_address(where)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/var/tmp/env/lib64/python3.11/site-packages/dns/inet.py", line 98, in af_for_address
    raise ValueError
ValueError

I am able to correctly get an axfr using dig:

dig @ad.example.org -t axfr example.org
#truncated
;; Query time: 320 msec
;; SERVER: 10.33.0.100#53(ad.example.org) (TCP)
;; WHEN: Thu Apr 27 13:45:55 CEST 2023
;; XFR size: 8091 records (messages 16, bytes 261872)

Adding some print() into the code, the problematic record seems to be c5d1c200-2ce8-42b3-bde4-72aea693b480._msdcs.example.org 600 IN CNAME ad.example.org. although it seems valid to me.

Did I do anything wrong?

yzguy commented 1 year ago

This is not due to the record, it's actually because dnspython seemingly doesn't allow you to put a DNS name in the host variable, it requires an IP. If you put in an IP instead of ad.example.org does it error out? If so then that confirms it. The dns.inet.af_for_address is used to determine what address family it is, which obviously a DNS name cannot be used to determine if it's IPv4/IPv6 and therefore it errors.

Surprised dnspython doesn't allow that, per the doc for dns.query.xfr it expects an IPv4 or IPv6 address.

I don't see it being unreasonable to just detect if it's an IP or a DNS and do a quick query for the IP and feed that in. We could use dnspython to do it, but I suspect just using socket.gethostbyname would be easier and not require us to specify a record type (A/CNAME/etc) like dnspython requires.

I do note our providers docs do show using a DNS name (which I created and I believe I did actually test against), so I'll just to test again just to confirm, it's been a bit so I can't actually be sure if I did. Starting to think I didn't though.

Addressing #10 may be a solution as well, if the functionality of that new function handles DNS names.

ross commented 1 year ago

Gonna move this over into https://github.com/octodns/octodns-bind since the AxfrSource lives there now

Edit: doh, looks like @yzguy beat me to it. That's what I get for going off the notification email and not double checking once I landed here.

jbertozzi commented 1 year ago

I confirm that using the ip address in the host variable fixed it.

Thank you for the hint.

yzguy commented 1 year ago

@jbertozzi wonderful, thank you for checking. I'll take a look this weekend hopefully to add the functionality to handle if it's a DNS name. It's definitely desirable functionality

yzguy commented 1 year ago

@jbertozzi This fix is now in v0.0.3 release of the provider

octodns / octodns-bind

AxfrProvider unable to dump when Host is a DNS name #18