search

StackExchange / dnscontrol

Infrastructure as code for DNS!
https://dnscontrol.org/
MIT License
3.13k stars 399 forks source link

Provider request: Netcup #705

Closed eifelmicha closed 4 years ago

eifelmicha commented 4 years ago

Netcup (https://www.netcup.eu/ ) is a provider located in Germany. They offer an API to handle DNS Updates. It would be nice to handle this with dnscontrol.

The Link to the API Documentation is inside this wiki page: https://www.netcup-wiki.de/wiki/DNS_API

Also i found a bash implementation that might help to easily port this to dnscontrol: https://github.com/linux-insideDE/ncdapi/blob/master/ncdapi.sh

kordianbruck commented 4 years ago

This is close to impossible. I've started to work on this, but their Soap / "JSON" api is hard to work with. The much needed listallDomains endpoint unfortunately is only available to resellers. See https://forum.netcup.de/anwendung/sonstige-anwendungen/10921-dns-api-listalldomains-freigeben/

Without this, this will be impossible.

You can try with the ncdapi.sh to run this:

./ncdapi.sh -l          
Error: {"serverrequestid":"","clientrequestid":"","action":"listallDomains","status":"error","statuscode":4020,"shortmessage":"Function not available","longmessage":"This function is available for resellers.","responsedata":""}
eifelmicha commented 4 years ago

Thx for taking a look into this. really appreciated. I liked your forum post. If it helps i can also ping the support on this, paying extra money for some SLA.

Is there atleast some possible way, if this endpoint is not opened, that if a concrete domain is set it atleast can update values?

kordianbruck commented 4 years ago

I'd need to look more into how dnscontrol does the checking for the diff. That might work with only some basic functionality. I'll work on this more next weekend. I got the fetching for one domain and all RRs working already. Might be just enough.

tlimoncelli commented 4 years ago

dnscontrol shouldn't require anything like listallDomains for just DNS updates. For example, BIND isn't able to list all domains. Dnscontrol assumes that the domains listed in dnsconfig.js exist. If the assumption is wrong, the API will give an error when the updates are done.

I am sorry that you have to deal with SOAP. It is a terrible protocol. I even make a joke about it in this article: https://queue.acm.org/detail.cfm?id=3375635

Hopefully the humor in that article will make you smile and give you the fortitude to continue trying to make a Netcup provider for Dnscontrol.

Best, Tom

kordianbruck commented 4 years ago

Thanks @tlimoncelli - great article :+1:

I've drafted some changes in the PR #718 - @eifelmicha if you have some time to test, that'd be great. Probably still a few problems in there that I haven't noticed yet.

eifelmicha commented 4 years ago

@kordianbruck Thx very much. Did some simple tests with cname,caa,a stuff and it worked.

tlimoncelli commented 4 years ago

Could you run this test?

cd PATH_AS_NEEDED/dnscontrol/integrationTest
export NETCUP_CUSTOMER_NUMBER="FILL IN"
export NETCUP_KEY="FILL IN"
export NETCUP_PASSWORD="FILL IN"
export NETCUP_DOMAIN=domain.tld
go test -v -verbose -provider NETCUP

If those tests pass, we're basically done.

eifelmicha commented 4 years ago 
--- FAIL: TestDNSProviders (294.42s)
    --- FAIL: TestDNSProviders/fqdm.example (294.06s)
        --- PASS: TestDNSProviders/fqdm.example/Clean_Slate:Empty (2.54s)
        --- PASS: TestDNSProviders/fqdm.example/00:A:Create_an_A_record (2.46s)
        --- PASS: TestDNSProviders/fqdm.example/00:A:Change_it (0.76s)
        --- PASS: TestDNSProviders/fqdm.example/00:A:Add_another (2.40s)
        --- PASS: TestDNSProviders/fqdm.example/00:A:Add_another(same_name) (2.41s)
        --- FAIL: TestDNSProviders/fqdm.example/00:A:Change_a_ttl (0.23s)
        --- PASS: TestDNSProviders/fqdm.example/Post_cleanup:Empty (1.12s)
        --- PASS: TestDNSProviders/fqdm.example/01:CNAME:Create_a_CNAME (2.47s)
        --- PASS: TestDNSProviders/fqdm.example/Post_cleanup:Empty#01 (0.51s)
        --- PASS: TestDNSProviders/fqdm.example/02:MX:MX_record (2.46s)
        --- PASS: TestDNSProviders/fqdm.example/Post_cleanup:Empty#02 (0.62s)
        --- FAIL: TestDNSProviders/fqdm.example/03:Null_MX:Null_MX (0.65s)
        --- PASS: TestDNSProviders/fqdm.example/Post_cleanup:Empty#03 (0.22s)
        --- FAIL: TestDNSProviders/fqdm.example/04:NS:NS_for_subdomain (0.23s)
        --- PASS: TestDNSProviders/fqdm.example/Post_cleanup:Empty#04 (0.23s)
        --- PASS: TestDNSProviders/fqdm.example/05:IGNORE_function:Create_some_records (4.35s)
        --- PASS: TestDNSProviders/fqdm.example/Post_cleanup:Empty#05 (0.80s)
        --- PASS: TestDNSProviders/fqdm.example/06:single_TXT:Create_a_TXT (2.34s)
        --- PASS: TestDNSProviders/fqdm.example/Post_cleanup:Empty#06 (0.51s)
        --- PASS: TestDNSProviders/fqdm.example/07:ws_TXT:Change_a_TXT_with_ws_at_end (2.39s)
        --- PASS: TestDNSProviders/fqdm.example/Post_cleanup:Empty#07 (0.56s)
        --- FAIL: TestDNSProviders/fqdm.example/08:empty_TXT:TXT_with_empty_str (0.64s)
        --- PASS: TestDNSProviders/fqdm.example/Post_cleanup:Empty#08 (0.22s)
        --- PASS: TestDNSProviders/fqdm.example/09:Case_Sensitivity:Create_CAPS (2.33s)
        --- PASS: TestDNSProviders/fqdm.example/Post_cleanup:Empty#09 (0.50s)
        --- PASS: TestDNSProviders/fqdm.example/10:IDNA:Internationalized_name (2.32s)
        --- PASS: TestDNSProviders/fqdm.example/Post_cleanup:Empty#10 (0.54s)
        --- PASS: TestDNSProviders/fqdm.example/11:IDNAs_in_CNAME_targets:IDN_CNAME_AND_Target (2.44s)
        --- PASS: TestDNSProviders/fqdm.example/Post_cleanup:Empty#11 (0.46s)
        --- PASS: TestDNSProviders/fqdm.example/12:page_size:99_records (205.42s)
        --- PASS: TestDNSProviders/fqdm.example/Post_cleanup:Empty#12 (29.43s)
        --- PASS: TestDNSProviders/fqdm.example/13:Large_updates_***SKIPPED(disabled_by_only)***:Empty (0.24s)
        --- PASS: TestDNSProviders/fqdm.example/14:CAA:CAA_record (2.47s)
        --- PASS: TestDNSProviders/fqdm.example/Post_cleanup:Empty#13 (0.49s)
        --- PASS: TestDNSProviders/fqdm.example/15:CAA_with_;:CAA_many_records (2.58s)
        --- PASS: TestDNSProviders/fqdm.example/Post_cleanup:Empty#14 (0.51s)
        --- PASS: TestDNSProviders/fqdm.example/16:NAPTR_***SKIPPED(CanUseNAPTR_not_supported)***:Empty (0.21s)
        --- PASS: TestDNSProviders/fqdm.example/17:PTR_***SKIPPED(CanUsePTR_not_supported)***:Empty (0.39s)
        --- PASS: TestDNSProviders/fqdm.example/18:SRV:SRV_record (2.44s)
        --- PASS: TestDNSProviders/fqdm.example/Post_cleanup:Empty#15 (0.50s)
        --- PASS: TestDNSProviders/fqdm.example/19:SRV_w/_null_target:Null_Target (4.56s)
        --- PASS: TestDNSProviders/fqdm.example/Post_cleanup:Empty#16 (0.83s)
        --- PASS: TestDNSProviders/fqdm.example/20:SSHFP_***SKIPPED(CanUseSSHFP_not_supported)***:Empty (0.26s)
        --- PASS: TestDNSProviders/fqdm.example/21:TLSA_***SKIPPED(CanUseTLSA_not_supported)***:Empty (0.25s)
        --- PASS: TestDNSProviders/fqdm.example/22:TXTMulti:Create_TXTMulti_1 (2.48s)
        --- PASS: TestDNSProviders/fqdm.example/Post_cleanup:Empty#17 (0.54s)
        --- PASS: TestDNSProviders/fqdm.example/23:ALIAS_***SKIPPED(CanUseAlias_not_supported)***:Empty (0.28s)
        --- PASS: TestDNSProviders/fqdm.example/24:AZURE_ALIAS_***SKIPPED(CanUseAzureAlias_not_supported)***:Empty (0.25s)
        --- PASS: TestDNSProviders/fqdm.example/25:R53_ALIAS_***SKIPPED(CanUseRoute53Alias_not_supported)***:Empty (0.22s)
=== RUN   TestDualProviders
    TestDualProviders: integration_test.go:272: Clearing everything
    TestDualProviders: integration_test.go:278: Adding nameservers from another provider
    TestDualProviders: integration_test.go:281: Running again to ensure stability
--- PASS: TestDualProviders (0.90s)
FAIL
exit status 1
FAIL    github.com/StackExchange/dnscontrol/v3/integrationTest  295.328s

Tested with 1 domain of mine and go version go1.14.2 linux/amd64

tlimoncelli commented 4 years ago 
        --- FAIL: TestDNSProviders/fqdm.example/00:A:Change_a_ttl (0.23s)

Since TTLs aren't supported, you might consider changing the code to set all TTLs to 300 (on both the desired and found data) right before the diff is done. Also add a note (if it doesn't already exist) in the docs warning people about that.

It would be awesome if get-zones installed a default TTL like 300 rather than 0.

        --- FAIL: TestDNSProviders/fqdm.example/04:NS:NS_for_subdomain (0.23s)

I would just skip this test. On integrationTest/integration_test.go:618 add the provider to not("DNSIMPLE", "EXOSCALE"),

        --- FAIL: TestDNSProviders/fqdm.example/03:Null_MX:Null_MX (0.65s)

Skip this test. Modify the not() near integrationTest/integration_test.go:612

        --- FAIL: TestDNSProviders/fqdm.example/08:empty_TXT:TXT_with_empty_str (0.64s)

I'd just skip this text too. Look near integration_test.go:651

kordianbruck commented 4 years ago

It would be awesome if get-zones installed a default TTL like 300 rather than 0.

0 is more obvious that its not supported though right? The 300 might confuse users even more.

I would just skip this test. On integrationTest/integration_test.go:618 add the provider to not("DNSIMPLE", "EXOSCALE"),

done

Skip this test. Modify the not() near integrationTest/integration_test.go:612

done

I'd just skip this text too. Look near integration_test.go:651

done

tlimoncelli commented 4 years ago

Good point. What TTL value is returned by the API when you get the zone? It is probably best to just update the desired records to store that value.

kordianbruck commented 4 years ago

The API doesn't have a value for any RR TTL. I didn't implement getting the zone, because surprise, that is a feature only available to resellers. (At least it didn't work for me)

tlimoncelli commented 4 years ago

Ah! Well then, 0 should be just fine! :-)

kordianbruck commented 4 years ago

Thanks @tlimoncelli for your help and for merging this :tada: