search

octodns / octodns-cloudflare

Cloudflare DNS provider for octoDNS
MIT License
24 stars 17 forks source link

Unknown provider class: octodns_cloudflare.CloudflareProvider #66

Closed jaspermayone closed 10 months ago

jaspermayone commented 10 months ago

`2023-10-29T18:26:27 [8179962624] INFO Manager init: config_file=./config/production.yaml, (octoDNS 1.2.1) 2023-10-29T18:26:27 [8179962624] INFO Manager _config_executor: max_workers=1 2023-10-29T18:26:27 [8179962624] INFO Manager _config_include_meta: include_meta=False 2023-10-29T18:26:27 [8179962624] INFO Manager _config_auto_arpa: auto_arpa=False 2023-10-29T18:26:27 [8179962624] INFO Manager init: global_processors=[] 2023-10-29T18:26:27 [8179962624] INFO Manager init: global_post_processors=[] 2023-10-29T18:26:27 [8179962624] INFO Manager init: provider=config (octodns.provider.yaml 1.2.1) 2023-10-29T18:26:27 [8179962624] ERROR Manager get{}_class: Unable to import module octodns_cloudflare.CloudflareProvider Traceback (most recent call last): File "/Users/jasper/Desktop/Coding/dns-old/venv/lib/python3.11/site-packages/octodns/manager.py", line 330, in _get_named_class module, version = self._import_module(module_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/jasper/Desktop/Coding/dns-old/venv/lib/python3.11/site-packages/octodns/manager.py", line 316, in _import_module module = import_module(current) ^^^^^^^^^^^^^^^^^^^^^^ File "/opt/homebrew/Cellar/python@3.11/3.11.6/Frameworks/Python.framework/Versions/3.11/lib/python3.11/importlib/init.py", line 126, in import_module return _bootstrap._gcd_import(name[level:], package, level) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "", line 1204, in _gcd_import File "", line 1176, in _find_and_load File "", line 1140, in _find_and_load_unlocked ModuleNotFoundError: No module named 'octodns_cloudflare' Traceback (most recent call last): File "/Users/jasper/Desktop/Coding/dns-old/venv/lib/python3.11/site-packages/octodns/manager.py", line 330, in _get_named_class module, version = self._import_module(module_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/jasper/Desktop/Coding/dns-old/venv/lib/python3.11/site-packages/octodns/manager.py", line 316, in _import_module module = import_module(current) ^^^^^^^^^^^^^^^^^^^^^^ File "/opt/homebrew/Cellar/python@3.11/3.11.6/Frameworks/Python.framework/Versions/3.11/lib/python3.11/importlib/init.py", line 126, in import_module return _bootstrap._gcd_import(name[level:], package, level) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "", line 1204, in _gcd_import File "", line 1176, in _find_and_load File "", line 1140, in _find_and_load_unlocked ModuleNotFoundError: No module named 'octodns_cloudflare'

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/Users/jasper/Desktop/Coding/dns-old/venv/bin/octodns-dump", line 8, in sys.exit(main()) ^^^^^^ File "/Users/jasper/Desktop/Coding/dns-old/venv/lib/python3.11/site-packages/octodns/cmds/dump.py", line 50, in main manager = Manager(args.config_file) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/jasper/Desktop/Coding/dns-old/venv/lib/python3.11/site-packages/octodns/manager.py", line 123, in init self.providers = self._config_providers(providers_config) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/jasper/Desktop/Coding/dns-old/venv/lib/python3.11/site-packages/octodns/manager.py", line 215, in _config_providers _class, module, version = self._get_named_class( ^^^^^^^^^^^^^^^^^^^^^^ File "/Users/jasper/Desktop/Coding/dns-old/venv/lib/python3.11/site-packages/octodns/manager.py", line 335, in _get_named_class raise ManagerException( octodns.manager.ManagerException: Unknown provider class: octodns_cloudflare.CloudflareProvider, ./config/production.yaml, line 8, column 5`

ross commented 10 months ago

Can you verify that octodns-cloudflare is installed.

Assuming you are using pip, pip freeze should list everything out.

If it's not in the list the install instructions are in this repo's README.md

jaspermayone commented 10 months ago

I am using pip.

Output from pip freeze

WARNING: Skipping /opt/homebrew/lib/python3.11/site-packages/Pygments-2.16.1.dist-info due to invalid metadata entry 'name'
WARNING: Skipping /opt/homebrew/lib/python3.11/site-packages/packaging-23.2.dist-info due to invalid metadata entry 'name'
WARNING: Skipping /opt/homebrew/lib/python3.11/site-packages/certifi-2023.7.22.dist-info due to invalid metadata entry 'name'
WARNING: Skipping /opt/homebrew/lib/python3.11/site-packages/pyparsing-3.1.1.dist-info due to invalid metadata entry 'name'
alabaster==0.7.13
ansible==8.1.0
ansible-core==2.15.1
anytree==2.8.0
attrs==23.1.0
autopage==0.5.1
Babel==2.12.1
beautifulsoup4==4.12.2
blinker==1.6.2
bullet==2.2.0
cachetools==5.3.1
certifi==2023.5.7
cffi @ file:///private/tmp/cffi-20231014-5567-1hyue8f/cffi-1.16.0
chardet==5.1.0
charset-normalizer==3.1.0
click==8.1.3
cliff==4.3.0
cmd2==2.4.3
colorama==0.4.6
configobj==5.0.8
configparser==5.3.0
cryptography==41.0.1
distlib==0.3.6
dnspython==2.4.2
docutils==0.20.1
et-xmlfile==1.1.0
filelock==3.12.2
Flask==2.3.2
Flask-Login==0.6.2
fqdn==1.5.1
gnupg==2.3.1
humanfriendly==10.0
idna==3.4
imagesize==1.4.1
importlib-metadata==6.6.0
ipwhois==1.2.0
itsdangerous==2.1.2
Jinja2==3.1.2
lxml==4.9.2
MarkupSafe==2.1.3
mccabe==0.7.0
natsort==8.4.0
octodns==1.2.1
openpyxl==3.1.2
ordered-set==4.1.0
packaging==23.1
pbr==5.11.1
pexpect==4.8.0
platformdirs==3.5.3
pluggy==1.0.0
prettytable==3.8.0
property-manager==3.0
protobuf==4.24.4
psutil==5.9.5
ptyprocess==0.7.0
PyAFF==0.1
pycparser @ file:///private/tmp/pycparser-20231013-5787-1anpge2/pycparser-2.21
pyflakes==3.0.1
Pygments==2.14.0
PyLaTeX==1.4.1
pymarc==5.1.0
pyperclip==1.8.2
pypng==0.20220715.0
pyproject-api==1.5.2
python-dateutil==2.8.2
python-nmap==0.7.1
python-secrets==23.4.2
python3-nmap==1.6.0
PyYAML @ file:///private/tmp/pyyaml-20230913-6026-7ryz4v/PyYAML-6.0.1
qrcode==7.4.2
requests==2.31.0
resolvelib==1.0.1
setuptools-scm==7.1.0
simplejson==3.19.1
six==1.16.0
snowballstemmer==2.2.0
soupsieve==2.4.1
Sphinx==7.0.1
sphinxcontrib-applehelp==1.0.4
sphinxcontrib-devhelp==1.0.2
sphinxcontrib-htmlhelp==2.0.1
sphinxcontrib-jsmath==1.0.1
sphinxcontrib-qthelp==1.0.3
sphinxcontrib-serializinghtml==1.1.5
stevedore==5.1.0
tox==4.6.2
typing_extensions==4.6.3
urllib3==2.0.3
verboselogs==1.7
virtualenv==20.23.1
wcwidth==0.2.6
Werkzeug==2.3.6
xkcdpass==1.19.3
XlsxWriter==3.1.2
yamlreader==3.0.4
zipp==3.15.0
jaspermayone commented 10 months ago

oh... hold on it is reinstalled now

jaspermayone commented 10 months ago

❯ octodns-validate --config-file=config.yaml 2023-10-29T20:30:00 [8179962624] ERROR Manager Invalid provider config Traceback (most recent call last): File "/opt/homebrew/lib/python3.11/site-packages/octodns/manager.py", line 359, in _build_kwargs v = environ[env_var]


  File "<frozen os>", line 679, in __getitem__
KeyError: 'CLOUDFLARE_TOKEN'
Traceback (most recent call last):
  File "/opt/homebrew/lib/python3.11/site-packages/octodns/manager.py", line 359, in _build_kwargs
    v = environ[env_var]
        ~~~~~~~^^^^^^^^^
  File "<frozen os>", line 679, in __getitem__
KeyError: 'CLOUDFLARE_TOKEN'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/homebrew/bin/octodns-validate", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/octodns/cmds/validate.py", line 43, in main
    manager = Manager(args.config_file)
              ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/octodns/manager.py", line 123, in __init__
    self.providers = self._config_providers(providers_config)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/octodns/manager.py", line 218, in _config_providers
    kwargs = self._build_kwargs(provider_config)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/octodns/manager.py", line 362, in _build_kwargs
    raise ManagerException(
octodns.manager.ManagerException: Incorrect provider config, missing env var CLOUDFLARE_TOKEN, config.yaml, line 9, column 5
jaspermayone commented 10 months ago

My config.yaml looks like this tho

---
providers:
  zones:
    class: octodns.provider.yaml.YamlProvider
    directory: ./zones
    enforce_order: false

  cloudflare:
    class: octodns_cloudflare.CloudflareProvider
    # email: 'env/CLOUDFLARE_EMAIL'
    token: env/CLOUDFLARE_TOKEN

zones:
  example.com.:
    sources:
      - zones
    targets:
      - cloudflare
      # - digitalocean
jaspermayone commented 10 months ago

I am following the guide at https://medium.com/devlan-io/octodns-github-actions-cloudflare-digital-ocean-redundant-multi-cloud-dns-hosting-for-free-134acdcc4f03

and am using venv.

I made a .env file and ran source .emv

jaspermayone commented 10 months ago

Not sure what still needs to happen at this point 🤷🏻‍♂️

ross commented 10 months ago

There may need to be an export before each of the variable lines in the .env file.

jaspermayone commented 10 months ago

That is working, but when I run octodns-dump --config-file config.yaml --output-dir=zones/ mydomain.com. cloudflare

I get 

2023-10-29T20:39:35  [8179962624] INFO  Manager __init__: config_file=config.yaml, (octoDNS 1.2.1)
2023-10-29T20:39:35  [8179962624] INFO  Manager _config_executor: max_workers=1
2023-10-29T20:39:35  [8179962624] INFO  Manager _config_include_meta: include_meta=False
2023-10-29T20:39:35  [8179962624] INFO  Manager _config_auto_arpa: auto_arpa=False
2023-10-29T20:39:35  [8179962624] INFO  Manager __init__: global_processors=[]
2023-10-29T20:39:35  [8179962624] INFO  Manager __init__: global_post_processors=[]
2023-10-29T20:39:35  [8179962624] INFO  Manager __init__: provider=zones (octodns.provider.yaml 1.2.1)
2023-10-29T20:39:35  [8179962624] INFO  Manager __init__: provider=cloudflare (octodns_cloudflare 0.0.3)
2023-10-29T20:39:35  [8179962624] INFO  Manager dump: zone=mydomain.com., output_dir=zones/, output_provider=None, lenient=False, split=False, sources=['cloudflare']
2023-10-29T20:39:35  [8179962624] INFO  Manager dump: using custom YamlProvider
Traceback (most recent call last):
  File "/opt/homebrew/bin/octodns-dump", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/octodns/cmds/dump.py", line 51, in main
    manager.dump(
  File "/opt/homebrew/lib/python3.11/site-packages/octodns/manager.py", line 879, in dump
    zone = self.get_zone(zone)
           ^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/octodns/manager.py", line 969, in get_zone
    raise ManagerException(f'Unknown zone name {idna_decode(zone_name)}')
octodns.manager.ManagerException: Unknown zone name mydomain.com.
jaspermayone commented 10 months ago

Am I running the command wrong?

jaspermayone commented 10 months ago

@ross Thank you so much for your help so far.

ross commented 10 months ago

You'll need to replace example.com.: with you donain in config yaml

jaspermayone commented 10 months ago

fixed, now I get this though.

octodns-dump --config-file config.yaml --output-dir=zones/ jaspermayone.com. cloudflare
2023-10-29T20:46:16  [8179962624] INFO  Manager __init__: config_file=config.yaml, (octoDNS 1.2.1)
2023-10-29T20:46:16  [8179962624] INFO  Manager _config_executor: max_workers=1
2023-10-29T20:46:16  [8179962624] INFO  Manager _config_include_meta: include_meta=False
2023-10-29T20:46:16  [8179962624] INFO  Manager _config_auto_arpa: auto_arpa=False
2023-10-29T20:46:16  [8179962624] INFO  Manager __init__: global_processors=[]
2023-10-29T20:46:16  [8179962624] INFO  Manager __init__: global_post_processors=[]
2023-10-29T20:46:16  [8179962624] INFO  Manager __init__: provider=zones (octodns.provider.yaml 1.2.1)
2023-10-29T20:46:16  [8179962624] INFO  Manager __init__: provider=cloudflare (octodns_cloudflare 0.0.3)
2023-10-29T20:46:16  [8179962624] INFO  Manager dump: zone=jaspermayone.com., output_dir=zones/, output_provider=None, lenient=False, split=False, sources=['cloudflare']
2023-10-29T20:46:16  [8179962624] INFO  Manager dump: using custom YamlProvider
Traceback (most recent call last):
  File "/opt/homebrew/bin/octodns-dump", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/octodns/cmds/dump.py", line 51, in main
    manager.dump(
  File "/opt/homebrew/lib/python3.11/site-packages/octodns/manager.py", line 881, in dump
    source.populate(zone, lenient=lenient)
  File "/opt/homebrew/lib/python3.11/site-packages/octodns_cloudflare/__init__.py", line 453, in populate
    records = self.zone_records(zone)
              ^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/octodns_cloudflare/__init__.py", line 389, in zone_records
    zone_id = self.zones.get(zone.name, False)
              ^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/octodns_cloudflare/__init__.py", line 175, in zones
    resp = self._try_request('GET', '/zones', params=params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/octodns_cloudflare/__init__.py", line 129, in _try_request
    return self._request(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/octodns_cloudflare/__init__.py", line 152, in _request
    raise CloudflareError(resp.json())
octodns_cloudflare.CloudflareError: Invalid request headers
jaspermayone commented 10 months ago

YAML file

---
providers:
  zones:
    class: octodns.provider.yaml.YamlProvider
    directory: ./zones
    enforce_order: false

  cloudflare:
    class: octodns_cloudflare.CloudflareProvider
    # email: 'env/CLOUDFLARE_EMAIL'
    token: env/CLOUDFLARE_TOKEN

zones:
  jaspermayone.com.:
    sources:
      - zones
    targets:
      - cloudflare
      # - digitalocean
ross commented 10 months ago

I believe that's the error that happens when the token/credentials are incorrect. I won't really be able to help much with sorting that out due to the nature of the problem.

jaspermayone commented 10 months ago

It is my global cloudflare token tho, so not sure how it possibly could be incorrect.

jaspermayone commented 10 months ago

I wasn't specifying email as well, specifying email resolved issues.

jaspermayone commented 10 months ago

Thanks for your help.