dbt-labs / dbt-core

dbt enables data analysts and engineers to transform their data using the same practices that software engineers use to build applications.
https://getdbt.com
Apache License 2.0
9.68k stars 1.61k forks source link

[Bug] "no loader for this environment specified" when running a simple macro #10289

Closed moseleyi closed 3 months ago

moseleyi commented 3 months ago

Is this a new bug in dbt-core?

Current Behavior

I execute a macro using dbt run-operation and receive an error

Expected Behavior

Macro is executed

Steps To Reproduce

  1. Add a simple macro and name the file generate_full_name.sql in /macros folder
    {% macro generate_full_name(first_name, last_name) %}
    {{ first_name }} || ' ' || {{ last_name }}
    {% endmacro %}
  2. Type dbt run-operation generate_full_name --args '{first_name: test, last_name: test2}'

Relevant log output

19:00:03  Running with dbt=1.8.0
19:00:04  Registered adapter: postgres=1.8.0
19:00:04  Unable to do partial parsing because saved manifest not found. Starting full parse.
19:00:04  Encountered an error:
no loader for this environment specified
19:00:04  Traceback (most recent call last):
  File "C:\Users\<user>\AppData\Local\Programs\Python\Python312\Lib\site-packages\dbt\cli\requires.py", line 138, in wrapper
    result, success = func(*args, **kwargs)
                      ^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\<user>\AppData\Local\Programs\Python\Python312\Lib\site-packages\dbt\cli\requires.py", line 101, in wrapper
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\<user>\AppData\Local\Programs\Python\Python312\Lib\site-packages\dbt\cli\requires.py", line 218, in wrapper
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\<user>\AppData\Local\Programs\Python\Python312\Lib\site-packages\dbt\cli\requires.py", line 247, in wrapper
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\<user>\AppData\Local\Programs\Python\Python312\Lib\site-packages\dbt\cli\requires.py", line 294, in wrapper
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\<user>\AppData\Local\Programs\Python\Python312\Lib\site-packages\dbt\cli\requires.py", line 320, in wrapper
    ctx.obj["manifest"] = parse_manifest(
                          ^^^^^^^^^^^^^^^
  File "C:\Users\<user>\AppData\Local\Programs\Python\Python312\Lib\site-packages\dbt\parser\manifest.py", line 1898, in parse_manifest
    manifest = ManifestLoader.get_full_manifest(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\<user>\AppData\Local\Programs\Python\Python312\Lib\site-packages\dbt\parser\manifest.py", line 330, in get_full_manifest
    manifest = loader.load()
               ^^^^^^^^^^^^^
  File "C:\Users\<user>\AppData\Local\Programs\Python\Python312\Lib\site-packages\dbt\parser\manifest.py", line 435, in load
    self.parse_project(
  File "C:\Users\<user>\AppData\Local\Programs\Python\Python312\Lib\site-packages\dbt\parser\manifest.py", line 733, in parse_project
    parser.parse_file(block)
  File "C:\Users\<user>\AppData\Local\Programs\Python\Python312\Lib\site-packages\dbt\parser\base.py", line 484, in parse_file
    self.parse_node(file_block)
  File "C:\Users\<user>\AppData\Local\Programs\Python\Python312\Lib\site-packages\dbt\parser\base.py", line 445, in parse_node
    self.render_update(node, config)
  File "C:\Users\<user>\AppData\Local\Programs\Python\Python312\Lib\site-packages\dbt\parser\models.py", line 348, in render_update
    super().render_update(node, config)
  File "C:\Users\<user>\AppData\Local\Programs\Python\Python312\Lib\site-packages\dbt\parser\base.py", line 421, in render_update
    context = self.render_with_context(node, config)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\dawid\AppData\Local\Programs\Python\Python312\Lib\site-packages\dbt\parser\base.py", line 270, in render_with_context
    get_rendered(parsed_node.raw_code, context, parsed_node, capture_macros=True)
  File "C:\Users\<user>\AppData\Local\Programs\Python\Python312\Lib\site-packages\dbt\clients\jinja.py", line 146, in get_rendered
    rendered = render_template(template, ctx, node)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\<user>\AppData\Local\Programs\Python\Python312\Lib\site-packages\dbt_common\clients\jinja.py", line 539, in render_template
    return template.render(ctx)
           ^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\<user>\AppData\Local\Programs\Python\Python312\Lib\site-packages\dbt_common\clients\jinja.py", line 144, in render
    return self.environment.handle_exception()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\<user>\AppData\Local\Programs\Python\Python312\Lib\site-packages\jinja2\environment.py", line 939, in handle_exception
    raise rewrite_traceback_stack(source=source)
  File "<template>", line 1, in top-level template code
TypeError: no loader for this environment specified

Environment

- OS: Windows 11
- Python: 3.12.3
- dbt-core: 1.8.0
- dbt-postgres: 1.8.0

Which database adapter are you using with dbt?

postgres

Additional Context

No response

dbeatty10 commented 3 months ago

Thanks for reporting this @moseleyi

I wonder if this could be caused by a combination of Python 3.12 and dbt in a Windows environment. Does it work if you use Python 3.11 instead?

When I tried this same code with Python 3.12 in macOS, it worked without an issue:

$ dbt run-operation generate_full_name --args '{first_name: test, last_name: test2}'                   
21:34:27  Running with dbt=1.8.2
21:34:28  Registered adapter: postgres=1.8.1
21:34:28  Unable to do partial parsing because saved manifest not found. Starting full parse.
21:34:29  Found 414 macros
moseleyi commented 3 months ago

Perhaps.. but last time I had multiple versions of Python, we had other issues with adapters, so would prefer not to install another version of python.. Wasn't 1.8.0 tested with Python 12?

Can you test it with 3.12?

dbeatty10 commented 3 months ago

@moseleyi Did you figure out what the issue was for you?

Wasn't 1.8.0 tested with Python 3.12?

Our current CI processes run unit and integration tests for the following Python versions:

We are using Ubuntu 22.04.4 LTS to run our unit tests across all of those Python versions.

We are using Ubuntu 20.04.6 LTS to run our integration tests across all of those Python versions.

Then for Python 3.8 only, we are also running our integration tests using macOS 12.7.5 and Microsoft Windows Server 2022 10.0.20348.