Understanding where `EvalError::NoSuchKey` comes from

[vlad-ivanov-name]

Follow-up to https://github.com/martinohmann/hcl-rs/issues/184.

Consider a terraform-like environment where local variables can be defined in locals blocks and where locals can reference each other. One way to resolve those references would be to evaluate values in a loop, every time topologically sorting the graph of references and evaluating more missing values.

Variables are referenced through an object local, e.g. local.variable_name. Right now if the variable is not defined in the evaluation context, the library will return the following error:

https://github.com/martinohmann/hcl-rs/blob/796707ab4ebfe6b83996aef4cce8d58ae2d51ced/crates/hcl-rs/src/eval/error.rs#L208-L209

The problem is, it's not clear what expression triggered the error. In the following code:

locals {
  var_combined = "${local.var_a}-${something.var_a}"
  var_a = "a"
}

the error will be something like:

        EvalError(
            "var_combined",
            Error {
                inner: ErrorInner {
                    kind: NoSuchKey(
                        "var_a",
                    ),
                    expr: Some(
                        TemplateExpr(
                            QuotedString(
                                "${local.var_a}-${something.var_a}",
                            ),
                        ),
                    ),
                },
            },
        )

with this error it seems to be impossible to tell whether the var_a key is missing from local object, or from some other object in code.

would it be possible to include the expression that produced the object itself in the error? or somehow else keep track of the source?

[vlad-ivanov-name]

Solved this in a slightly different way -- with hcl::edit visitors.

struct TraversalCollectorVisitor {
}

impl hcl::edit::visit::Visit for TraversalCollectorVisitor {
    fn visit_traversal(&mut self, node: &hcl::edit::expr::Traversal) {
        let identifier = if let hcl::edit::expr::Expression::Variable(variable) = &node.expr {
            let ident = variable.value();
            Some(ident.as_str())
        } else {
            None
        };

        let get_attr = if let Some(operator) = node.operators.first() {
            let operator = operator.value();

            if let hcl::edit::expr::TraversalOperator::GetAttr(ident) = operator {
                let ident = ident.value();
                Some(ident.as_str())
            } else {
                None
            }
        } else {
            None
        };

        eprintln!("identifier: {:?}", identifier);
        eprintln!("get_attr: {:?}", get_attr);
    }
}

given hcl::Expression expr:

let expr = hcl::edit::expr::Expression::from(expr);
let mut visitor = TraversalCollectorVisitor {};
visitor.visit_expr(&expr);

Leaving this issue open because this probably does not cover 100% of edge cases, like for example iterating over local in a for loop and then indexing the resulting expression. But then again one could simply forbid this in the language.

[martinohmann]

The problem you described surfaces because in hcl-rs the TemplateExpr type internally stores raw template strings which are only parsed during evaluation. If an error occurs while evaluating this freshly parsed template string, the expr reported in the error will be the TemplateExpr and not the expression within that string that caused the error.

This is an issue that is totally on me because I took a wrong design decision when initially creating hcl-rs. In hcl-edit such issue does not exist since there's no such thing as a TemplateExpr containing unparsed strings. But hcl-edit also does not have evaluation support.

I'd like to fix this in hcl-rs, but it'll be quite a bit work and might also break a lot of existing code. I tried it once, thinking it'll be a 20min adventure, but nope :joy:

Maybe at one point in the future I will have some time for this, or some other brave sailor will help with fixing this.