Extension.Contents + Expression.Evaluate: Report .pqm file name with parse/syntax errors

[bgribaudo]

Preflight Checklist

• [X] I have installed the latest version of Power Query SDK.
• [X] I have checked existing resources, including the common issues and the release notes.
• [X] I have searched for similar issues.

Problem

(This relates to a Microsoft-recommended PQ pattern that is unique to connectors. I hope this is the correct place for this request.)

Extension.Contents + Expression.Evaluate are commonly used in connectors to load M code from .pqm files. When this is done, if a .pqm file contains a syntax problem, Expression.Evaluate will raise an appropriate error. However, this error does not indicate which .pqm file in the extension contains the error.

Desired Solution

To help with debugging, especially when a connector includes multiple .pqm files, could errors raised by Expression.Evaluate include the .pqm file’s name?

Alternatives and Workarounds

No response

Additional Context

Implementation idea:

• Modify Expression.Evaluate to check its first argument (document—the source code argument) for metadata field Origin. If found, include its value as a Detail on any errors that may be raised by/from Expression.Evaluate.
• Modify the "include .pqm file" pattern template to set this metadata field—something like:

  Extension.LoadFunction = (name as text) =>
  let
      binary = Extension.Contents(name),
      asText = Text.FromBinary(binary)
  +                     meta [Origin = [Type = "EmbeddedFunction", FileName = name]]
  in
      Expression.Evaluate(asText, #shared);

[mattmasson]

Changing Expression.Evaluate is out of scope at this time.

Loading module files in this way was really meant to be temporary... ideally we'd have an actual module referencing system that allows proper reuse and versioning... but I don't see that being funded in the near future.

How about changing the definition of Extension.LoadFunction to something like:

Extension.LoadFunction = (name as text) =>
  let
      binary = Extension.Contents(name),
      asText = Text.FromBinary(binary)
  in
      try
        Expression.Evaluate(asText, #shared)
      catch (e) =>
        error Error.Record("Expression.Error", Text.Format("[#{0}] #{1}", {name, e[Message]}), e);

[7:23:11 PM]    [Info]  RunTestBattery result [
    {
        "ActivityId": "9ba74e60-ccfd-4881-a438-86cb26491ab9",
        "DataSourceAnalysis": [
            {
                "Kind": "DataSource",
                "FunctionName": "TestTestTest.Contents",
                "DataSource": {
                    "Kind": "TestTestTest",
                    "NormalizedPath": "TestTestTest",
                    "Path": "TestTestTest"
                }
            }
        ],
        "Details": "[TestModule.pqm] [1,9-1,10] The type identifier is invalid.",
        "EndTime": "2022-11-03T23:23:11.422378+00:00",
        "Method": "PQTest.RunTest",
        "Name": "TestTestTest.query.pq",
        "StartTime": "2022-11-03T23:23:09.2928229+00:00",
        "RowCount": 0,
        "Status": "Failed",
        "Type": "PQTest.Expression",
        "Error": {
            "Message": "[TestModule.pqm] [1,9-1,10] The type identifier is invalid.",
            "Details": {
                "Reason": "Expression.Error",
                "Message": "[1,9-1,10] The type identifier is invalid.",
                "Detail": "{[...]}",
                "Message.Format": "[1,9-1,10] The type identifier is invalid."
            }
        }
    }
]

[bgribaudo]

That's a great idea. Thanks, @mattmasson!

I've posted a PR to do something like this to the base Extension.LoadFunction template -- https://github.com/MicrosoftDocs/powerquery-docs/pull/328.