Table github_workflow failing when workflow content is commented

[gabrielsoltz]

Describe the bug

Hello team, when querying the table github_workflow, when there is an invalid YAML file or a YAML file that is all commented, like:

# name: Test

# on:
#   pull_request:
#     branches:
#       - master

# jobs:
# ....

The plugin fails with the error: Error: github: failed to populate column 'pipeline': EOF

The problem is that this error breaks the whole query. Is it possible to ignore the invalid YAML files so we can query the other workflows and maybe have some WARNING instead of an ERROR?

This is especially annoying when querying the github_workflow table using JOIN with another table, for example, github_my_repository. When this error occurs, the whole query breaks, instead of continuing with the next repository.

Debugging the failing repository is really tedious; you basically need to check repository by repository. The only way to fix this from the steampipe side is to exclude those repositories from the query. If an improvement like this is implemented, we could at least have the other workflows from the repo in the query.

I understand that I need to fix those invalid YAMLs, but Github allows you to push them, so in large implementations, it's possible to find these mistakes. In my case, from time to time, all my GitHub-related queries start failing because someone pushed an invalid YAML.

Thank you 🤟

Steampipe version (steampipe -v) Example: v0.23.2

Plugin version (steampipe plugin list) Example: 0.41.0

To reproduce

1. Add an invalid YAML file to a repository, or a file completely commented.
2. `select name, path, state, workflow_file_content from github_workflow where repository_full_name = 'repository-name'

Expected behavior A clear and concise description of what you expected to happen.

Additional context Add any other context about the problem here.

[ParthaI]

Hi @gabrielsoltz,

I've just submitted a PR to enhance the error messaging. Now, instead of disregarding errors altogether, the error message includes the HTML URL of the workflow file where the failure occurred. Here's how it looks now:

Error: github: failed to populate column 'pipeline': failed to parse the workflow file 'https://github.com/<Organization>/<Repository>/blob/main/.github/workflows/test.yaml'

In addition:

• We're planning to implement the ability to ignore errors based on either error message or error code.
• This approach seems applicable in the plugin rather than just for a specific table.
• However, there's currently no specific timeline for this enhancement.

Thank you!