Bot parsing entries from issue template

[AdrienWehrle]

Now that @weiji14 implemented the issue template for a new database entry, let's get a bot to create the associated YAML file in the database!

Useful resources

https://stackoverflow.com/questions/58597010/how-to-access-a-github-issue-comment-body-using-github-actions

[AdrienWehrle]

It looks like we could turn the issue body into an environment variable that we could then pass to e.g. a short python script creating the YAML file at the right place and with the right file name

[weiji14]

To make it clearer what we're working with, maybe open a sample issue at https://github.com/APECS-Earth-Observation/Polar-EO-Database/issues/new?assignees=octocat&labels=enhancement%2Ctriage&template=new_database_entry.yml&title=%5BNew%5D%3A+ and see how the resulting issue looks like? Edit: see #36

Looking at https://docs.github.com/en/communities/using-templates-to-encourage-useful-issues-and-pull-requests/syntax-for-githubs-form-schema#keys, there's a unique id key that is meant to identify every element filled in the form (e.g. sensor-name, sensor-type, dataset-level, etc). I'm wondering if that would help to create key-value pairs.

[AdrienWehrle]

Maybe that's our ticket? https://github.com/marketplace/actions/issue-body-parser

[weiji14]

I think if we search a bit more, we'll find someone who's already done it. This one https://github.com/peter-murray/issue-forms-body-parser by the same author looks a bit closer to what we want. I also found https://github.com/imjohnbo/extract-issue-template-fields and a few others that look close. Just need to find the right keyword to search for the right thing!

[AdrienWehrle]

I'm trying out with that one :) https://github.com/edumserrano/github-issue-forms-parser

[weiji14]

This one too look promising - https://github.com/stefanbuck/github-issue-parser. They even have a blog post https://stefanbuck.com/blog/codeless-contributions-with-github-issue-forms walking through a few examples.

[AdrienWehrle]

I'm playing with some examples here: https://github.com/APECS-Earth-Observation/Polar-EO-Database/tree/parsing-action/.github/workflows

[AdrienWehrle]

I'm not sure I see how to trigger them though

[weiji14]

I'm not sure I see how to trigger them though

I think you'll need to merge that GitHub Action to the main branch first (which isn't ideal during experimentation) for it to be triggered by new issues. Maybe test it on your personal fork?

[AdrienWehrle]

Hmm, still doesn't run...

[weiji14]

Check your settings on https://github.com/AdrienWehrle/Polar-EO-Database, you might need to enable GitHub Actions first (usually it's disabled by default on forks). Then, open an issue on your personal fork to see if it triggers a run.

[AdrienWehrle]

They are enabled but no issue possible on a fork..!

[weiji14]

Enable 'Issues' in settings first at https://github.com/AdrienWehrle/Polar-EO-Database/settings

[AdrienWehrle]

Oh nice thanks, never thought of having Issues enabled on a fork! Testing can start :)

[AdrienWehrle]

Does work! https://github.com/AdrienWehrle/Polar-EO-Database/actions/runs/2863406491

[weiji14]

Excellent! So it looks like the https://github.com/stefanbuck/github-issue-parser one is working. Now we need to chain that parsed JSON output into something like https://github.com/peter-evans/create-pull-request. Maybe also setup a proper trigger (e.g. only run the action when a particular label like 'approved' is set) for security reasons. Great progress though, let's continue tomorrow!

[AdrienWehrle]

One more step :+1: https://github.com/AdrienWehrle/Polar-EO-Database/pull/3

[AdrienWehrle]

Remaining steps are:

• [x] set file name using satellite name
• [x] convert JSON to YAML

I'm happy to work on this :smile:

[weiji14]

Nice, I recommend https://github.com/mikefarah/yq (which has a GitHub Action) to both convert JSON to YAML (e.g. yq -P sample.json) and get the filename from the YAML directly (e.g. yq eval 'sensor-name' sample.yaml). Have personally been using it at https://github.com/weiji14/gmt-feedstock/blob/gmt-dev-builds/.github/workflows/bumpdevversion.yml#L47-L65 for some crazy automation :laughing:

[AdrienWehrle]

Only saw your comment now! I used a simple python script to convert, rename and git add the right file, see https://github.com/AdrienWehrle/Polar-EO-Database/pull/6/commits/f5acb8144ba8f365ab70e838e2dfb8a980ac0897 and associated workflow here. I'll prepare a PR shortly with some nice formatting for the PR etc..!

[weiji14]

Haha you're too fast! Python works too, just thought yq is nicer because it's literally a one-liner and one less Python script to maintain :slightly_smiling_face: Either way, this is looking really promising!

[AdrienWehrle]

I went for Python because we might want to do more stuff than just converting at some point? Don't know what exactly yet, but at least the option is there!