guyarb / golang-test-annotations

A github action which annotates failed tests.
MIT License
34 stars 10 forks source link

can you add a link to showing this #4

Closed alexec closed 3 years ago

guyarb commented 3 years ago

hey @alexec! Can you elaborate on your request? I didn't understand which link you are talking about, or where to add it

alexec commented 3 years ago

I'd like to see a build that is using this feature

guyarb commented 3 years ago

Updated the readme, now it contains a link to an example repo.

alexec commented 3 years ago

I get a 404 error:

https://github.com/guyarb/golang-test-annotations-example/actions/runs/505258482

alexec commented 3 years ago

Any chance you could take a look at my set-up:

https://github.com/argoproj/argo/pull/4925

I didn't seem to get errors reported.

guyarb commented 3 years ago

@alexec , i opened the example repo to be public :)

and it looks like the output of your UI tests are not in the format that go test -json ./... > test.json

can you post the test.json file from your CI?

alexec commented 3 years ago

maybe it is gotestsum --jsonfile=test.json -- not working - I will investigate

guyarb commented 3 years ago

@alexec do you have any conclusions?

alexec commented 3 years ago

sorry - I gave up :(

guyarb commented 3 years ago

can you switch the test command line to go test -json ./... > test.json?

displague commented 3 years ago

@guyarb similar experience with https://github.com/equinix/terraform-provider-metal/pull/173/checks?check_run_id=3246845567

image

The errors were not annotated to the PR view. I'm not sure if this would have to be merged first or if the PR introducing golang-test-annotations should have been annotated.

https://github.com/equinix/terraform-provider-metal/pull/173/files

guyarb commented 3 years ago

@displague , I do see the errors were annotated https://github.com/equinix/terraform-provider-metal/actions/runs/1099517484 image image

did i misunderstand you?

displague commented 3 years ago

@guyarb I see what you mean. I was expecting inline PR comments on failed tests, like https://github.com/reviewdog/reviewdog.

image

Is the problem that the test file/function/lines were not modified in the PR? If they were, perhaps the annotation would appear on that function?

displague commented 3 years ago

Here's a better example of what I was expecting:

https://github.com/WillAbides/go-test2action/pull/1/files

https://github.com/WillAbides/go-test2action/blob/main/internal/internal.go https://github.com/WillAbides/ezactions/blob/master/commander.go#L106 (looks like you can specify col= too)

displague commented 3 years ago

Perhaps https://github.com/isaacs/github/issues/284#issuecomment-860297388 is related and actions have no way to comment on lines (tests) that are not included in the PR.

guyarb commented 3 years ago

@displague, indeed when the PR added a failed test it adds preview on the diff (for https://github.com/guyarb/golang-test-annotations-example/pull/1/files)