search

filiph / linkcheck

Fast link checker
https://pub.dartlang.org/packages/linkcheck
MIT License
397 stars 51 forks source link

`--connection-failures-as-warnings` flag does not work with GitHub Action #116

Open pepopowitz opened 1 year ago

pepopowitz commented 1 year ago

I have a GitHub workflow that is running linkcheck to crawl a public site, and it fails pretty regularly with connection failures.

I configured the workflow to use the --connection-failures-as-warnings, hoping that would suppress errors.

It appears to suppress the errors in terms of the statistics, but the workflow still fails when linkcheck runs into connection failures.

Useful details (hopefully?)

The workflow I've configured to call linkcheck with connection-failures-as-warnings

An example of the failing workflow

The output from the failing workflow: 

Crawling...

Provided URLs failing:
https://stage.docs.camunda.io/docs/1.3/components/zeebe/open-source/community-contributions/ (connection failed)

https://stage.docs.camunda.io/docs/0.2[6](https://github.com/camunda/camunda-platform-docs/actions/runs/4088441799/jobs/7084401079#step:6:7)/components/zeebe/open-source/community-contributions/
- (20:2195) '1.3' => https://stage.docs.camunda.io/docs/1.3/components/zeebe/open-source/community-contributions/ (connection failed)

https://stage.docs.camunda.io/docs/1.0/components/zeebe/open-source/community-contributions/
- (20:2193) '1.3' => https://stage.docs.camunda.io/docs/1.3/components/zeebe/open-source/community-contributions/ (connection failed)

https://stage.docs.camunda.io/docs/1.1/components/zeebe/open-source/community-contributions/
- (20:2193) '1.3' => https://stage.docs.camunda.io/docs/1.3/components/zeebe/open-source/community-contributions/ (connection failed)

https://stage.docs.camunda.io/docs/1.2/components/zeebe/open-source/community-contributions/
- (20:2193) '1.3' => https://stage.docs.camunda.io/docs/1.3/components/zeebe/open-source/community-contributions/ (connection failed)

https://stage.docs.camunda.io/docs/1.3/components/zeebe/open-source/get-help-get-involved/
- (20:6889) 'Open Sou..' => https://stage.docs.camunda.io/docs/1.3/components/zeebe/open-source/community-contributions/ (connection failed)
- (20:[7](https://github.com/camunda/camunda-platform-docs/actions/runs/4088441799/jobs/7084401079#step:6:8)27[8](https://github.com/camunda/camunda-platform-docs/actions/runs/4088441799/jobs/7084401079#step:6:9)) 'Communit..' => https://stage.docs.camunda.io/docs/1.3/components/zeebe/open-source/community-contributions/ (connection failed)
- (20:13336) 'Previous..' => https://stage.docs.camunda.io/docs/1.3/components/zeebe/open-source/community-contributions/ (connection failed)

https://stage.docs.camunda.io/docs/1.3/components/zeebe/technical-concepts/
- (20:8527) 'Open Sou..' => https://stage.docs.camunda.io/docs/1.3/components/zeebe/open-source/community-contributions/ (connection failed)

https://stage.docs.camunda.io/docs/1.3/components/zeebe/technical-concepts/architecture/
- (20:8718) 'Open Sou..' => https://stage.docs.camunda.io/docs/1.3/components/zeebe/open-source/community-contributions/ (connection failed)

https://stage.docs.camunda.io/docs/1.3/components/zeebe/technical-concepts/clustering/
- (20:8702) 'Open Sou..' => https://stage.docs.camunda.io/docs/1.3/components/zeebe/open-source/community-contributions/ (connection failed)

https://stage.docs.camunda.io/docs/1.3/components/zeebe/technical-concepts/exporters/
- (20:8577) 'Open Sou..' => https://stage.docs.camunda.io/docs/1.3/components/zeebe/open-source/community-contributions/ (connection failed)
- (54:1085) 'NextComm..' => https://stage.docs.camunda.io/docs/1.3/components/zeebe/open-source/community-contributions/ (connection failed)

https://stage.docs.camunda.io/docs/1.3/components/zeebe/technical-concepts/internal-processing/
- (20:8774) 'Open Sou..' => https://stage.docs.camunda.io/docs/1.3/components/zeebe/open-source/community-contributions/ (connection failed)

https://stage.docs.camunda.io/docs/1.3/components/zeebe/technical-concepts/partitions/
- (20:8702) 'Open Sou..' => https://stage.docs.camunda.io/docs/1.3/components/zeebe/open-source/community-contributions/ (connection failed)

https://stage.docs.camunda.io/docs/1.3/components/zeebe/technical-concepts/process-lifecycles/
- (20:8718) 'Open Sou..' => https://stage.docs.camunda.io/docs/1.3/components/zeebe/open-source/community-contributions/ (connection failed)

https://stage.docs.camunda.io/docs/1.3/components/zeebe/technical-concepts/protocols/
- (20:86[9](https://github.com/camunda/camunda-platform-docs/actions/runs/4088441799/jobs/7084401079#step:6:10)4) 'Open Sou..' => https://stage.docs.camunda.io/docs/1.3/components/zeebe/open-source/community-contributions/ (connection failed)

https://stage.docs.camunda.io/docs/1.3/components/zeebe/zeebe-overview/
- (20:6828) 'Open Sou..' => https://stage.docs.camunda.io/docs/1.3/components/zeebe/open-source/community-contributions/ (connection failed)
- (20:12529) 'communit..' => https://stage.docs.camunda.io/docs/1.3/components/zeebe/open-source/community-contributions/ (connection failed)

https://stage.docs.camunda.io/docs/8.0/components/zeebe/open-source/community-contributions/
- (20:2236) '1.3' => https://stage.docs.camunda.io/docs/1.3/components/zeebe/open-source/community-contributions/ (connection failed)

https://stage.docs.camunda.io/docs/components/zeebe/open-source/community-contributions/
- (20:2232) '1.3' => https://stage.docs.camunda.io/docs/1.3/components/zeebe/open-source/community-contributions/ (connection failed)

https://stage.docs.camunda.io/docs/next/components/zeebe/open-source/community-contributions/
- (20:2238) '1.3' => https://stage.docs.camunda.io/docs/1.3/components/zeebe/open-source/community-contributions/ (connection failed)

Stats:
  310397 links
    4285 destination URLs
    5403 URLs ignored
      21 warnings
       0 errors

And a screenshot of that because it looks like I can't see details of that workflow in an incognito window, so you probably can't either:

image
notnmeyer commented 8 months ago

ran into this as well. looks like linkcheck exiting 1 when there are warnings is the issue. only looked at this for a couple minutes but i dont think you can currently reconcile the GitHub action and ignoring warnings.

pretty sure the move is to use the binary or docker image directly and handle the exit code 1 scenario with some shell.

i think what's needed here is an option to exit 0 when there warnings but no errors.

edit: in my case, all the connection failures were for auto-generated static assets that i didnt need to check so i was able to use a skip-file pattern to work around the issue.