aduermael commented 7 years ago

Problem description

I have a pending PR ( #1052 ) to check all links to local resources. But now I need all broken links to be fixed in order to merge it. There are a lot, and in many cases, I personally don't know how to fix them.

Can we please divide it up and get this done as soon as possible?

Thank you!

html_test.go:55: /allpagelinks/index.html
    broken: <a href="404.md">
html_test.go:55: /apidocs/overview/index.html
    broken: <a href="apidocs/overview.md" class="button darkblue-btn nomunge" style="color:#FFFFFF; width:100%; margin: 0px;">

EDIT

1052 has been merged. But then changes for `1.13` have been merged, without running the tests. So we now have new broken links. I listed them in that spreadsheet

aduermael commented 7 years ago

cc @johndmulhausen

sanscontext commented 7 years ago

For the Docker Cloud API docs ones, if you navigate to the page it works:

https://docs.docker.com/apidocs/docker-cloud/#service goes to https://docs.docker.com/docker-cloud/apps/service-scaling/ which is a valid page

I think that what’s going on is the link checker is looking at files that are generated by the build, but not actually linked anywhere. I believe the make release build process there builds a “single page” layout, and then a series of one-pagers per topic, but AFAICT we don't actually use the single-pagers anywhere. They're broken because it generates them into a subfolder, but from the same source files as those that generate the working ones (and which don't trigger the link checker).

aduermael commented 7 years ago

Thanks @joaofnfernandes for your commit! I updated the list (~15% of it has been fixed)

aduermael commented 7 years ago

@sanscontext i don't understand your comment. Can you point me to specific lines that shouldn't appear?

aduermael commented 7 years ago

@sanscontext I fixed links in /apidocs/docker-cloud/includes: #1087 (waiting for CI to pass)

sanscontext commented 7 years ago

@aduermael Those files are programmatically generated. Did you fix them and see if running make release in that dir clobbers your changes?

sanscontext commented 7 years ago

@aduermael Please take a look at https://github.com/docker/docker.github.io/blob/master/apidocs/cloud-api-source/README.md for more background.

aduermael commented 7 years ago

@sanscontext oh no, I didn't... :( I'll try.

aduermael commented 7 years ago

@sanscontext But shouldn't that be triggered in the Dockerfile?

sanscontext commented 7 years ago

@aduermael These were migrated wholesale from the original cloud-docs repo (which I didn't set up, my understanding is that was migrated from an earlier tutum repo), and the docs change so infrequently it hasn't been worth it. (It's also nice to be able to run make release and preview the Slate generated docs before you commit.)

aduermael commented 7 years ago

@sanscontext Ok, so can I fix content directly in apidocs/cloud-api-source? Or do we keep pulling that from elsewhere?

sanscontext commented 7 years ago

(We had a long offline conversation about this, thanks for your patience. :) ) Yup, as long as you update it in the source, it'll build the fix each time.

aduermael commented 7 years ago

@sanscontext thanks for your patience too! :) And thanks for merging #1087. I rebased #1052 and updated the failing test logs in that issue, it's getting better!

sanscontext commented 7 years ago

I just pushed a PR for Hub and Cloud fixes. FWIW, what you're using as a validator isn't picking up our "redirects". (Which explains how we've had so many "broken" links that are still "working". 🤔 )

aduermael commented 7 years ago

@sanscontext That's intended. We need redirects for links on external websites not too be broken, but internally, it would be better be able to build a website that never (or almost never) needs them. Also, it's easier to test... Still I will add some tests later for all our redirects, to make sure none of them are broken, and that we don't have circular redirects. :)

sanscontext commented 7 years ago

Merged #1088, whittling away. I can probably take another crack at these with a regex tbh. Most of them are from reorg into the engine subfolder.

johndmulhausen commented 7 years ago

@sanscontext Thanks for whittling at this. @aduermael Does the original post have the latest broken links report, after the work of @sanscontext ?

aduermael commented 7 years ago

@johndmulhausen I just updated it, I was waiting for #1052 CI to run tests. :) Thanks @sanscontext!

aduermael commented 7 years ago

It got much smaller! (used to be 147 lines, now 51!) 💪

aduermael commented 7 years ago

@johndmulhausen about this one:

html_test.go:55: /allpagelinks/index.html
        broken: <a href="404.md">

Are you sure we don't want to link to /404.md instead? (that one exists)

sanscontext commented 7 years ago

Okay, I think we're down to the core engine issues, the 404 and some api docs (layouts?) stuff, and the question of where to point the 'datacenter' urls (do we make a latest?)

aduermael commented 7 years ago

@sanscontext @johndmulhausen @joaofnfernandes I updated the list.

aduermael commented 7 years ago

That one simply needs a / I guess:

html_test.go:55: /apidocs/overview/index.html
        broken: <a href="apidocs/overview.md" class="button darkblue-btn nomunge" style="color:#FFFFFF; width:100%; margin: 0px;">

aduermael commented 7 years ago

Hmmm... Actually it should link to github.com, that's the button to edit the page.

aduermael commented 7 years ago

@joaofnfernandes, @sanscontext, @johndmulhausen I rebased #1052 and updated the logs.

aduermael commented 7 years ago

@joaofnfernandes can you fix these ones please?

html_test.go:55: /docker-for-mac/index.html
        broken: <a href="/datacenter/dtr/2.1/index.md">
    html_test.go:55: /docker-for-windows/index.html
        broken: <a href="/datacenter/dtr/2.1/index.md">
html_test.go:55: /registry/index.html
        broken: <a href="/datacenter/dtr/2.1/index.md">

That page can't be found: https://docs.docker.com/sorry/#/datacenter/dtr/2.1/index/

sanscontext commented 7 years ago

Hm, yeah there's no index in that dir - we could drop them into guides though. (https://docs.docker.com/datacenter/dtr/2.1/guides/)

joaofnfernandes commented 7 years ago

I've just merged #1113 to fix those /datacenter paths.

aduermael commented 7 years ago

@johndmulhausen About this one:

html_test.go:55: /apidocs/overview/index.html
        broken: <a href="apidocs/overview.md" class="button darkblue-btn nomunge" style="color:#FFFFFF; width:100%; margin: 0px;">

It's the "Edit This Page" button (https://docs.docker.com/apidocs/overview/index.html).

The link should be overridden when using that template: _layouts/docs.html But for some reason, it doesn't work for /apidocs/overview.md. Do you have any clue?

aduermael commented 7 years ago

@johndmulhausen can we just remove 404.md from allpagelinks.md? It would be the easiest way to fix that issue...

aduermael commented 7 years ago

@joaofnfernandes there's still one broken link in /engine/userguide:

html_test.go:55: /engine/userguide/eng-image/image_management/index.html
        broken: <a href="/datacenter/dtr/2.1/index.md">

We don't pull that section from docker/docker it seems, so we can fix it ourselves.

Same comment for /engine/understanding-docker/index.html actually...

joaofnfernandes commented 7 years ago

1125 addresses those two

aduermael commented 7 years ago

@joaofnfernandes @johndmulhausen @sanscontext

YES! I updated the logs, only 2 issues remaining! 🙂

html_test.go:55: /allpagelinks/index.html
    broken: <a href="404.md">
html_test.go:55: /apidocs/overview/index.html
    broken: <a href="apidocs/overview.md" class="button darkblue-btn nomunge" style="color:#FFFFFF; width:100%; margin: 0px;">

I suggest we remove 404.md from allpagelinks.md.

@joaofnfernandes is working on a fix for the second one.

aduermael commented 7 years ago

@joaofnfernandes any progression on your fix for edit buttons?

joaofnfernandes commented 7 years ago

It seems the build for that PR failed while fetching from github. Will try to re-run the build.

aduermael commented 7 years ago

@joaofnfernandes Cool! I tested your changes, it seems to be working! :)

So only that one remains:

html_test.go:55: /allpagelinks/index.html
    broken: <a href="404.md">

I would like to simply remove it from allpagelink.md... (adding an exception because it's generated)

cc @johndmulhausen

aduermael commented 7 years ago

@joaofnfernandes thanks! just rebased #1052, I'll update the logs.

johndmulhausen commented 7 years ago

Tests still failing; @mstanleyjones @sanscontext @aduermael @joaofnfernandes @londoncalling

10:39:12 --- FAIL: TestURLs (4.75s)
10:39:12    html_test.go:55: /compose/compose-file/index.html
10:39:12        broken: <a href="/engine/userguide/dockervolumes.md">
10:39:12    html_test.go:55: /compose/django/index.html
10:39:12        broken: <a href="/engine/tutorials/dockerimages.md#building-an-image-from-a-dockerfile">
10:39:12    html_test.go:55: /compose/gettingstarted/index.html
10:39:12        broken: <a href="/engine/tutorials/dockerimages.md#building-an-image-from-a-dockerfile">
10:39:12    html_test.go:55: /compose/install/index.html
10:39:12        broken: <a href="/engine/installation/mac/" target="_blank">
10:39:12        broken: <a href="/engine/installation/windows/" target="_blank">
10:39:12    html_test.go:55: /compose/rails/index.html
10:39:12        broken: <a href="/engine/tutorials/dockerimages.md#building-an-image-from-a-dockerfile">
10:39:12    html_test.go:55: /docker-for-mac/faqs/index.html
10:39:12        broken: <a href="/engine/reference/api/docker_remote_api.md">
10:39:12    html_test.go:55: /docker-for-mac/index.html
10:39:12        broken: <a href="/engine/reference/index.md">
10:39:12    html_test.go:55: /docker-for-windows/faqs/index.html
10:39:12        broken: <a href="/engine/reference/api/">
10:39:12    html_test.go:55: /docker-for-windows/index.html
10:39:12        broken: <a href="/engine/reference/index.md">
10:39:12    html_test.go:55: /docker-hub/index.html
10:39:12        broken: <a href="/engine/tutorials/dockerimages/">
10:39:12    html_test.go:55: /docker-hub/repos/index.html
10:39:12        broken: <a href="/engine/tutorials/dockerimages/">
10:39:12    html_test.go:55: /docker-id/index.html
10:39:12        broken: <a href="../engine/reference/api/docker_io_accounts_api/">
10:39:12    html_test.go:55: /engine/api/getting-started/index.html
10:39:12        absolute: <a href="https://docs.docker.com/engine/installation/">
10:39:12    html_test.go:55: /engine/deprecated/index.html
10:39:12        absolute: <a href="https://docs.docker.com/engine/#feature-deprecation-policy">
10:39:12        absolute: <a href="https://docs.docker.com/registry/recipes/mirror/">
10:39:12    html_test.go:55: /engine/examples/index.html
10:39:12        broken: <a href="../tutorials/dockerizing.md">
10:39:12    html_test.go:55: /engine/extend/index.html
10:39:12        broken: <a href="../reference/commandline/index.md">
10:39:12    html_test.go:55: /engine/extend/plugins_graphdriver/index.html
10:39:12        broken: <a href="/docs/extend/index.md">
10:39:12    html_test.go:55: /engine/getstarted/last_page/index.html
10:39:12        absolute: <a href="https://docs.docker.com/docker-hub/">
10:39:12    html_test.go:55: /engine/getstarted-voting-app/node-setup/index.html
10:39:12        broken: <a href="/engine/reference/commandline.md">
10:39:12    html_test.go:55: /engine/getstarted-voting-app/test-drive/index.html
10:39:12        broken: <a href="index#docker-stackyml">
10:39:12    html_test.go:55: /engine/installation/binaries/index.html
10:39:12        broken: <a href="linux-postinstall.md">
10:39:12    html_test.go:55: /engine/installation/index.html
10:39:12        broken: <a href="linux/SUSE.md">
10:39:12    html_test.go:55: /engine/installation/linux/linux-postinstall/index.html
10:39:12        broken: <a href="../userguide/storagedriver/index.md">
10:39:12    html_test.go:55: /engine/reference/builder/index.html
10:39:12        broken: <a href="/engine/tutorials/dockerrepos/#/contributing-to-docker-hub">
10:39:12        broken: <a href="/engine/tutorials/dockerrepos/">
10:39:12    html_test.go:55: /engine/reference/commandline/docker/index.html
10:39:12        broken: <a href="checkpoint/">
10:39:12    html_test.go:55: /engine/reference/commandline/dockerd/index.html
10:39:12        broken: <a href="../api/docker_remote_api.md">
10:39:12    html_test.go:55: /engine/reference/commandline/image_pull/index.html
10:39:12        broken: <a href="/engine/reference/commandline/daemon/#insecure-registries">
10:39:12    html_test.go:55: /engine/swarm/admin_guide/index.html
10:39:12        broken: <a href="../reference/api/docker_remote_api_v1.24.md#36-nodes">
10:39:12    html_test.go:55: /engine/swarm/how-swarm-mode-works/nodes/index.html
10:39:12        broken: <a href="../../reference/api/index.md">
10:39:12    html_test.go:55: /engine/swarm/manage-nodes/index.html
10:39:12        broken: <a href="../reference/commandline/index.md">
10:39:12    html_test.go:55: /engine/swarm/networking/index.html
10:39:12        broken: <a href="../reference/commandline/index.md">
10:39:12    html_test.go:55: /engine/swarm/services/index.html
10:39:12        broken: <a href="security/trust/content_trust.md">
10:39:12        broken: <a href="../reference/commandline/index.md">
10:39:12    html_test.go:55: /engine/swarm/stack-deploy/index.html
10:39:12        broken: <a href="how-swarm-mode- works/services.md">
10:39:12    html_test.go:55: /engine/tutorials/dockervolumes/index.html
10:39:12        broken: <a href="usingdocker.md">
10:39:12        broken: <a href="dockerimages.md">
10:39:12        broken: <a href="dockerrepos.md">
10:39:12    html_test.go:55: /engine/tutorials/index.html
10:39:12        broken: <a href="dockerizing.md">
10:39:12        broken: <a href="usingdocker.md">
10:39:12        broken: <a href="dockerimages.md">
10:39:12        broken: <a href="dockerrepos.md">
10:39:12    html_test.go:55: /engine/userguide/index.html
10:39:12        broken: <a href="../tutorials/dockerizing.md">
10:39:12        broken: <a href="../tutorials/dockerimages.md">
10:39:12        broken: <a href="../tutorials/usingdocker.md">
10:39:12        broken: <a href="../tutorials/dockerrepos.md">
10:39:12    html_test.go:55: /engine/userguide/intro/index.html
10:39:12        broken: <a href="../tutorials/dockerizing.md">
10:39:12        broken: <a href="../tutorials/usingdocker.md">
10:39:12        broken: <a href="../tutorials/dockerimages.md">
10:39:12    html_test.go:55: /engine/userguide/networking/default_network/dockerlinks/index.html
10:39:12        broken: <a href="../../../tutorials/usingdocker.md">
10:39:12        broken: <a href="../../../tutorials/usingdocker.md">
10:39:12    html_test.go:55: /index.html
10:39:12        broken: <a href="engine/tutorials/dockerrepos.md">
10:39:12    html_test.go:55: /machine/get-started/index.html
10:39:12        broken: <a href="/engine/installation/mac/" target="_blank">
10:39:12        broken: <a href="/engine/installation/windows/" target="_blank">
10:39:12    html_test.go:55: /swarm/install-manual/index.html
10:39:12        broken: <a href="/engine/installation/cloud/cloud-ex-aws/">
10:39:12    html_test.go:55: /swarm/overview/index.html
10:39:12        broken: <a href="/engine/reference/api/docker_remote_api/">
10:39:12    html_test.go:55: /swarm/provision-with-machine/index.html
10:39:12        broken: <a href="/engine/installation/mac/">
10:39:12        broken: <a href="/engine/installation/mac/">
10:39:12    html_test.go:55: /swarm/swarm-api/index.html
10:39:12        broken: <a href="/engine/reference/api/docker_remote_api/">

^ This is blocking merges on a number of PRs. If this is because of out-of-date forks then perhaps the link checker should, for now, only be checking lines of delta?

aduermael commented 7 years ago

@johndmulhausen please merge this one and ask authors to rebase: #1302

We fixed all broken links a few days ago, but then commits for 1.13 got merged without testing.

I'm improving our tests, also fixing a few of those links. (see #1301) Hopefully that PR will be ready tomorrow or the day after.

mdlinville commented 7 years ago

I merged #1302. I'll rebase vnext-engine and vnext-compose to get it.

aduermael commented 7 years ago

Thank you @mstanleyjones!

johndmulhausen commented 7 years ago

@aduermael I completely support what you're doing with the CI tests and look forward to the day when we can make them mandatory. But for now, you should expect that future PRs will be merged without testing until we have a Jenkins system that doesn't block PRs for days because of implementation issues, such as incompatibility with existing forks, tests not starting for days, tests not finishing for days, etc. Obviously, given those issues, we had to make the call to proceed with the 1.13 release without Jenkins tests, but I'm sure if we all work the issue we can put an end to the bypass. Sorry we have to turn off the test (#1302) for now; thank you for all your hard work!

aduermael commented 7 years ago

@johndmulhausen I added timeouts: #1301 (15 minutes for checkout, 30 minutes for actual tests)

aduermael commented 7 years ago

@johndmulhausen, @sanscontext, @joaofnfernandes, @mstanleyjones:

I created a spreadsheet based on output from #1301. Redirects don't generate errors anymore, if we fix all the red ones, checks will be successful:

https://docs.google.com/spreadsheets/d/1y_svpzGHNkZmuQJjzrTzq7TC0a72S-iuj7NDxo3dQXc/edit?usp=sharing

aduermael commented 7 years ago

@johndmulhausen, @sanscontext, @joaofnfernandes, @mstanleyjones:

I updated the spreadsheet, I need your help now! :) You can let comments for the ones that can't easily be solved.

https://docs.google.com/spreadsheets/d/1y_svpzGHNkZmuQJjzrTzq7TC0a72S-iuj7NDxo3dQXc/edit#gid=15989068

docker-robott commented 1 year ago

Closed issues are locked after 30 days of inactivity. This helps our team focus on active issues.

If you have found a problem that seems similar to this, please open a new issue.

/lifecycle locked

docker / docs

Broken links #1068

Problem description

1052 has been merged. But then changes for `1.13` have been merged, without running the tests. So we now have new broken links. I listed them in that spreadsheet

1125 addresses those two

docker / docs

Broken links #1068

Problem description

1052 has been merged. But then changes for 1.13 have been merged, without running the tests. So we now have new broken links. I listed them in that spreadsheet

1125 addresses those two

1052 has been merged. But then changes for `1.13` have been merged, without running the tests. So we now have new broken links. I listed them in that spreadsheet