search

opencpn-manuals / plugins

Part of opencpn-manuals/main, not maintained as a separate manual
4 stars 3 forks source link

Many broken links. #6

Closed leamas closed 3 years ago

leamas commented 3 years ago

Linkchecker reveals no less than 96 broken links in the plugins manual. I ran linkchecker -r8 docs/index.htmlafter a local build.

Attaching the logfile: linkcheck.log.gz

leamas commented 3 years ago

I have added a "check links" step to the build. Example of output here. Note that the complete log is available as a link. Latest log is at http://ix.io/3vEa

Rasbats commented 3 years ago

Errrr ... thanks. Guess my work on the manuals was not checked properly. Will need some work but easily checked with VS Code. (Links in the pages)

leamas commented 3 years ago

There is also a w3c linkchecker. It's very slow, too slow for regular builds but finds more errors. I might try to find some way to use it optionally. Perhaps.

Rasbats commented 3 years ago

Each manual usually has a single page. This should be (relatively) easy to check.

rgleason commented 3 years ago

If these links are for screenshots, files and other content, we probably still have them in the user manual, and we can dig them out.

Rasbats commented 3 years ago

Yes ... I have a complete set of the files that are needed.

Rasbats commented 3 years ago

@leamas : Which linkchecker are you using locally?

leamas commented 3 years ago

Two. The first is 'linkchecker' from https://linkcheck.github.io/linkchecker/. This is a quick test which is done on all builds. You can see the results in the logs at for example https://github.com/opencpn-manuals/development/actions (i. e., under the Actions button).

The second is checklink from https://metacpan.org/release/W3C-LinkChecker. This is much slower, but finds more problems. Currently experimenting on the development manual. Here, you can add [full-linkcheck] to the commit message. The build will then run this; an example on results here.

leamas commented 3 years ago

Ah... local builds. I use the same there, both of them.

leamas commented 3 years ago

I have added new checks using the W3C-LinkChecker script. This is much more aggressive, and finds more errors. Some notes in INSTALL.md.

tl;dr Any commit with the string [full-linkcheck] in the commit message will run the new checks which takes about 20 minutes, See results under the Actions button,

Rasbats commented 3 years ago

[full-linkcheckj] should be [full-linkcheck] I think.

leamas commented 3 years ago

indeed "blushes"

Rasbats commented 3 years ago

Too many of my commits in the main branch.

Reread WORKFLOWS.adoc and now using testing branch.

Rasbats commented 3 years ago

@rgleason : Working on my many link errors. Would appreciate some help with checking the plugins where you host the published managed/non-managed versions (about 20). These are listed in master-plugins.txt and beta-plugins.txt.

My PR for chartscale_pi show the typical errors. Apart from missing images the rest are due to the use of link: instead of xref: for an intra document reference. VS Code does not allow a preview for these 'links', as it expects to find the link on the Internet website. However Firefox does show the xref link and is good for checking.

At this end I am starting with my published plugins.

Once we have done these intend updating sources.state and we can do another build of the manuals.

leamas commented 3 years ago

I have found a quick way to check for unresolved Antora links after making a local build, like this (assuming the html files lives in docs/):

$ grep -r  'class="page unresolved"' docs

It's fast and produces output like:

docs/AlternativeWorkflow/index.html:<p><a href="#Plugin-Dependencies.adoc" class="page unresolved">A Note on Plugin Dependencies</a></p>
docs/ocpn-dev-manual/5.3.1/pm-tp-language-nuance.html:<p><a href="#pi_installer_dev_setup.adoc" class="page unresolved">PI Manager Dev Setup</a>  FIXME: broken links</p>

The html filename is the same as the .adoc file besides the extension. I mostly use git grep to dig further. The links outputted are red if browsing the site with the browser.

leamas commented 3 years ago

None of the bad lilnks are visible when building the development manual. Thanks!

I guess the other errors should be fixed as well, but there is now no need from the development part i. e., it is possible to lower the priority.

Rasbats commented 3 years ago

A lot of bad links are due to "Back to top of page". The Contents sidebar takes care of this. Intend removing these intra document links unless anyone can see a reason for keeping them.

leamas commented 3 years ago

I recognize those, memories form when I made the first attempts to move from dokuwiki->Antora. I just removed them.

leamas commented 3 years ago

Now, something terrible has happened. Checking links in development manual (which rightfully now include plugins) I get the following:

Processing  file:///home/runner/work/development/development/docs/opencpn-plugins/0.1/index.html

This may take some time if the document has many links to check.

List of broken links and other issues:

file:///home/runner/work/development/development/docs/opencpn-plugins/0.1/index.html    
 Lines: 30, 32, 62, 67, 96, 99, 102, 111, 114, 117, 120, 123, 126, 129, 132, 135, 138, 141, 150, 153, 156, 159, 162, 171, 174, 177, 180, 183, 186, 189, 192, 201, 204, 207, 216, 219, 222, 225, 234, 237, 240, 243, 252, 255, 258, 261, 264, 267, 270, 273, 329, 361, 364, 395, 396
  Code: 200 (no message)
 To do: Some of the links to this resource point to broken URI fragments
    (such as index.html#fragment).
The following fragments need to be fixed:
    weather_routing:ROOT:weather_routing.adoc   Line: 243
    fugawi:ROOT:fugawi.adoc         Line: 123
    odraw:ROOT:odraw.adoc           Line: 201
    opencpn-master-plugins:twocan:twocan.adoc   Line: 270
    statusbar:ROOT:statusbar.adoc   Line: 252
    radar:ROOT:Home.adoc            Line: 99
    watchdog:ROOT:watchdog.adoc     Line: 204
    oesenc:ROOT:oesenc.adoc         Line: 120
    opencpn-master-plugins:squiddio:squiddio.adoc   Line: 171
    otcurrent:ROOT:otcurrent.adoc   Line: 186
    tactics:ROOT:tactics.adoc       Line: 216
    sar:ROOT:sar.adoc               Line: 207
    vdr:ROOT:vdr.adoc               Line: 159
    opencpn-master-plugins:wmm:wmm.adoc Line: 192
    debugger:ROOT:debugger.adoc     Line: 261
    googleearth:ROOT:googleearth.adoc   Line: 174
    s63_vector_charts:ROOT:s63_vector_charts.adoc   Line: 114
    find-it:ROOT:find-it.adoc       Line: 156
    odometer:ROOT:odometer.adoc     Line: 273
    rotationctrl:ROOT:rotationctrl.adoc Line: 132
    nv_charts:ROOT:nv_charts.adoc   Line: 111
    dead_reckoning:ROOT:dead_reckoning.adoc Line: 183
    projections:ROOT:projections.adoc   Line: 141
    bsb4_charts:ROOT:bsb4_charts.adoc   Line: 117
    nmea_converter:ROOT:nmea_converter.adoc Line: 162
    route_great_circle:ROOT:route_great_circle.adoc Line: 180
    celestial_navigation:ROOT:celestial_navigation.adoc Line: 177
    windvane:ROOT:windvane.adoc     Line: 225
    dash-t:ROOT:dash-t.adoc         Line: 150
    rtlsdr:ROOT:rtlsdr.adoc         Line: 102
    pypilot:ROOT:pypilot.adoc       Line: 264
    weatherfax:ROOT:weatherfax.adoc Line: 234
    launcher:ROOT:launcher.adoc     Line: 258
    calculator:ROOT:calculator.adoc Line: 255
    vfkaps:ROOT:vfkaps.adoc         Line: 129
    shipdriver:ROOT:shipdriver.adoc Line: 267
    climatology:ROOT:climatology.adoc   Line: 240
    iacfleet:ROOT:iacfleet.adoc     Line: 237
    polar:ROOT:polar.adoc           Line: 222
    photolayer:ROOT:photolayer.adoc Line: 126
    otidalplan:ROOT:otidalplan.adoc Line: 329
    chartscale:ROOT:chartscale.adoc Line: 135
    objsearch:ROOT:objsearch.adoc   Line: 138
    ais_radar_display:ROOT:ais_radar_display.adoc   Line: 96
    opencpn-master-plugins:ge2kap:ge2kap.adoc   Line: 189
    logbook:ROOT:logbook.adoc       Line: 153
    sweep_plot:ROOT:sweep_plot.adoc Line: 219
Anchors

Found 2 anchors.

This is not as it used to be (?) What's happening?

Rasbats commented 3 years ago

@Rasbats : Is back. Plugins manual after the last authoring commit looks sane. What commit is being used for the main manual?

./plugins https://github.com/opencpn-manuals/plugins 4fe583e771fb8f92520f678871bbed59cfe2eba7

This commit:

Microsoft Windows [Version 6.3.9600]
(c) 2013 Microsoft Corporation. All rights reserved.

C:\Users\Mike>cd C:\Projects\opencpn-manuals\plugins

C:\Projects\opencpn-manuals\plugins>git show 4fe583e771fb8f92520f678871bbed59cfe2eba7
commit 4fe583e771fb8f92520f678871bbed59cfe2eba7
Author: Rasbats <miker@members.swis.net>
Date:   Mon Aug 9 10:47:09 2021 +0100

    Renaming

    Rename plugins/plugins to plugins/manual. At the same time change manual and manual-beta to master-plugins and beta-plugins

diff --git a/.gitignore b/.gitignore
index 8b13789..d8f8d46 100644
--- a/.gitignore
+++ b/.gitignore
@@ -1 +1 @@
-
+docs
diff --git a/docs/.nojekyll b/docs/.nojekyll
deleted file mode 100644
index e69de29..0000000
diff --git a/docs/404.html b/docs/404.html
deleted file mode 100644
index 7ccab17..0000000
--- a/docs/404.html
+++ /dev/null
:

commit 4fe583e771fb8f92520f678871bbed59cfe2eba7
Author: Rasbats <miker@members.swis.net>
Date:   Mon Aug 9 10:47:09 2021 +0100

    Renaming

    Rename plugins/plugins to plugins/manual. At the same time change manual and manual-beta to master-plugins and beta-plugins

diff --git a/.gitignore b/.gitignore
index 8b13789..d8f8d46 100644
--- a/.gitignore
+++ b/.gitignore
@@ -1 +1 @@
-
+docs
diff --git a/docs/.nojekyll b/docs/.nojekyll
deleted file mode 100644
index e69de29..0000000
diff --git a/docs/404.html b/docs/404.html
deleted file mode 100644
index 7ccab17..0000000
--- a/docs/404.html
+++ /dev/null
@@ -1,520 +0,0 @@
:

Is this name change the problem?

leamas commented 3 years ago

The name change was a problem, but it was fixed long ago. I'll try to make local tests to see if it's about the plugin manual or how it's referenced in development. Stay tuned.

Rasbats commented 3 years ago

How about the indentation in the start paths for the plugins directories?

leamas commented 3 years ago

I have done a local check on the output from plugins/site.yml, attached. It doesn't look good (:

checklink.log.gz

leamas commented 3 years ago

need some sleep...

leamas commented 3 years ago

Hm... looking at this excerpt from log file:

List of broken links and other issues:

file:///home/mk/src/opencpn-manuals/plugins/docs/opencpn-plugins/0.1/authoring/TODO.html    
 Lines: 7, 31, 33, 171, 189, 192, 270
  Code: 200 (no message)
 To do: Some of the links to this resource point to broken URI fragments
    (such as index.html#fragment).
The following fragments need to be fixed:
    opencpn-master-plugins:twocan:twocan.adoc   Line: 270
    opencpn-master-plugins:squiddio:squiddio.adoc   Line: 171
    opencpn-master-plugins:ge2kap:ge2kap.adoc   Line: 189
    opencpn-master-plugins:wmm:wmm.adoc Line: 192

The other modules are referenced as for example opencpn-master-plugins:twocan:twocan.adoc . However, when I look into manual/master-plugin/antora.yml I see

name: opencpn-plugins
title: OpenCPN Plugins
version: '0.1'
nav:

That is, the reference above should be changed like opencpn-master-plugins:twocan:twocan.adoc -> opencpn-plugins:twocan:twocan.adoc

There are other options, I guess. One could update antora.yml to match opencpn-master-plugins. But I wonder if not the best might be to use the default value here i. e., opencpn-master-plugins:twocan:twocan.adoc -> twocan:twocan.adoc. This way it would work without changes when used in the beta-plugins context.

Thoughts?

leamas commented 3 years ago

Another problem seems to be about missing files. For example, there are references to launcher:launcher.adoc:

 git grep launcher.adoc
dokuwiki plugins.txt:xref:launcher:launcher.adoc[launcher] +
master-plugins/modules/ROOT/pages/plugins.adoc:* xref:launcher:ROOT:launcher.adoc[launcher #]  
master-plugins/modules/ROOT/partials/nav-plugins.adoc:** xref:launcher:ROOT:launcher.adoc[launcher]

But I cannot find any module launcher and no file launcher.adoc:

$ find . -name launcher.adoc
$

Have you some files locally that are not checked in to git?

leamas commented 3 years ago

... or have I missed to update the sources? Stay tuned..

Rasbats commented 3 years ago

site.yml has a bad page-resource url entry. This is resulting in many of the broken links. Fixing (I hope).

leamas commented 3 years ago

I have pushed a simple fix to nav-plugins.adoc, it fixes just a few errors. You need to rebase.

oplaydo commented 3 years ago

Have you removed the settings option for this repo? It was used as an easy route to the website!!!

Rasbats commented 3 years ago

Thanks for picking up the mistakes in nav-plugins.adoc. Now checking the log for the site.yml fix.

Rasbats commented 3 years ago

Settings is back.

leamas commented 3 years ago

Have you removed the settings option for this repo? It was used as an easy route to the website!!!

Not that I'm aware of.... or did I?

Settings is back.

OK. Time for a new build and check?

leamas commented 3 years ago

If you are talking about the settings at the very bottom of site.yml they are only used to populate the ui links under Resources (unless you have been using them in the adoc files)

Rasbats commented 3 years ago

Red herring. Now checking a new build.

leamas commented 3 years ago

I pushed a typo fix (possibly left-over). Nothing important.

Rasbats commented 3 years ago

Thanks. Where in the build process does sources.state get used? Intending a source-state.sh update to check some of the dev manuals PR that have been accepted.

Rasbats commented 3 years ago

Here?

# Update dependencies and build site
if [ -f source-state.sh ]; then ./source-state.sh restore; fi
antora $PLAYBOOK
touch $SITE_DIR/.nojekyll
leamas commented 3 years ago

Yup

Rasbats commented 3 years ago

So if I just do source-state.sh update everything will be taken care of. Is that correct?

leamas commented 3 years ago

I wonder if the best way could be to remove all plugins, check that everything is fine and the add them in small batches, fixing errors before adding more?

leamas commented 3 years ago

So if I just do source-state.sh update everything will be taken care of. Is that correct?

Basically, you need to check in sources.state.

Rasbats commented 3 years ago

I wonder if the best way could be to remove all plugins, check that everything is fine and the add them in small batches, fixing errors before adding more?

The problem is that the PR for devs sometimes takes a long time to action. I have a list of all the plugins and I can test before the PR.

I have a plugins manual that works, although it has broken links. Notice in the developer manual that when we go to plugins manual from development the home page (house symbol) always links to the plugins manual.

Is this a version issue? I have been using v0.1. Develoment uses v5.3.1.

leamas commented 3 years ago

I have a plugins manual that works, although it has broken links.

Yes, and those links are visible also in the developer manual. But yes, also from the developer manual things "works", sort of.

leamas commented 3 years ago

Is this a version issue? I have been using v0.1. Develoment uses v5.3.1.

Not a problem, the overall linking is fine: https://opencpn-manuals.github.io/development

leamas commented 3 years ago

The problem is that the PR for devs sometimes takes a long time to action. I have a list of all the plugins and I can test before the PR.

Yes, I can understand that. But if we want to came to something stable we need to fix the broken links in "your " repos, and then create PRs from those.

Question is if we are at a point where we could start working with the plugins one-by-one. Are we?

leamas commented 3 years ago

So if I just do source-state.sh update everything will be taken care of. Is that correct?

Basically, you need to check in sources.state.

... and to do that sources.state must be updated using ./source-state.sh save. I have a test build running after updating sources.state this way. Shall I push it to main?

Rasbats commented 3 years ago

Shall I push it to main?

Yes.

Question is if we are at a point where we could start working with the plugins one-by-one. Are we?

How about me pruning site.yml, just leaving the plugins I manage. A recent basic linkcheck showed no errors in ShipDriver manual for instance but the full-linkcheck did. With a pruned sources.state and the trimmed site.yml I can move forward later with the PR for further plugins.

leamas commented 3 years ago

OK, sounds good to me.

I just pushed main.

Another thing you perhaps could do is file bugs for say six or nine most important plugins which needs to be fixed. An then assign 2(3) to me, 2(3) to Rick etc. But then we need @rgleason to be in the loop.

Thoughts?

Rasbats commented 3 years ago

Let me see what happens with my plugins. That would take care of 13/54! Still suspicious of the w3c full linkcheck. EDIT: I already have forks for the others in my oplaydo account.