Fix Help (Offline and Online)

[arcturus2]

Prerequisites

• [X] Put an X between the brackets on this line if you have done all of the following:
  • Running the latest version of Constellation
  • Attached the Support Package via Help > Support Package
  • Checked the FAQs: https://github.com/constellation-app/constellation/wiki/FAQ
  • Checked that your issue isn't already filed: https://github.com/constellation-app/constellation/issues
  • Checked that there is not already a module that provides the described functionality: https://github.com/constellation-app/constellation/wiki/Catalogue-of-Repositories

Description

The build in help feature in NetBeans has been broken after NetBeans has moved to Apache. The following sums up the reason taken from NETBEANS-656.

JavaHelp is not part of the donation of NetBeans to Apache, since JavaHelp is a separate project and not part of NetBeans itself. Plus, it is licensed in such a way that it can't be distributed at this time from NetBeans in Apache. So, yes, JavaHelp is not going to work, at the moment.

NetBeans 11.3 has removed the broken "Help Contents" menu and started redirecting users to the online documentation. Based on discussions with developers, we would like to consider following the same approach.

The requirements are as follows:

• Access to offline help - Given we still want Constellation to be used stand alone, having access to help without locally is still important.
• Ability to show help in context - the blue help buttons in the Data Access View should open the appropriate help page

If the way the help pages are built can be re-purposed for online help that would be ideal so that we can host them on GitHub pages etc.

Random thoughts:

• Following on from discussions, having to use a browser to show the help page isn't a bad design; many applications do this (even some git clients do it on Windows).
  • We were contemplating using the a web view but noting that it is a cut down version of what a browser can do so taking advantage of JavaScript and CSS is limited.
• If we can make this generic then we could make this available as a viable solution for others using NetBeans as platform. Perhaps we could donate this back to NetBeans they would find it useful?
• The current thinking is that there may be a step in our Ci (i.e Travis) to compile the help pages and that should be fine.

Fixing this will address the issue with #15

[arcturus2]

Made this a priority request because we need help to exist (should users want to see it)

[cygnus-x-1]

There's no reason you couldn't use both the WebView and an external browser - just make it a preference.

[formalhaut69]

Not sure if this is appropriate for this ticket but Bubble tree arrangement has no dialogue on it's help page.

[arcturus2]

Remember to close #15 when this closes

[github-actions[bot]]

This issue is stale because it has been open for 6 months with no activity. Consider reviewing and taking an action on this issue.

[GammaVel]

In Release 2.3 the new Welcome page has a Getting Started Section which links to https://constellation.readthedocs.io/en/latest/. This has version 2.0 in the Title. It could be a good idea to make this version agnostic and remove the 2.3 reference. Screenshot below:

[GammaVel]

Possibly relates to #1083

[arcturus2]

The motivation here is to keep in step with what NetBeans is doing. When NetBeans became an Apache project they were not able to keep the built in help provided by the "jhall" library because it wasn't officially donated to Apache. In the latest version of NB 12.4 the link to Help opens a browser window, removing the built in help support entirely.

Over an year ago @algol60 proposed an elegant solution using Spinx (https://www.sphinx-doc.org/en/master/) and writing a custom module to allow the ability to open help links in Constellation in a browser. This could be html files hosted in a local Jetty Server that runs in Constellation or the external documentation being https://constellation.readthedocs.io.

Sphinx is used by most popular Python libraries and uses an intermediate format called Restructured Text. This can be converted to html.

The original PR is https://github.com/constellation-app/constellation/pull/504 which puts what @algol60 did. One of the issues during the PR review was that the documentation files were in their own directory rather than in a module which means that if you turn off a module, the help pages still exist. Work was supposed to get underway to rectify this but kept on being pushed back.

When I picked up this task again I first started with a quick review to see what if anything had changed in terms of technologies. I concluded that readthedocs.io was still the place we should host the help pages but I noticed that they also supported Markdown. This would be a better fit in terms of what other Java projects use and MkDocs (https://www.mkdocs.org/) was a supported site generator we could use instead of Sphinx that readthedocs.io supported.

There is also ongoing updates happening to the existing help system so i decided to make the process I follow repeatable so its primarily written as a script.

The branch is called feature/online-offline-help and the script which does the work so far is https://github.com/constellation-app/constellation/blob/feature/online-offline-help/build_help.sh which converts the existing .html files to .md using a program called Pandoc (https://pandoc.org/). For example you can have a look at https://github.com/constellation-app/constellation/blob/feature/online-offline-help/CoreHistogramView/src/au/gov/asd/tac/constellation/views/histogram/docs/histogram-view.md which is the markdown version of the html version also in that folder.

The file mkdocs.yml located https://github.com/constellation-app/constellation/blob/feature/online-offline-help/mkdocs.yml is used to build the site and you can do so by running mkdocs serve and there should be an index page appearing. To setup MkDocs see https://www.mkdocs.org/getting-started/.

Running mkdocs serve allows for quicker previewing of changes but the idea at the end for offline help would be to locally serve the output from mkdocs build.

The things that need to be done are:

1. With a local copy of the server, how can we link a button action to open a specific help page?

The way this currently works is using a mapping file to link a unique string to a html file, see https://github.com/constellation-app/constellation/blob/feature/online-offline-help/CoreHistogramView/src/au/gov/asd/tac/constellation/views/histogram/docs/histogram-map.xml and the table of contents folder is determined by https://github.com/constellation-app/constellation/blob/feature/online-offline-help/CoreHistogramView/src/au/gov/asd/tac/constellation/views/histogram/docs/histogram-toc.xml.

The solution @algol60 came up with which is implemented in PR #504 is to have a mapping file https://github.com/constellation-app/constellation/blob/sphinx-help/docs/help_map.txt that is loaded at startup and used to serve the page via classes in https://github.com/constellation-app/constellation/tree/sphinx-help/CoreWebServer/src/au/gov/asd/tac/constellation/webserver/help that use Lookup to prioritise the handling of NetBeans help.

Each Restructured text file has a help-id comment which is what is used to build help_map.txt/ For more info see https://raw.githubusercontent.com/constellation-app/constellation/feature/online-offline-help/docs/source/views/histogram.rst. Potentially we could do the same with markdown instead?

One thing I have not tested is if any of the classes like SphinxHelpDisplayer still relies on jhall.

2. Do we just serve them in a browser or build a quick view to serve the help page?

For the first iteration we can just open a browser I think if that's simpler but just putting it out there in case a built in view would be easier to work with.

3. How do we overlay help pages from other module suites like constellation/constellation and constellation/constellation-adaptors?

Something also not properly tested in PR #504 which was another reason to not merge it was how it works with help files from different module suites that depend on Constellation - if they have the same filename does that replace the file? How is the table of contents updated if it is a different file?

4 Providing a built version for readthedocs.io

The readthedocs.io requires a folder in a repository that they can checkout, run a script and build the site it hosts. Given the online version will probably need to have help pages from constellation-app/constellation, constellation-app/constellation-adaptors and AustralianCyberSecurityCentre/constellation_cyber_plugins it makes it easier to have the help pre-built in a repository. The intention would be to use https://github.com/constellation-app/readthedocs for this purpose which we should be able to build using GitHub Actions and then tell readthedocs to simply checkout constellation-app/readthedocs.

Finally, one of my aspirations here is that I would like to be able to develop the online/offline help in a generic way so that we could share this with the developer community that have the same problem as a solution they can use for applications they have written that also use NetBeans as a platform (like we do).

I've recently asked a question on the NetBeans mail list about the future plans Apache NetBeans have with help and there isn't really anything in the pipeline. To this end, I would like to try and implement a solution which is a light touch on NetBeans platform code so that we don't stuck with too many issues in the future should NetBeans not have a library we need.

Happy to provide more information as needed.

[antares1470]

This all looks good in testing. Thanks again to @Delphinus8821 and @aldebaran30701 for your efforts on this