Doxter, A PureBasic Docs Generator

[tajmone]

Ciao @SicroAtGit,

In these days I've been busy working on a new CLI tool for the project: Doxter.

• Doxter.html — Documentation (Live HTML Preview)
• Doxter.pb

Doxter is a tool to generate AsciiDoc documentation from PureBasic source files, by using a special notations in comments delimiters to markup tagged regions of text and code that will be exported to an AsciiDoc source document where the various regions will be sorted according to their tag’s weight.

The new command line tool Doxter, will allow us to maintain documentation inside the source code, and then parse the source and produce an AsciiDoc document. For example, Doxter.pb contains its own documentation in the source, and Doxter extract this AsciiDoc file from it:

• Doxter.asciidoc

... which is then converted via Asciidoctor to HTML:

• Doxter.html

The auto-generated ADocs will be subdivided in tagged regions, therefore every document (including documents independent from source files) can import regions from the auto-generate doc of every module and source file, and when Doxter updates their doc files, every document in the project will be automatically updated.

This will grante a documentation that is always on par with the source code(s) it refers to.

Hopefully, this will be a major breakthrough in terms of maintanance of all the documentation of the tools and modules — which has lately become messy and unkempt, where the docs often refer to old and obsolete version of the modules code, effectively becoming a source of confusion instead of a helpful resource (especially now that I've been away from the project, I'm having trouble working out the current state of the code). This won't happen again with the new system.

Also, I've designed Doxter to be fully compatible with the current Header comments parser used by the tools in development: the two system can cohexist in the same document without interfering with each other.

Maybe in the future (not an immediate future, though) we can thing of integrating also this new tool in the project maintainance and allow, for example, some resources of the Archiv to have a self-contained documentation, which can then be automatically extracted and converted to HTML at every maintainance update operation.

Doxter has a small number of procedures (quite long ones though), but it could be quite easily made into a module, so that it could also be imported by other tools, and not just be a CLI tool. But lets leave that to later...

Anyhow, to conclude, Doxter works in the same spirit of the current tool under development:

• Exploit PB comment delimiter as parsing markers.
• Use a small set of markers in a creative way.
• Make sure that these markers and the formatting markup used in the commments look intuitive and human readable, without turning the source file into something hugly.
• Rely on a powerful external tool to handle the document formatting and conversion.

I'm looking forward for your feedback and opinion on Doxter!

[SicroAtGit]

The new command line tool Doxter, will allow us to maintain documentation inside the source code, and then parse the source and produce an AsciiDoc document.

Very good tool. For the tool logo you seem to have been inspired by the logo of the TV series "Dexter". I also think it's just the right way to have the documentation in the source code. It's easier to handle because you can edit both (code and documentation) directly and prevent the documentation from being out of date.

The auto-generated ADocs will be subdivided in tagged regions, therefore every document (including documents independent from source files) can import regions from the auto-generate doc of every module and source file, and when Doxter updates their doc files, every document in the project will be automatically updated.

Very good and useful feature. I think this is a feature that AsciiDoc has compared to Markdown. AsciiDoc is apparently much more functional than Markdown.

where the various regions will be sorted according to their tag’s weight.

The feature to influence the sorting of the contents for the documentation is a good idea.

Hopefully, this will be a major breakthrough in terms of maintanance of all the documentation of the tools and modules — which has lately become messy and unkempt, where the docs often refer to old and obsolete version of the modules code, effectively becoming a source of confusion instead of a helpful resource (especially now that I've been away from the project, I'm having trouble working out the current state of the code). This won't happen again with the new system.

Yes, so it will be better if code and documentation can be edited in the same file. So everything stays up to date.

Maybe in the future (not an immediate future, though) we can thing of integrating also this new tool in the project maintainance and allow, for example, some resources of the Archiv to have a self-contained documentation, which can then be automatically extracted and converted to HTML at every maintainance update operation.

Yes, we can do that. I once wrote you that I would like all include files from the archive to be documented as with the PB help. This will be a lot of work and I don't know yet if we should put so much work into the project. Maybe it would be better to wait with this step until there are more people helping with the archive. But as you've mentioned before, the archive management tools should be well documented.

The fact that the tool is standalone is also good, because the tool is certainly also very useful outside the CodeArchiv project.

I noticed that you created a separate repository for the Doxter tool: https://github.com/tajmone/doxter Will you manually keep the code up-to-date at both repositories or will you use "git subtree" or "git submodule"?

[tajmone]

I'm glad you like it. I've been investing some energy in this tool right now because is going to save considerable time in the nearby future. Also, I see it as an extension of the system created for the CodeArchiv tools.

I noticed that you created a separate repository for the Doxter tool: https://github.com/tajmone/doxter Will you manually keep the code up-to-date at both repositories or will you use "git subtree" or "git submodule"?

Yes, manually. I don't want to bring in all the extra files (examples, build scripts, etc.), especially since we'll end up using a different custom build of Highlight.js here. I'll just use a file synchronizer tool (either Viceversa Pro or Beyond Compare 4) which selectively copies files from one folder to another and CRC checks them.

In the long run, we might end up using just the Doxter API, as part of the tools. Today I've split Doxter's engine code into a module, which soon will be fully standalone from the CLI app.

But this is in regard to Doxter's presence in the assets folder, as a tool to build module's docs. As for the CodeArchiv, I'd love to eventually see a full copy of Doxter among the resources, so end users only need to keep up to date the Archiv and they'll also have an up to date Doxter version. The repo is needed for collaborative work, and the GitHub Pages website, the Wiki, etc.

For the CodeArchiv storage, we could use a module and subfolder it, as I'll be keeping the code along the CodeArchiv standars anyway. Once we get to that stage, probably I'll just be importing the files from that folder via relative paths.

So, the current solution is temporary anyway.