search

typhonjs-node-tjsdoc / tjsdoc

Provides common orchestration for TJSDoc.
Mozilla Public License 2.0
11 stars 2 forks source link

Ongoing refactoring #2

Open typhonrt opened 7 years ago

typhonrt commented 7 years ago

This issue tracks the progress of the initial ongoing refactoring occurring with TJSDoc.

Just dropping a notice for those w/ early access. I suppose watch this repo to get updates on progress. @dandv, @raveclassic, @ratson, @skurger, @connor4312

ratson commented 7 years ago

Looking forward to see the code~

typhonrt commented 7 years ago

It's coming. I hope to have things wrapped up with the initial effort committed by Wednesday or earlier. There have been a lot of changes that have benefited from doing this work offline. Everything has been examined and hardened.

typhonrt commented 7 years ago

@ratson, well, you know software.. right? Ah, it's going to be a few more days as a few great features recently got added to the TODO list that I finished yesterday. I don't expect any more items to be added, so I'm working down the final list which is now mostly getting to an 100% documented code base and other housekeeping like alphabetically sorting all methods in files, etc. So a few more days, maybe through the weekend, but likely not longer than that.. Gotta 💯% this.. ;)

typhonrt commented 7 years ago

OK... So I decided to spend a few more days to finalize what I think is a solid initial public release of tjsdoc. I'll switch the repo from private to public on initial commit and publish to NPM, so that everyone can start using this effort.

Some of the cool features I just finished (in addition to dozens more I haven't mentioned yet):

There are many more enhancements as well. My remaining TODO list is to separate out the single page output to module scope overview for functions / typedefs, convert and / or fold in my previous ESDoc plugins and glob matching for sources / multi root repos. Switch to material-components-web for mobile responsive template usage + useful components with zepto/velocity integration. Then finish up documentation and create a website for TJSDoc. So a little more to do, but if I keep up the hustle things should launch next week or sooner. A bit more of a delay, but I think launching publicly out of the gate is a good thing and all of the above definitely will separate TJSDoc from ESDoc in terms of look and feel.

typhonrt commented 7 years ago

Ugh... It keeps going on and on... ;P You know that final 10% that takes longer than the 1st 90.... I'll give a little update on current progress at the end, but do have some musings on possibly creating 4 main repo / NPM modules for release. 1 common runtime, 1 common publisher, 1 babylon, 1 typescript.... I have everything separated into a common runtime and babylon runtime which extends things that is all plugin driven such that you set "config.runtime": "tjsdoc-babylon" in the config and the babylon runtime is loaded and chugs along. For typescript, yep, tjsdoc-typescript. Everything is handled with custom config overrides for default file extensions along with anything else pertinent for the particular runtime.

One of the cool things is that the common runtime tjsdoc could still be installed separately, but you must also include the runtime of choice and set the config parameter. Where I am thinking the 3 separate runtime repos come into play is being able to release tjsdoc-babylon as a CLI enabled module which includes the runtime pre-configured. A simple Object assign would do it. 

import TJSDocCLI     from tjsdoc/src/TJSDoc.js
import TJSDocBabylon from './TJSDocBabylon.js'

/** 
 * Provides an overidden TJSDoc CLI implementation setting the runtime to TJSDocBabylon.
 * @inheritdoc
 */
export default class TJSDocBabylonCLI extends TJSDocCLI
{
   /* @override @inheritdoc */
   exec(runtime) { super(TJSDocBabylon); }
}

That's it.. TJSDocBabylon has a simple override for the generate method:

import TJSDoc from 'tjsdoc';

/**
 * Provides an overridden version of TJSDoc with a default runtime assigned to `tjsdoc-babylon`.
 * @inheritdoc
 */
export default class TJSDocBabylon
{
   /* @inheritdoc */   
   static generate(config) { TJSDoc.generate(Object.assign({ runtime: 'tjsdoc-babylon' }, config)); }
}

Likewise something as simple as that for the typescript version. Having separate CLI runtimes for vanilla tjsdoc where the user manages the runtime and even plugs in a complete 3rd party one if so desired or the two specific CLI / modules both could be installed even globally at the same time and used independently.

This brings up testing... There will be a separate GitHub repo that holds all the tests shared between the runtimes. The runtimes only having to produce the DocDB and the plugin system provides ample opportunity to mock and load the minimum runtime which can be data driven (tests work for babylon & typescript transparently). The static HTML publisher is also a separate module with all of the HTML tests accompanying it.

The alternative and lazy way is to just run everything in the same repo and assume technical debt for a quick / lazy win. I think the above strategy is good to get to before public release as any short term ramp up speed for other developers will be worth it. I think it will more than make up for things being a more user-friendly experience as well for not that much more work which also benefits from doing this out of the gate; IE no downstream change to multiple repos / user confusion.

Any thoughts? I'm still reasonably confident for a launch before 2 weeks, but maybe 10 days. Hard to say. If all goes well from here that'd be great. I'd really like to get this done before GDC for a public release.

Other cool stuff... I did finally get the config resolution / extension / validation loading perfect and into a separate repo. It works hand in hand with the plugin manager with custom resolution for plugin deduplication in all extension scenarios and has extensive pre / post validation steps along with setting any missing default values between pre / post. It also is a plugin and while a class is the default export it is consumed in TJSDoc as purely a plugin.

This gives some insight into how the rest of TJSDoc is structured. Everything is event based over a shared eventbus which is injected via the plugin system. There are ~85 specific event bindings when completely configured and there are no dependencies at all between even reasonably fine grained plugin breakdown. The plugin system actually sends out a proxy so that all event binding registrations made from the plugin are transparently managed. Unload a plugin and automatically all event bindings are unregistered without the plugin authors intervention.

Here are the two points that registers new ConfigResolver plugin instances. First internally in an instance L296-L326 and then there is a module scoped plugin method L467-L477.

A bit of a mouthful I suppose especially the snippet below... I'll probably just quit it after the next code block.. 8D

In typhonjs-babylon which just exports a module scoped onPluginLoad method along with the default class with the runtime preset thus loads all the Babylon specific plugins including kicking off the common runtime. Here that is in entirety:

import path       from 'path';
import TJSDoc     from 'tjsdoc';
import ConfigData from 'tjsdoc/src/ConfigData';

/**
 * Provides an overridden version of TJSDoc with a default runtime assigned to `tjsdoc-babylon`.
 * @inheritdoc
 */
export default class TJSDocBabylon extends TJSDoc
{
   /* @inheritdoc */   
   static generate(config) { TJSDoc.generate(Object.assign({ runtime: 'tjsdoc-babylon' }, config)); }
}

/**
 * Adds all Babylon runtime plugins.
 * @param {PluginEvent} ev - The plugin event.
 */
export function onPluginLoad(ev)
{
   const eventbus = ev.eventbus;
   const dirPath = path.resolve(__dirname);

   eventbus.trigger('plugins:add', { name: 'tjsdoc' });
   eventbus.trigger('plugins:add', { name: `${dirPath}/doc/` });
   eventbus.trigger('plugins:add', { name: `${dirPath}/parser/` });

   // Adds an instance of typhonjs-config-resolver with overridden common TJSDoc config resolver data for usage with
   // Babylon / JS source code.
   eventbus.trigger('plugins:add', { name: 'tjsdoc-config-resolver', target: 'typhonjs-config-resolver', options:
   {
      eventPrepend: 'tjsdoc',
      resolverData: ConfigData.createResolverData(eventbus,
      {
         defaultValues:
         {
            'includes': ['\\.(js|jsx|jsm)$'],
            'pathExtensions': ['.js', '.jsx', '.jsm'],
            'test.includes': ['\\.(js|jsx|jsm)$']
         }
      })
   } });
}
typhonrt commented 7 years ago

So I almost made the GDC "artificial goal" of getting TJSDoc fully shipped, but one module remains to be finished; that is the static HTML publishing module. I've certainly got my upgraded version of the ESDoc version and template working, but I need to get things shifted over to the new material-components-web based publisher in addition to adding the ability for plugins to add menu items with static html destinations, roll in of my enhanced left hand navigation, and new addition of a file / module overview output for functions / typedefs, etc.. I certainly plan to get TJSDoc launched next week, so things will continue to progress, but the new publisher might take a few days. There is some exciting news to follow the module overview...

So... Things ended up with the following modules for the main ES6+ / Babylon distribution:

missing / not checked in currently (pending update):

with test and test utils for all modules with direct github dev dependencies for:

The exciting news is that I came up with a good way to work locally for development using Babel environment options for .babelrc, the dev-<x> NPM scripts, and a Babel plugin (babel-plugin-module-resolver). If all of the repos are checked out locally one can run TJSDoc against the local modules due to the module resolver including full test support across all modules with the dev-<x> NPM scripts. This allows development to occur in a reasonably transparent process across all modules locally.

Though at this time checking out the modules manually is the general situation I'll be creating in the next month or so a tjsdoc-dev CLI which will automatically checkout or fork depending on desired setup a developer environment including WebStorm project files.

The main runtime specific parts of the code for Babylon which need to have Typescript implementations are the following:

I'll certainly be creating an architecture document to describe how everything fits together.

typhonrt commented 7 years ago

So the rewrite of the static HTML publisher with material-components-web works well. I still have a bit to finish up before publishing this module, but an interesting issue was posted for file watching / incremental doc regeneration. I have taken this on as it really allowed the opportunity to make sure the TJSDoc runtime is robust and fine tuned. Of course it's tricky and took a while, but tjsdoc-plugin-watcher is complete and well tested including various stress tests.

Now tjsdoc-plugin-watcher doesn't actually do the regeneration of docs, but sets up file watching on the source and test files in the onComplete plugin callback altering the control flow of TJSDoc. It keeps TJSDoc alive and posts events which any other plugin can pick up when changes occur. There is a terminal option for user input and options for headless / silent operation. I'll be starting tjsdoc-plugin-watcher-doc-regeneration soon and this plugin which depends on tjsdoc-plugin-watcher completes very fast incremental doc regeneration. I have to work out the TaffyDB merging and resolution of doc data, but I expect things to take about a second or less to regenerate a single changed file making it suitable for live regeneration.

Interestingly enough I started with gaze, but it had some silly bugs and fell down in rapid regeneration of all docs / starting / destroying gaze repeatedly. It turns out that chokidar didn't have these issues.

So yes... A bit more of a delay on the launch of TJSDoc, but well worth it IMHO. It may still be 7-10 days before things are fully published and live on NPM.

typhonrt commented 7 years ago

Alright an update is certainly due. The live doc regeneration feature is well under way at this point though it really took some time to restructure the internal data handling of TJSDoc. Now everything works via the DocDB and there is no separate AST storage. Having separate data sources caused complications when it came to deleting old doc data and merging new data. The main DocDB is also now available at all times during execution along with event bindings. The doc generation process also has now been thoroughly overhauled with some initial performance (memory) optimizations.

All doc data generation now happens by static classes instead of creating new objects just for generating data. All generated doc object data is immediately inserted doc by doc into the DB when they are processed versus storing everything in an array with one bulk insert into the DocDB (note: for ESDoc there is no DocDB and the TaffyDB instance is only available in the publisher subsystem with additional access methods in an abstract class DocBuilder. This changes the plugin callback for processing doc objects before insertion as there is now onHandleDocObject which is invoked for every doc object before insertion versus sending one array of bare doc data to a plugin callback. There is a major benefit to this as the AST and specific AST node for the doc object is included along with the already processed doc data which makes it possible for plugins to intelligently provide further processing of doc data with the connected AST node. This should allow Flow typing support for instance to be added as a plugin versus building it into the standard default processing as the specific node associated with the doc object is available for processing which contains the additional flow type nodes. Right after onHandleDocObject is invoked the doc data is filtered to remove the AST data and node unless outputASTData is true in the target project TJSDoc config. By removing AST data flowing through the system this greatly reduces the memory overhead as the AST data is usually rather huge.

Another massive improvement that was necessary was restructuring the old DocFactory which is now DocGenerator. One of the biggest hacks / inefficiencies / and near unintelligible code sections of course non-commented as well of ESDoc is DocFactory. To process the AST in one pass ESDoc rewrites the AST copying / synthesizing class and variable nodes via JSON.parse(JSON.stringify(node)) that are associated with export nodes and placing them at the end of the AST body. This allows processing to occur in a single pass, but destroys the AST and makes the whole aspect of outputting AST moot as it is no longer properly formatted. It took ~10+ days or so to work out a two pass algorithm for processing export nodes in place that didn't require destroying the AST. This is possible because of the incremental addition of doc data to the DocDB such that any updates necessary for the 2nd pass on export nodes modifies the existing doc data via queries to the DB. Removing all of this copying and modification of the AST was a huge memory optimization win and the code is a lot clearer and will greatly aid supporting TypeScript. I'm still adding in comments as this was / is the most complex part of the system. This whole process was possible by unifying on an always persistent DocDB. I hope to get memory usage much lower, but it's now about half of the overhead of ESDoc.

Other interesting updates is that there is a very clean integration of live-server via a wrapper typhonjs-live-server which provides the capability for graceful shutdown of live-server without hanging the process. An accompanying TJSDoc plugin tjsdoc-plugin-live-server starts and shutsdown live-server with the control flow of TJSDoc. A browser is opened automatically for the doc destination source and when docs are regenerated the current page is updated immediately, so this is going to be a very nice feature when complete.

For the latter part of the week I'm finishing the doc regeneration process with final merge support and incremental publishing of sub sections of the DocDB. The nice thing is that the merge process updates the target source file changed and any dependent files as well which certainly will be pertinent once @inheritdoc support is added.

So things are moving along. I'm certainly glad that this feature was taken on now as it really required cleaning up the internals, how data is stored / accessed, the plugin callback mechanisms and initial performance concerns. This also means that all technical debt assumed from ESDoc has been reversed. There is now no part of the architecture that relies on what I'll label as bad hacks.. Everything has been thoroughly vetted and re-architected as necessary.

typhonrt commented 7 years ago

I know it's been awhile since the last update. I had to take a break as my mind was melting a bit with the live doc regeneration work. The good news is that initial support is working and the results are very nice. For a single file changed without dependencies via extension (child classes) live doc regeneration from saving the file to results showing in the browser is less than a second! Each additional dependent file regenerated adds about a 1/3 of a second each mostly due to markdown to HTML conversion in the publishing module. This is also on my 2011 MBP, so a more powerful box should really be quick. I hope to finish up the live regeneration functionality tomorrow with a bunch of tests and then get back on working on the new material-components-web publisher module.

I do have some questions though to anyone following this thread:

For file additions / deletions a full regeneration of all docs is necessary to catch all corner cases without putting in a great deal of effort to root out all of the cases. This is still reasonably fast as TJSDoc is spun up and ready to regenerate all files. Is it reasonable to assume when adding and deleting files that a full regeneration of docs is acceptable? My answer is yes right now and that reduces quite a bit of complexity.

The standard use case is changing an existing file adding or modifying docs. I have forward / child relationships (re: future @inheritdoc support) updating and likely will add reverse / parent relationships for when say the extends keyword is removed from a class such that parent docs update as well which is mostly just updating the indirect / direct subclass relationships at the top of the class HTML. Anything beyond that IMHO should require a full regeneration.

It's easy to do a full regeneration with the watcher functionality as there is a terminal and one just types in 'regen'.

Also for the time being I'm not supporting modification of the published manual output if the related markdown files change or the index / README. This will require a full regen. I may update this, but I'm on the fence on these cases. I may very well support it once I finish off final publisher module efforts.

I think it's important to support single source and test source live doc regeneration where the user is incrementally adding docs to a single file with the expectation that for more complex use cases a full regeneration of all docs may be necessary.

typhonrt commented 7 years ago

Once again a long overdue update. I'm about ~80%+ done with the completely rearchitected publisher module which no surprise has turned out to be a lot of work. Like the rest of TJSDoc everything is plugin based now and various sections of the publisher separated into actions, components, and content (file, class, module) plugins along with internal implementation plugins.

One of the nastier "god objects" of ESDoc is DocBuilder. If you are trying to create a 3rd party plugin which outputs new doc pages you must extend it which makes things rather fragile. Of course all existing ESDoc plugins that output new doc pages break with the v1.0.0 release of ESDoc. All of the doc data / taffy DB access found in DocBuilder has been moved to DocDB which is persistently available across the TJSDoc runtime. Rather than requiring fragile extension of DocBuilder like ESDoc the rest of the general functionality of DocBuilder is now available via events over main plugin eventbus. So any 3rd party plugin authors don't need to extend anything to create plugins which output new doc pages as all the functionality to do so is accessible by events.

Another small, but pertinent change is a runtime pattern to dynamically create the top navigation menu. This is and remains to be a huge PITA for 3rd party plugin authors creating plugins which output new doc pages. It's hard coded in ESDoc and any plugin which outputs new pages which needs to add a menu link in the top nav menu needs to rewrite all HTML output. This is very fragile, but TJSDoc dynamically creates the top nav allowing 3rd party plugins to insert links in an organized manner not requiring any bulk rewriting of all HTML pages output.

Things also have a refined structure moving the HTML, images, CSS locally to each content component versus a bulk copied over template directory. This was easy with HTML / images, but handling CSS in a modular manner turned out to be quite problematic, so it has taken up a good deal of recent effort. With ESDoc there is a static all inclusive style.css that with v1.0.0 has a minor improvement of breaking out the different sections, but still all imported in a static manner in style.css. If you are a 3rd party plugin author that needs to provide extra styling for any doc pages perhaps you could try to hook into the user scripts system, but this is fragile with the only other option being to rewrite all HTML pages depending on what is being added. For instance this was necessary for esdoc-plugin-enhanced-navigation. I guess it should be noted that 3rd party plugins can easily replace internal components like the navigation as it's composed by events versus being hard coded.

In TJSDoc the solution that I'm well onto the way of completing is a separate utility plugin which integrates PostCSS in an event driven manner available over the plugin eventbus. Like other utilities it is separate of TJSDoc though not published on Github yet. One can create a grouping for a CSS file to be composed via appending / prepending directly various subsections along with optionally running PostCSS against a particular subsection with particular PostCSS processing plugins. On final output of the composed CSS file once again it can be processed by PostCSS. So now all CSS is localized to the component that it depends upon and it's possible for 3rd party plugin developers to integrate CSS into the main style.css. I'm also thoroughly going through the existing style.css with an eye to which PostCSS plugins like autoprefixer and precss which will greatly aid theming and creating 3rd party theme oriented plugins to style TJSDoc output with the minimum of fuss. The nice thing for development is that source maps are tracked across all the CSS components which make up a final file, so it's possible to enable source maps for development purposes in Chrome, etc. I'm also quite pleased to announce a light and dark theme for TJSDoc as well; the dark one being my default / favorite!

All material-component-web integration has been complete for a while, but I'm also going to be taking the time to fully complete and update all of my extensive ESDoc plugins verifying that they work with TJSDoc seamlessly, so I have plenty of complex 3rd party plugins as a test bed to verify that integration for plugin authors is considerably smoother and non-fragile compared to ESDoc.

Right now the publisher module is still not committed along with the final watcher / doc regenerator plugin. I've been working on these offline due to the many changes involved. So my commit log on Github is a bit deceptive over the past month or so. Work continues and TJSDoc will get out there. All of this baking is a good thing, but is going on a bit.

I suppose I should comment that I have reduced the amount of time that I've been working on TJSDoc as I did hit a burn out moment near the end of April. I was pushing ~80+ hours a week on TJSDoc from the beginning of the year until then... I'm hitting ~30-40 hours a week now and have started deliberate pool / billiards training which has been a hobby I've neglected for a while as far as getting better. I've finally made the jump to the big tables (9') leaving the bars behind and wake up around 3-4am to work on TJSDoc until 11am then head out to the pool hall for 5 hours of practice (membership hours). I'll be switching to a different pool hall to practice in July which thankfully has membership hours between 3pm and 8pm, so I can get up around 6am which is much welcome, so this month has been a bit crazy. Hopefully in 3 months or so I'll have a 9' table at my home which will help even more w/ time management. With any luck within 6 months or so I'll be at the semi-pro level and plan to travel around to all regional tournaments across California and further coming up, so just not banging away at TJSDoc for 80+ hours a week; perhaps staying more sane... That is debatable though... ;P

typhonrt commented 7 years ago

OK... It may have seemed like my commits died again over the last couple of weeks, but after much gnashing of teeth and offline work a major architecture overhaul is complete! Now I didn't mention in my last update the quandary introduced by incorporating PostCSS into TJSDoc. For the astute you'll know that ESDoc is and hence TJSDoc (was) a synchronous runtime and for those with experience with PostCSS you'll know that it's an asynchronous API and that the synchronous API has been deprecated. While the sync API is still there it will only work with PostCSS plugins that support it. So PostCSS was the straw that broke the camels back so to speak though going async was on my mind for a while... Of course this allows full PostCSS integration and I'll mainly be focusing on the cssnext plugin for the main supported plugin for adding modern CSS features to TJSDoc.

TJSDoc is now a fully asynchronous architecture and has thoroughly adopted async / await! In addition to modifications to typhonjs-plugin-manager which now fully supports async plugin lifecycle operation all plugin loading and the majority of control flow of TJSDoc is asynchronous ready. I'll list a bunch of the async callbacks below, but any method appended with Async is awaited upon where it is invoked / events dispatched. So any 1st or 3rd party TJSDoc plugin may utilize asynchronous operations via using async / await itself or by returning a Promise in the callback.

This is super cool as it opens up so many more possibilities for plugins to interface with the outside world. In particular I'll now be supporting a plugin that queries the Github API (and potentially other SCM systems in the future) to find the exact links to commits when documentation is generated versus providing links to just the master branch which can change over time for all source and navigation linked in generated documentation. Additionally the future CI plugin to automatically upload generated docs to a web app and / or a separate repo can now be completed entirely as a TJSDoc plugin versus more heavyweight / external system. I'm sure the larger community will also come up with many creative ideas on how to integrate documentation generation via plugins with the outside world as well which will be exciting to see!

One should note that ESDoc likely will never become asynchronous or if it does it will once again have to up the Node requirement to v8.x and be a major breaking change to all users / plugin authors. The ESDoc maintainer removed usage of Babel and thus is now dependent on Node support for language functionality. IMHO that was a misstep as documentation generation is a development utility and as such it should run as widely as possible across as many versions of Node as possible. TJSDoc embraces Babel and all stage 2 functionality for implementation and doc generation out of the box.

Here is a mostly comprehensive list of all TJSDoc asynchronous callbacks / plugin hooks: onHandleConfigAsync onHandleDocDBAsync onHandlePostPublishAsync onHandlePrePublishAsync onHandlePublishAsync onHandleSearchIndexAsync onHandleVirtualAsync onRuntimeCompleteAsync onRuntimePreGenerateAsync onRuntimeRegenerateAsync onRuntimeShutdownAsync onRuntimeStartAsync

...and from the plugin manager: onPluginLoad / onPluginUnload

I have to say though that the gnashing of teeth was real and I almost threw in the towel on the async / await conversion as it was fairly tricky. The one bug that got me was using await with a ternary operator. This doesn't do what you may think it does: await typeof TJSDoc.default === 'function' ? TJSDoc.default.generate(config) : TJSDoc.generate(config);

indeed parenthesis are necessary: await (typeof TJSDoc.default === 'function' ? TJSDoc.default.generate(config) : TJSDoc.generate(config));

This is from a utility for testing, so all tests that generate docs which there are many did not work correctly and control flow exited before the end of the TJSDoc runtime completed. I spent days trying to find the problem and this was the last place I looked after copious divide and conquer elimination!

So work should now be progressing again and I'll be back on finalization of the publishing module tomorrow and hopefully TJSDoc will be out sooner than later. I really don't foresee any more major stalling architecture overhauls, so it's just crossing off everything else on the TODO list. Hang in there folks!

typhonrt commented 7 years ago

Now things have been progressing, but again most of the work has been offline regarding the publisher module. I'm getting close to getting everything committed, but as always am sweating the details. The modular CSS support with PostCSS is coming along and a new organization for all official theme plugins and theming resources is located in typhonjs-node-tjsdoc-themes which is still in its infancy. It's taking a bit of time to restructure all of the CSS and getting things hooked up with CSSNext support regarding variable support for easy theme modification. What has taken even more time over the last several weeks is modifying the actual content published such that all output is mobile responsive oriented; regardless of screen size (mobile, tablet, full screen web) the user experience works out well. On small format devices TJSDoc looks and responds very much like a typical mobile app experience compared to a full screen web experience that needs to be scrolled around significantly (IE use ESDoc on a mobile device or check it out with Chrome dev tools).

One recent significant upgrade is swapping out marked for markdown-it which is a maintained and significantly more advanced markdown to HTML converter that also supports plugins. I'm just about to break out support for markdown-it into a separate utility module with TyphonJS plugin support / event bindings. The cool thing is that users will be able to add markdown-it plugins in a projects TJSDoc configuration file and / or TJSDoc plugins can interface with it as well and the PostCSS system for when extra resources are needed such as additional CSS, etc. For instance FontAwesome is now integrated into TJSDoc and a fun markdown-it plugin for testing is markdown-it-fontawesome which allows FA icons to be added to markdown documentation easily. Since material design icons are also included by default if a markdown-it plugin doesn't already exist I'll create a markdown-it-material-icons plugin. If there are any other interesting quality of life plugins that make the documentation output in line with what Github supports I'll also add them as default.

There are no immediate needs insofar as development assistance as admittedly I'm the bottleneck right now in finishing the publisher module. However once it is available I will do an initial alpha release of TJSDoc and it would be great for folks that have been following on thus far to start using things with a round of handling any issues before any public announcements are made. I'd like to complete the official support web site along with architecture / plugin documentation for 3rd parties before an official launch.

jasonmacdonald commented 7 years ago

Did the project die? Such a shame as it seems like something people need. :(

typhonrt commented 7 years ago

Thanks @jasonmacdonald for the push to provide an update. I was bemoaning doing another update as there is nothing to report on the advancement of TJSDoc as I have taken time off to sort out everything else in life besides OSS development; lots of various non-coding projects / home improvement, etc.

From a technical perspective this delay isn't horrible because Babylon 7 is on the horizon and well into the beta cycle. Switching to Babylon 7 for AST generation / parsing will definitely help TJSDoc from an architecture perspective due to the new built in support for Flow & Typescript which should provide a unified runtime for JS / Flow / Typescript. typhonjs-escomplex and various modules will also be updated to Babylon 7 and this will greatly aid in adding unified complexity analysis data to the generated DocDB directly in TJSDoc.

Basically I'm in the process of optimizing everything else in life before making a final push on TJSDoc which I'd like to dedicate full time+ effort through release / publicizing / initial maintenance which I estimate as a 2-4 month sustained effort.

Extraneous life details. I've acquired a 9' pool table for intense training that I'm doing now to get up to the semi-pro / pro level and there are various custom projects to get this setup functioning smoothly; just installed a motorized raise / lower mount + remote dimmer control of an LED light setup yesterday. Since June I found a regular schedule of getting up at 6am and trying to fit in 8 hours of coding + 5 hours of practice a day at a pool hall to not be optimal for variable hours of coding or practice.

For me at least when I do my best coding getting in the zone often requires 12-20 hour pushes as necessary to fully complete a complex task in whole and leads to a multiphasic schedule not compatible with a regular off site and locked hours (3-8pm) for pool training. Hence I was considerably less productive between June - September for coding in general.

I guess the good news is that I should be done with all the outside projects by the end of October and should have an optimum ability to code and do intense pool training around the clock on site. Work on TJSDoc will continue and won't die as a considerable amount of effort is already invested and indeed it is something I and people need.. :D

typhonrt commented 6 years ago

Hi folks... Happy new years! I just want to give a status update on how things will progress this year for TJSDoc. It's true that I'm halfway through a sabbatical and plan to return to TJSDoc in April; I have extended it to 6 months which is longer than I originally anticipated / last updates. I'm turning 40 in March and it's just been nice to take some time off from coding after all these years. I'm spending the next 3 months practicing pool / billiards full time attempting to get competitive at the open / pro level. I've got all the tools / 9' table and such setup at my place now and am doing ~8+ hours of practice a day. San Francisco is a bar pool city / scene (small tables) and there is no intermediary step between that and open / pro tournaments on the West Coast on the big tables. It's something I always wanted to do, but never found the time to put in the deliberate practice required. Now I certainly won't turn pro as there is no $$$ in pool, but I do want to make the pros sweat a bit.. I have already made great progress and will be supercharging that in the next 3 months; perhaps I'll be making the TV / streaming table fairly often at upcoming tournaments so you guys can tune in... If this is the extent of any "midlife crisis" then I think I'm doing OK... ;P

I was supposed to take this time off at the beginning of last year, but the fire got lit under my tail to hard fork and begin work on TJSDoc in earnest given how things played out with the ESDoc maintainer at the time after much patience ran its course. As things go it did take much longer to get things out especially as the unique features of TJSDoc developed admittedly along w/ a bit of feature creep. Very good creep in my opinion bringing new functionality and architecture well beyond ESDoc. It should be noted that TJSDoc hard forked prior to the less than ideal architecture changes IMHO found in the post v1.0 reality of ESDoc. TJSDoc was forked at the right time to capture the best of breed ideas embodied early in ESDoc extending them significantly.

I know it's kind of a pain to wait for a tool like TJSDoc as good documentation generation let alone the ability to document highly modular / many-repo projects, a major goal of TJSDoc, is not possible w/ existing tools. Keep in mind though that the cost in time to do so is not cheap given the full time effort. Given the expenses incurred for development (significantly more than I thought!) and no immediate ROI potential this effort is slightly awkward as the cost involved is something that normally would be allocated for a for profit product. Under the hood things are fairly complex which is why it's hard to bring in any contributors / other maintainers at this moment. A large amount of the core code with reasonable inline comments is available to review; sans the publisher module. Albeit, it may be a bit tricky to follow the eventbus messaging between modules. I actually plan to add a plugin which creates graphs illustrating eventbus usage between modules / components and other docs for the pattern. TJSDoc will likely be the biggest thing as far as impact I will likely contribute to the larger open source Javascript / Typescript communities.

Ultimately, I think it's good that I am taking this time off, so that I can come back to TJSDoc w/ enthusiasm and not only get it launched, but have the energy to then build a community with an open organization style and provide ongoing maintenance as I expect TJSDoc will get some legs on launch. It would have been less ideal to launch then disappear for 6 months in my opinion as that is the pattern all too common w/ ESDoc in regard to progress or lack thereof. I fully expect to put in at least a solid 3 months full time+ push to launch TJSDoc April - June then be highly involved as a community grows, but openly welcome contributors and new maintainers. When April rolls around I'll be back to long days of coding and just put in 1-2 hours a day of pool practice; with a table at my home this is easily possible any time of day now!

As mentioned previously from a technical perspective this launch window will also nicely coincide with launching against Babel / Babylon 7 with a nice buffer for initial teething issues that may come up with the next major Babylon release. I'm really excited for Typescript support in Babylon 7. This upgrade likely will involve significant core architecture work and it's just great to get it done upfront before an official launch of TJSDoc. Also as mentioned previously I will be updating typhonjs-escomplex to Babylon 7 expanding support for ECMAScript and Typescript and provide integration w/ TJSDoc for code complexity analysis that can be seamlessly enabled into the doc DB generation and final presentation. This will definitely be a killer feature!

Thanks for hanging in there. TJSDoc is not going to die! I'm confident on aiming for the end of Q2 launch window.