search

EnterTheNameHere / esdoc-monorepo

Monorepo for custom esdoc and all its packages.
Other
7 stars 1 forks source link

How to document dynamically generated methods? #31

Open VividVisions opened 9 months ago

VividVisions commented 9 months ago

Since there is no @function tag, there's currently no way to document dynamically added methods. Or have I missed something?

EnterTheNameHere commented 9 months ago

Can you provide a minimal example code? It would help to check how the parser processes it.

VividVisions commented 9 months ago

Here you are:

/**
 * Test class
 */
class Test {
   /**
    * Foo function.
    */
   foo(str) {
      return str + '!';
   }
}

['bar', 'baz'].forEach(name => {
   Test.prototype[name] = function()  {
      return this.foo(name);
   }
});

// With JSDoc or other tools, I would've documented the functions like this:
// (Maybe with a @memberof, when the comments are outside of the class.)

/**
 * @name #bar
 * @function
 * @description ...
 * @returns ...
 */

/**
 * @name #baz
 * @function
 * @description ...
 * @returns ...
 */
EnterTheNameHere commented 8 months ago

So I did some research into what can be done, and to not let you wait for too long, this is what I found:

@function tag functionality is not at all implemented. Simple hacking it in quickly is not easy task as there is no posibility to change code here and there and it would fit in as a piece of puzzle called ESDoc.

Unfortunately ESDoc is old, from ES5- era, and it shows. My fork updates dependencies, replaces unsupported packages, puts all in single monorepo and allows use of modern JS, If you have project, which used ESDoc, you can replace old ESDoc with my fork and it "should work"™.

If you are looking for proper documentation generator, I recommend checking other tools and not waste your precious time with ESDoc.

I have plans how to make ESDoc better, but I won't lie, I'm slow and few features I work on are in various states of completion and I'm not ready to trust them for retail use...

That's TL;DR, I'm sorry I don't have a solution for you.

I'm including some more info on why it's not quick hacking to implement @function - you don't need to continue reading, I'm leaving it here as notes which should be left somewhere...

ESDoc is abandoned, so I forked it, updated dependencies, replaced unsupported dependencies, added some new functionality here and there... The whole process of how ESDoc works can be divided into three places (phases):

(phase 1) and (phase 2) are done by core ESDoc, (phase 3) is done by publish-html-plugin

So we take file, get AST tree from it, and we traverse this tree, trying to recognize known patters - this is class declaration, this is member declaration, this is function declaration, this is exported etc.

For each recognized pattern a *Doc entry (ClassDoc, MemberDoc, FunctionDoc) is created. It will receive AST node, AST tree and extracted tags as parameters to it.

This is where first obstacle is found, "extracted tags": Obviously JSDoc is in comments. Comments are attached as leadingComment or trailingComment to each AST node. ESDoc then in (phase 1) splits comment's text by @ character and array of slightly edited text (sanitization, trimming) is passed to *Doc.

There is no JSDoc parsing/lexing done. ESDoc just looks if there is @export or @external or typedef in comments, otherwise it cuts AST nodes and let *Doc's do their own work. That's all what (phase 1) does.

So *Doc is created, having AST node, AST tree and extracted tags to work with. This is where (phase 2) is done - *Doc decides what is the name of entry, what longname should be, access privacy (private, protected, public), is symbol it exported, what type is it, does it have author, version, since, does it have description etc.

Let's talk more about comments. AST node has comments attached as leadingComments and trailingComments:

/**
  * Leading comment
  */
{
    // Block AST node
}
/**
  * Trailing comment
  */

In (phase 1) comments are checked for @external, @export and typedef, as these can stand alone - not connected to any code AST node. They are treated as to be "virtual", because they do not exist - but not for documentation entry - they are used for type recognition. You can do @external {WebWorker} put-mdn-link-here and in presentation where type WebWorker is used, it will be link to mdn website where you can find info on WebWorker.

In (phase 1) comments are chopped by @ character, so:

/**
  * An example function with `useless` parameter, returning **true**.
  * And look,
  * It's on multiple lines...
  * @param {boolean} [useless=true] - better not be false
  * @returns {true}
  * @see {link-to-something-more-interesting-here}
  */
function foo(useless = true) {
    returns true;
}

will create an array with:

[
    {"tagName": "@desc", "tagValue": "An example function with `useless` parameter, returning **true**.\nAnd look,\nit's on multiple lines..."},
    {"tagName": "@param", "tagValue": "{boolean} [useless=true] - better not be false"},
    {"tagName": "@returns", "tagValue": "{true}"},
    {"tagName": "@see", "tagValue": "{link-to-something-more-interesting-here}"}
]

*Doc can extract data like name of parameter, what is description etc. from these. Notice the multiline text - the starting [space/tab]* is trimmed.

So ESDoc, in it's two phases, deals with comments like this:

/**
  * class node leading comment
  * - (phase 1) checks for @external, @extend needing special treatment
  * - all tags for class must be here, because ClassDoc will be looking here in (phase 2)
  */
class TestClass {
    /**
      * method node leading comment
      * - (phase 1) checks for @external, @extend needing special treatment
      * - all tags for foo must be here, because MethodDoc will be looking here in (phase 2)
      */
    foo(str) {
        return `${str}!`;
    }
    /**
      * Trailing comment
      * - is checked in (phase 1)
      */
}
/**
  * Trailing comment
  * - is checked in (phase 1)
  */

Now to our slightly expanded code example:

/**
 * Test class
 */
class Test {
   /**
    * Foo function.
    */
   foo(str) {
      return str + '!';
   }
}

/**
 * @name #bar
 * @function
 * @description ...
 * @returns ...
 */
['bar', 'baz'].forEach(name => {
   Test.prototype[name] = function()  {
      return this.foo(name);
   }
});

/**
  * @function #quax
  * @memberof TestClass
  */
TestClass['quax'] = function() {
    return this.foo('quax');
}

/**
  * @function
  */
const obviouslyNotAFunction = null;

/**
  * @function
  * @memberof ThisIsNotWhereIDidParkMyClass
  */
const thisIsNotTheFunctionYouAreLookingFor = 42;

// With JSDoc or other tools, I would've documented the functions like this:
// (Maybe with a @memberof, when the comments are outside of the class.)

/**
 * @name #baz
 * @function
 * @description ...
 * @returns ...
 */

Let's implement @function!

First, let's look at what ESDoc finds: class TestClass method foo variable obviouslyNotAFunction variable thisIsNotTheFunctionYouAreLookingFor

These are the 4 *Doc entries which will be created, ClassDoc, MethodDoc, VariableDoc and VariableDoc.

Where can @function tag be found?

@function #bar as:

@function #quax as:

@function (obviouslyNotAFunction) as:

@function (thisIsNotTheFunctionYouAreLookingFor) as:

@function #baz as:

There's one more place we can find @function tags - the File AST node, which have comments attached.

Now more about @function tag:

If it's present in leadingComment of detected *Doc, it should be easy to detect it in (phase 2) and ...change type of *Doc? Ok, with only reference to itself and AST nodes, changing our type to other *Doc needs somehow telling that to database. Different *Doc also require different behavior, so basically it's throwing out detected *Doc and creating new FunctionDoc/MethodDoc based on the @function.

If @function is in undetected AST node comment, it's effectively lost and cannot be detected in (phase 2). This case is not shown in the example code, but can easily happen with multiple undetected AST nodes one after other.

If @function is in trailingComment, it will be visited at least once in (phase 1). This is true for @function #bar, @function #quax, @function (obviouslyNotAFunction), @function (thisIsNotTheFunctionYouAreLookingFor) and at last @function #baz.

Yes, I know, we're telling ESDoc that those two variables are functions, but that's our order, ESDoc might warn something is fishy, but ultimately document them as global functions...

I think we found our place - (phase 1)! It visits leading and trailing comments and we can make it to look at all comments, not only those of detected nodes! If we are clever we can also gather more info ahead of creating *Doc and ease *Doc's work by providing more tags with data we extract!

It kinda works - we can replace normal FunctionDoc and MethodDoc (phase 1) creation by creating them ourselves. Feed it all other data like @name, @memberof, @desc and others and rejoice at seeing those in presentation!

But the data we fed to FunctionDoc and MethodDoc we created is not all shown or is incorrect? Ah, some tags are ignored, and instead data is extracted from AST node? Nice... Ok, let's hack that.

Method not pointing to correct class? Let me take longname of that class, since it exist - wait, I don't have reference to it... Does it exists already? Should I create it if it doesn't exist? I probably need reference to database... What if (phase 1) detects the class if I create it, and rewrites it? All I have is AST node and tree

Hmm, this class, which doesn't exist, is not shown, let's try to simulate it and create virtual... What if it exist in other file, which is processed by ESDoc?

What do I use as location of the function if it's virtual? Let's point to the place @function #name is. Wait, I have only text without context, if it's (phase 2), I need to get the comment reference and somehow find text which was edited, if it's multiline, or if user didn't specify a name with @function, but instead uses @name and @memberof or combination of those. If only JSDoc had AST too, ready for me to easily lex it.

So this way we cannot "create" functions and methods which are not detected by ESDoc in the first place. Ok, then let's teach ESDoc to detect: ['bar', 'baz'].forEach( (name) => { TestClass[name] = function() { return this.foo(name); } });

That's easy, we have Member with property, Assignment of function, preceded by Call with forEach, then name as parameter and name as property name, so we know it assigns elements of array which have bar and baz. Easy, let's tell ESDoc we found two functions - wait, it allows only one function (as a return value)? But I have two. Here I have to return what I found as a single return value. I cannot return two. Should I create one MemberDoc myself, and return the other as return value? Should I create both MethodDocs myself, and return null? Should returning of multiple detected nodes be implemented?

Oh, if I create MemberDoc, it doesn't have correct longname of parent class, because there's no ClassDeclaration above our node in the AST tree... And I don't have reference to parent, nor a way to query if parent whether class with that name even exist.

Ok, this is not simple hacking, this is where multiple test cases, not just unit tests needs to be made, and breaking something in (phase 2) and (phase 3) not recognizing changes is highly possible.

We didn't even talked about (phase 3). The publish-html-plugin has... hardcoded HTML in it (facepalm). So any change needs to be fitted to fit the hardcoded HTML, or changed in that plugin too. Want virtual class and functions/method? Well, if existing ClassDoc, FunctionDoc and MethodDoc are not really useful in this situation, too bad, there is hardcoded HTML. It's not that terrible... Whole plugin needs to be compiled with each smallest change and same code is repeated on multiple places. What is terrible is JSDoc parsing of types is done here... Not in ESDoc core, but in the plugin...

This short (pun intended) text is explanation why change needs to be done instead of hacking in the support for @function.

VividVisions commented 8 months ago

Thank you for this detailed response!

If you are looking for proper documentation generator, I recommend checking other tools and not waste your precious time with ESDoc.

Do you know any? I've looked at so many but none could handle modern code. Or Node.js' exports structure.

EnterTheNameHere commented 7 months ago

Sorry for taking two weeks,

well apart from JSDoc itself, which seems more like exporter than a documentation generator "suite", one interesting application seems to be documentation.js. That's all I know of.