search

TypeStrong / typedoc

Documentation generator for TypeScript projects.
https://typedoc.org
Apache License 2.0
7.69k stars 696 forks source link

Version 0.20 tracking #1364

Closed Gerrit0 closed 3 years ago

Gerrit0 commented 4 years ago

This issue tracks the work being done on the next feature release of TypeDoc. I will be posting a weekly update here covering what I've done this week, as well as what's next to be done.

You can try out a beta of 0.20 by installing typedoc@beta with npm i typedoc@beta.

Originally, this release was slated to be version 1.0, it contained a major rewrite which fixed most to all of the issues I have with the current implementation of TypeDoc. However, I've since realized that this was a bad idea - I broke too many things, so "library mode" will be releasing as 0.20 before the end of the year.

Old text This issue tracks the work being done on version 1.0. I will be posting a weekly update here covering what I've done this week, any new problems discovered, and updating the checklist in this post with what's been completed. The checklist in this post was originally taken from [this post](https://github.com/TypeStrong/typedoc/pull/1184#issuecomment-650809143) in the original PR, without including any of the already completed items. If you are interested in looking at the work in progress package, you can install `typedoc@dev`, which will be published whenever a change in version is pushed to the `library-mode` branch. At the current rate that I'm getting things done, it's looking like this won't be ready until November/December... I was really hoping for sooner, but can't devote every weekend to this, though I try to do at least one task per week ;) ## Tasks ### TypeDoc proper - Finish `ObjectReflection` converter (medium) - Fix converter for destructured outputs (small) - Add indexed signatures to models that can have them. I forgot they existed when rebuilding the models... (medium) - Detect and convert static class members (smallish) - Update plugins that were broken when rebuilding converters - Decorator plugin (medium) - Implements plugin (medium) - Deep comment plugin (small, probably merge with the existing comment plugin) - Test on a bunch of different projects, particularly large ones. (Apache Arrow, Jupyterlab) (??? depends on what issues are found) - (done) Review existing options to see which ones still make sense. (medium) - Update variable converter to possibly convert variables as functions (smallish) (#858, but without requiring `@function`, #401, ) - Sort order option (#112), see src/lib/plugins/sort.ts ### Comments - Make `@category` work again (medium) - Fix {@link} and [[link]] (medium) - Make `@inheritdoc` work (small/medium) ### Themes - Support hideGenerator flag (smallish, need to allow templates access to `application.options`) - (done) Make overriding templates easy for users (small/medium) - (done) Write docs on how to create a custom theme (medium, I want to do this right, not just the minimal page we currently have, particularly important because 1.0 breaks existing themes) - Talk with the people that built the markdown plugin about how to integrate (medium) - Make search work (medium) - Make sure mobile experience is decent (medium) - (done) Make light/dark code themes configurable (small) - Add icons for reflection types (medium) ### Plugins / Developer Docs - Take a look at existing plugins to see what events they use, if the existing emitted events meet those needs, or if we need other events. (medium) - Write "how to build a TypeDoc plugin" (medium) - Update the [development page](https://typedoc.org/guides/development/) on the website (medium) - Figure out what should be exported, and export only those symbols ### Extra (nice to have, but won't block a release) - Highlight headings according to the color scheme chosen by users, not just code snippets (medium? Tricky. I spent a few hours on it, failed) - Add a reflection for "union enums" microsoft/tsdoc#164 - Update models to make type parameters have a reflection - #1319. - Recognize global objects and combine them all into a single special module, - Figure out how `@event` ought to work. The old way of handling it flat won't work anymore... and it isn't particularly clear how it should be used. - Create a second theme ### Future (almost certainly not in 1.0, but I'm thinking about it) - JSON deserialization (#910) - Support for `///` comments microsoft/TypeScript#39930 ## Help out There's still quite a bit to be done. Help on any of the uncompleted items above is greatly appreciated. I could especially use some help on making the default theme better - frontend dev is not my specialty. If you are having trouble, feel free to reach out by making an issue for the task you're working on, through the [TypeScript Discord](https://discord.com/invite/typescript), or through the TypeDoc Gitter.
bennypowers commented 4 years ago

@Gerrit0 I'm happy to help with light/dark theme for typedoc, based on css custom properties and prefers-color-scheme. Maybe using this: https://github.com/GoogleChromeLabs/dark-mode-toggle

Gerrit0 commented 4 years ago

Thanks @bennypowers! We've chatted about this a bit on Discord.

What's been done this week is published as typedoc@1.0.0-dev.3.

Maintenance

Development

Work added

Gerrit0 commented 4 years ago

Fairly small update this week, I didn't have much time to work on TypeDoc.

Maintenance

Development

Gerrit0 commented 4 years ago

My apologies for a lack of an update last week - I went out with some friends and didn't handle the junk food well...

This week was also fairly unproductive... but at least I did one thing! Work has been very busy, not much brain left over for the weekend. Particularly for the rather tricky task of making comment parsing better. While working on {@link}, I realized there were several issues with the existing implementation which would cause problems with the later comment parsing/rendering work... so I set out to remedy that.

Development

Gerrit0 commented 4 years ago

This weekend I spent trying to avoid the rewrite. I haven't succeeded yet, but I made some good strides, I believe it is possible to (mostly) use the existing architecture and avoid spending a bunch of time rebuilding code that works today. Toward that end, I ripped out a few options:

I mostly have library mode working, but am still dealing with some issues where we sometimes need to create references, and sometimes not.... I suspect I'll have a working prototype next week.

Manish3323 commented 3 years ago

in our typescript library repo, we are experimenting with typedoc to generate api documentation. as typedoc@next, i could do mode="library", which i am guessing is a best mode for us as its a web library. we have both .ts and .tsx files that exposes modules and react-components to our users. when i do : mode="library", .tsx files are ignored. mode="modules" those are parsed and .tsx components are documented.

note : i'm not sure whether it's a bug or not.

my current ts config 

"compilerOptions":{
"jsx": "react",
... other ts config's
...
"typedocOptions": {
    "inputFiles": [
      "src/clients",
      "src/components",
      "src/models"
    ],
    "mode": "modules",
    "out": "../docs/ts-docs",
    "exclude": [
      "**/*+(**Impl|**PostCommand|**WsCommand|**index).ts",
      "**/decoders/*",
      "**/*+(**Utils).ts"
    ],
    "jsx": "react",
    "excludePrivate": true,
    "plugin" : "typedoc-plugin-external-module-map",
    "external-modulemap" : ".*\/(clients|components|models)\/",
    "categorizeByGroup": true,
    "categoryOrder": [
      "Services",
      "Agent-Service",
      "Others"
    ]
  }
}
Gerrit0 commented 3 years ago

I wasn't quite able to get library mode working this weekend - mostly due to needing to fix how TypeDoc handles type references. It used to use the getFullyQualifiedName function on TS's type checker, but this doesn't always create a unique name. Particularly, it doesn't work with any global symbols... so the detection for if we had already converted a symbol and needed to create a reference to it was broken, which is an essential part of library mode. While fixing that, I also ran into some other problems with how we deal with types.

I fixed these issues + restructured the type converters slightly, which makes it possible now to get warnings when TypeDoc doesn't properly implement a converter for some type and also improved the converter speed, lookup is now O(1) instead of O(n) with n being the number of converters. This revealed a few types that we don't handle right now that I didn't have time to get to. I'll probably skip them for now and just focus on getting something working next week.

Plan for next week:

With those done, I'll hopefully be able to put out a beta which works.

Gerrit0 commented 3 years ago

The beta is up! npm install typedoc@beta will install 0.20.0-beta.3. This build seems to work pretty well. I ran it on TypeDoc itself, and it produced significantly better output than file or modules mode (+ made obvious that we should be exporting more than we do...) There were more reference problems leftover that took some sorting out, but I think I've gotten all of them now.

I also spent some time this weekend trying to test this on some projects that I know are currently using the library mode build in #1184... for the most part, this seems to work equally as well, though I'm sure that people more familiar with the projects will be able to point out things I've missed.

A few notes about usage change:

  1. Including a tsconfig file is essential. If TypeDoc doesn't find it, tell it where it is with --tsconfig <path>
  2. You probably don't need the exclude option anymore. It is only used for filtering entry points if you give TypeDoc a folder instead of a file.
  3. If you re-export something from node_modules, it will be included in your docs. If you want to avoid this, use --excludeExternals --externalPattern "**/node_modules/*"
  4. inputFiles has been renamed to entryPoints to better reflect its usage.
  5. If you only provide one entry point, TypeDoc will not create a module, members will be included directly under the project. This behaves similarly to file mode. If you provide multiple entry points, TypeDoc will create one module for each entry point.

How library mode works:

  1. Start the compiler using the provided tsconfig
  2. For each user provided entry point (through the entryPoints option in a config file or on the CLI)
    1. Ask TypeScript for directly exported symbols and document them
  3. For each user provided entry point
    1. Ask TypeScript for reexported symbols and document them, or if already documented, create a reference

This two pass system makes the results more reasonable if you have more than one entry point.

Plan for next week:

Gerrit0 commented 3 years ago

This week:

Bonus: Had some extra time:

Plan for next week:

Stretch for next week:

bennypowers commented 3 years ago

If you're looking into inheritance, can I encourage you to spend some time on patterns like

export function CustomElementMixin<Base extends Constructor<HTMLElement>>(superclass: Base) {
  return class Mixed extends superclass {
    thing: number;
  }
}

Expected: I'd expect to see inherits HTMLElement, and maybe in a perfect world a link to MDN (that's even something a plugin should do)

Actual: Typescript ends up copying in the entire HTMLElement interface to the .d.ts, and as a result, typedoc ends up reporting on TEXT_NODE and other HTMLElement stuff that's better documented on MDN than my docs site 😄

So yeah this is probably a TS thing, but if you can hack together a nice fix, it would be nice not to have to https://github.com/apollo-elements/apollo-elements/blob/55e943d3eca62541a52881ab4a43ae7bf587a6d1/scripts/fix-typedoc.ts#L174-L189 (turns out JSDOM's closest implementation is wicked slow 🤷 ).

So no pressure, but if in the course of things you hack out a fix, I'd be glad to hear it

Gerrit0 commented 3 years ago

So... mixins != inheritence, not even close. Mixins = hack everything together in a construct which breaks all reasonable ways of interacting with the compiler API. I tossed that into a file and ran the current beta on it, and apparently I've already made the output worse.

image

I think mixins need their own special converter + reflection type. Useful docs for them aren't anywhere near the same as useful function docs.... I'll make a note in my list to look at that, but can't promise I'll get to it before releasing 0.20.

bennypowers commented 3 years ago

No pressure. Thanks for the update.

Gerrit0 commented 3 years ago

I wasn't able to get inheritance working properly this weekend - but I think I'll be able to finish it up next weekend. I also spent a bit of time on some other issues:

Next week:

Gerrit0 commented 3 years ago

"finish fixing inheritance" was apparently too ambitious. I made good progress, but still don't have everything working. I hope to have some time this week to finish up the remaining issues and publish the work for people to find all the edge cases I missed. The approach I'm taking (ask the type checker for almost everything) works well, but unfortunately has required making changes to literally every node converter. I still need to make the variable converter and class converter work properly again. (Started with interfaces, because statics are annoying)

marcofugaro commented 3 years ago

Thanks for your work on this @Gerrit0, it has been amazing.

I tried the new beta, and wanted to share some feedback, tought it might be useful to you.

The ex-library mode now works perfectly! There are no issues between module/classes.

There is only one issue I haven't been able to fix.

There is a class with a list of parameters, which apparently are in correct JSDoc format,

that aren't being shown in the class Typedoc page.

Any of your input on this would be greatly appreciated, maybe I'm doing something wrong?

The PR with the beta is here https://github.com/pmndrs/cannon-es/pull/54

farfromrefug commented 3 years ago

@Gerrit0 could i ask to bring back ignoreCompilerErrors ? In my case i omit on purpose a lot of typings files to make typdoc faster. It is on purpose and i know it wont have any consequence on my doc. I could really use that flag. You could set it to false by default. Thanks

Gerrit0 commented 3 years ago

(I pulled the above questions into new issues)

I've been busy. I took some of this week past off, so had extra time for TypeDoc.

  1. Shiki - TypeDoc used to use Highlight JS for syntax highlighting. For the most part, Highlight JS works well, however it maintains a custom parser. Originally, this was prompted by Highlight JS's missing support for tsx, but it has other benefits. Shiki makes it possible for us to get the individual themed tokens. This will make it possible to seamlessly switch between light/dark theme once I get to that... Shiki uses the same underlying code to highlight as VSCode does. However, it doesn't support all of the same languages as Highlight JS. typedoc --help will now print out a list of supported languages, to ease migration.
  2. Inheritance - It works properly now! TypeDoc will pick up inherited properties specified through generic types. #1386 for one example. - I did break one thing, TypeDoc now doesn't resolve type parameters in concrete classes inherited from generic ones. (e.g. class FooList extends Array<Foo> will have functions returning T... even though we know it should be Foo) I might be able to fix that today or tomorrow.
  3. Missing entry points - TypeDoc previously didn't give any sort of indication if it wasn't able to find one of the entry points requested, which lead to confusing behavior where documentation would be empty for no apparent reason.
  4. excludeExternals/externalPattern - External members are now determined based on the file they are declared in. This is a change to previous behavior where each file was determined to be external or not. The externalPattern option now defaults to **/node_modules/**.
  5. JSON schema - #1389 reported inaccurate JSON output types - this was possible because TypeDoc's serialization code used some casts to dynamically build up the output. This has now been fixed, the root JSON object created with --json is typed as import("typedoc").JSONOutput.ProjectReflection
  6. logLevel option - I've found it useful to add some logging to typedoc to track down issues, but much of it isn't useful for a normal user. This lets me add as many verbose logs as desired without cluttering the output unless requested.
hugomrdias commented 3 years ago

@Gerrit0 0.20.0-beta.24 looks awesome, seems like i can finally point typedoc to the src folder instead of type declarations (src is commonjs with JSdoc types)

One thing that looks broken is the export symbol name:

Screenshot 2020-11-30 at 13 46 17
Gerrit0 commented 3 years ago

That export name is correct - 0.20 documents what is exported under those names. TypeScript's equivalent of module.exports = is export =, there's no better name associated with that export.

However, because your index.is file does the more TS friendly exports.name =..., it should have the correct name there.

You probably don't want to pass TypeDoc the src directory. If you instead give it src/index.js you should have much better output.

hugomrdias commented 3 years ago 
export = Key;
import { Key } from "./key";

typedoc can't get Key from this ? you saying we can't use default exports with 0.20 ?

Anyhow changed everything to named exports and stuff starts to look nice and pretty, but still have errors in my utils file

TypeDoc exiting with unexpected error:
AssertionError [ERR_ASSERTION]: The expression evaluated to a falsy value:

  assert(declaration)

    at Object.convertTypeAlias (/Users/hd/code/pl/interface-datastore/node_modules/typedoc/dist/lib/converter/symbols.js:104:5)

declaration is undefined and symbol is this

<ref *2> [
  <ref *1> NodeObject {
    pos: 208,
    end: 248,
    flags: 4325376,
    modifierFlagsCache: 536875008,
    transformFlags: 0,
    parent: NodeObject {
      pos: 186,
      end: 252,
      flags: 4325376,
      modifierFlagsCache: 0,
      transformFlags: 0,
      parent: [NodeObject],
      kind: 311,
      comment: undefined,
      tags: [Array]
    },
    kind: 331,
    tagName: IdentifierObject {
      pos: 209,
      end: 216,
      flags: 4325376,
      modifierFlagsCache: 0,
      transformFlags: 0,
      parent: [Circular *1],
      kind: 78,
      originalKeywordKind: undefined,
      escapedText: 'typedef'
    },
    comment: undefined,
    typeExpression: NodeObject {
      pos: 217,
      end: 233,
      flags: 4325376,
      modifierFlagsCache: 0,
      transformFlags: 0,
      parent: [Circular *1],
      kind: 301,
      type: [NodeObject]
    },
    fullName: IdentifierObject {
      pos: 234,
      end: 248,
      flags: 4325376,
      modifierFlagsCache: 0,
      transformFlags: 0,
      parent: [Circular *1],
      kind: 78,
      originalKeywordKind: undefined,
      escapedText: 'PromiseOrValue'
    },
    name: IdentifierObject {
      pos: 234,
      end: 248,
      flags: 4325376,
      modifierFlagsCache: 0,
      transformFlags: 0,
      parent: [Circular *1],
      kind: 78,
      originalKeywordKind: undefined,
      escapedText: 'PromiseOrValue'
    },
    locals: Map(1) { 'T' => [SymbolObject] },
    symbol: SymbolObject {
      flags: 524288,
      escapedName: 'PromiseOrValue',
      declarations: [Circular *2],
      parent: [SymbolObject],
      isReferenced: 67108863,
      id: 354
    },
    localSymbol: SymbolObject {
      flags: 0,
      escapedName: 'PromiseOrValue',
      declarations: [Array],
      parent: undefined,
      exportSymbol: [SymbolObject]
    },
    id: 3317
  }
]

if i skip the convertTypeAlias call it renders but my utils.js file still gets the export= even though it only uses named exports (.d.ts generated by ts proves it).

Gerrit0 commented 3 years ago 
export = Key;
import { Key } from "./key";

typedoc can't get Key from this ? you saying we can't use default exports with 0.20 ?

While typedoc could figure out that your local variable is named Key - that is not what it is exported as. It is exported without a name. This is one of the bad things with node's module.exports = ... and export default. Authors of a library probably intend that users name their imports after the library (e.g. import React from "react")... but the name that they actually export from the library is default, or no name at all in the former case. If TypeDoc documented the export as the local name, it would be confusing to users of your library.

Can you open a new issue for the issues you're seeing with instructions for reproducing them? I'd like to keep this issue fairly clean for people looking for updates on the 0.20 release.

Gerrit0 commented 3 years ago

Since the last update v0.20.0-beta.26 is out with:

I also spent a fair bit of time attempting to solve the generic issue from #\2 above, but wasn't able to fully solve it this week.

simllll commented 3 years ago

thanks for the new release, I just played around with it. Looks great so far, one issue I stumpled upon: I'm in a monorepo,and my tsconfigs extend the tsconfig in the root, e.g.:

{
    "extends": "../../tsconfig.json",
    "compilerOptions": {
        "outDir": "./dist",
        "rootDir": "./src",
        "tsBuildInfoFile": "buildcache.tsbuildinfo",
        "target": "es5"
    },
    "include": ["./src"]
}

this somehow let typedoc think it needs to pick up the name and README from the root, instead of the current directory. The types are documented correctly though, only the "Project" name and the content of the readme file is wrong.

Thanks for your work!

Update 1: Interesting, in another project it acts differntly. the package name is applied correctly, but the package itself do not have a README, but a folder on level up has one, and this one is picked. The difference between the two monorepos is, that I hoist node modules in the one (where it's not working), and not in the other. It's the only difference I could find, the strange thing is.. typescript and typedoc are both installed in the "root" of the monorepo.

Gerrit0 commented 3 years ago

Hmm... that looks like an unintentional side effect of how the PackagePlugin grabs paths changing to deal with multiple projects - it used to get the root names from the singular program, but I'm guessing that what TypeScript considers "root names" due to a change in how the program is constructed, so it is including declaration files in node_modules among the root names - which would explain why hoisting makes a difference.

Looking at this again, it doesn't make much sense to me to use the root names here - it's more appropriate to look for a readme beginning with the common directory for the entry points.

This, as well as a fix to improve the speed for repos using project references, but not a solution style tsconfig, has now been published in 0.20.0-beta.27 (or will shortly... CI is running)

edgardmessias commented 3 years ago

The tag @event is not working after 0.20-beta.24 for methods

euberdeveloper commented 3 years ago

I tried the 0.20.0-beta.27, I am having some trouble because I use module-alias in my project but the --baseUrl option is disappeared.

vitaly-t commented 3 years ago

I've just tried 0.20.0-beta.27, and getting this:

Error: Tried to set an option (mode) that was not declared.
Error: Tried to set an option (excludeNotExported) that was not declared.

NodeJS - any version, OS - Windows 10.

My typedoc.json file:

{
  "name": "SUB-EVENTS v1.8",
  "readme": "README.md",
  "mode": "file",
  "out": "docs",
  "theme": "minimal",
  "excludeExternals": true,
  "excludePrivate": true,
  "excludeNotExported": true,
  "disableOutputCheck": true,
  "exclude": [
    "./extras/**/*.ts",
    "./test/**/*.ts"
  ]
}
Supereg commented 3 years ago

@vitaly-t remove the „mode“ option. It’s was removed as „library“ mode is now the default.

vitaly-t commented 3 years ago

@Supereg Ok, just did it, so the first error is gone, but the second one remains:

Error: Tried to set an option (excludeNotExported) that was not declared.

Is there some other default setting I need to apply for this option?

Note that if I just remove that option from configuration, I start getting this:

Error: No entry points provided
Supereg commented 3 years ago

@vitaly-t excludeNotExported was also removed I think, as this is now the default behavior.

Error: No entry points provided

You will need to set the entry point for your application e.g. your index.ts or whatever. entry point should be documented in the typedoc docs.

lillallol commented 3 years ago

Note that if I just remove that option from configuration, I start getting this:

Error: No entry points provided

@vitaly-t Read this

Gerrit0 commented 3 years ago

The tag @event is not working after 0.20-beta.24 for methods

Fixed in beta 28, no idea what I was thinking there... For some reason I changed the code so that @event would make methods public.

I tried the 0.20.0-beta.27, I am having some trouble because I use module-alias in my project but the --baseUrl option is disappeared.

baseUrl is a TypeScript option - so long as it is set properly in your tsconfig, TypeDoc should pick it up, if it isn't, please open a new issue to make tracking easier.

Beta 28 is out - I think I finally worked out all the edge cases with inheriting from generic classes. There's not much left on the todo list now - https://github.com/TypeStrong/typedoc/projects/5

Gerrit0 commented 3 years ago

+3 betas.. up to 31 now.

29 - Fixed missing comment detection on some variable-functions #1421. 30 - Normalize unions containing true | false to boolean - this only seems to show up in inferred return types #571. 31 - Fixed the @category tag, it was broken on projects with a single entry point due to categorization happening twice for the project.

I am aiming for the 28th for a full release. The one remaining bug I have to resolve is missing index signatures, after that the remaining tasks are mostly updating documentation.

I did pull one feature out of this release - respecting the OS theme in the default themes. This might still end up getting released this year, but I decided I wanted to fix some other issues with the themes while I'm at it, so am putting it off a bit longer.

Gerrit0 commented 3 years ago

0.20 is out! https://github.com/TypeStrong/typedoc/releases/tag/v0.20.0

Please open a new issue if you discover any issues with it :)

brandones commented 3 years ago

@Gerrit0 Finally getting this integrated into my project. Works like a dream. I don't see one of those "buy me a coffee/beer" buttons anywhere but just want to say you deserve it :beers: Thanks for all your work on this.