multi-pass generator

[shaunlebron]

I really want the referenced symbols in cljsdoc files to be validated against actual symbols. the original api-docs repo solved this by checking against the canonical symbol list in the separate api-refs repo. but now the repos have merged so that the generated pages have our manual docs.

so, we need a multi-pass system so that the manual docs can depend on the fully generated data as before.

For this, we need preliminary output directories, we'll call .cljs-api-cache/<version>, which will be used to incrementally build up intermediate results for each version before creating repo commits.

maybe we should use cljs-api.no-cljsdoc.edn for data that doesn't have cljsdoc info yet.

• first pass should:
  • determine latest version (check for existing data in output dir)
  • generate catalog data (starting from latest version folder) into .cljs-api-cache/<version>/cljs-api.no-cljsdoc.edn
• second pass
  • should use the most recent list of symbols (containing all to ever exist) to verify existence of symbols referenced by the cljsdoc files
  • create .cljs-api-cache/cljsdoc.edn for caching
  • then for each desired version:
  • read the current .cljs-api-cache/<version>/cljs-api.no-cljsdoc.edn
  • merge data with cljsdoc files, then create .cljs-api-cache/<version>/cljs-api.edn
  • generate the catalog pages (if desired)
• third pass create the catalog repo by creating commits from each version or just dump the latest data into the output directory if no repo is desired

This will also allow a faster CI process (generate all data, but only generate the most recent pages).

Need to add new options to cmd param:

opt	description	e.g.	default
:task	a side task to run instead of the main one	:cljsdoc :docset	nil
:version	version to process	<#> <version> :latest	:latest
:catalog?	create a catalog repo?	true false <#>	false
:skip-pages?	skip page-creation for previous versions?	true false	false
:skip-parse?	skip parsing versions if already cached?	true false	false

[shaunlebron]

working on this here: https://github.com/cljsinfo/cljs-api-docs/tree/multi-pass

[shaunlebron]

removing the single version feature to simplify things. its use-case of fast development is covered by the :skip-parse? and :skip-pages? options.