add package to auto-ingress reference readmes

[krisbitney]

I added two scripts. One clones repos. The other copies readmes from the clone directories, adds docusaurus headers, places them in the reference doc folders.

To add new repos to clone, you can update this:

type Repo = {
  org?: string; // defaults to "polywrap"
  name: string;
  branch: string;
}

const repos: Repo[] = [
  {
    name: "toolchain",
    branch: "origin"
  }
]

To add new readme -> doc transformations, you can update this:

// paths are relative to clone and reference doc dirs
const readmeToDocPaths: Record<string, string> = {
  "./polywrap/toolchain/packages/js/client/README.md": "./clients/js/client-js.md",
  "./polywrap/toolchain/packages/cli/README.md": "./cli/polywrap-cli.md"
}

Feedback welcome!

Closes https://github.com/polywrap/documentation/issues/230

[pileks]

One thing that came up while I was working on doc-snippets is that you could use glob patterns instead of fixed filenames, folders, etc. within readme-to-doc.

Take a look at the searchFiles function within doc-snippets Combine that with the lines 22-31 in combine.ts and you've got yourself a very configurable ingress pipeline! 😄

If things ever start getting more complicated than your current use case, you can simply reuse these.

[dOrgJelli]

Hey @krisbitney, this looks great! As for this:

The related issue mentions the possibility of copying the contents of .d.ts files into readme files as well, but this PR doesn't do that.

I don't think this will be necessary. These types can simply be listed within the package READMEs using the "doc-snippets" injection tool Jure made. This way it can work for all languages, and we don't have to build this type of functionality into the "readme ingress" tool.

[dOrgJelli]

Also a quick note on the developer experience of the tool, I don't think we should be doing ../toolchain assuming the toolchain repo is pre-cloned. I think the documentation repo should have the cloning of the various repos it's using for the docs apart of the setup process. This can be done in another small script inside of the "scripts/" folder.

Also one last note on "when to put something in scripts/ vs a separately managed project", I think a simple question we can ask is "will we use this script/tool in more than one project?"

For the doc-snippets tool the answer was "yes." because we wanted to use it for the docs & for package READMEs we publish. I'm not sure this is the case with "readme to docs" so we can eliminate some friction by just having it as a script.

[krisbitney]

All of this sounds good. I'll change this to a script that clones the repos and uses a mapping to automate the process better.

[krisbitney]

I replaced the cli tool with scripts and updated the PR description. Please read it and let me know what you think!

[krisbitney]

This looks good to me! When are we planning to hook this up to the build command? Will there be a follow up PR where we apply these READMEs to the docs?

We can do it now. The client and cli READMEs are already in the docs. The READMEs for other packages haven't been updated with full reference doc information yet in the current polywrap release (0.9.4), but I will connect them anyway because most of them provide useful information.