migrate dev docs to in-repo markdown

[josh-chamberlain]

Context

The gitbook docs at https://docs.pdap.io should be for people using PDAP as a software/service. Everything to do with the code should be in the README, /docs folders and markdown files in the respective repo. It's all already markdown, but might need some light formatting.

Plus, it's nice to be able to change the code + update the docs at the same time.

Requirements

• add a /docs folder to the data-sources-app repo
• move from this repo to there:
  • [ ] API docs
    • consider using https://github.com/tmcw/docbox or another open-source alternative
    • link to API docs from the gitbook docs left bar
  • [ ] hidden properties
  • [ ] data dictionaries (link to data dictionaries from side bar of docs)
    • we could remove any extraneous text to make it a single machine readable CSV or JSON file—it could even serve as documentation and source of truth for the app at the same time.
  • [ ] move Record types taxonomy to Terms & Definitions, delete the now-empty folder

Docs

lol

[maxachis]

@josh-chamberlain I've created two complimentary draft requests here:

1. 14

2. Migrate relevant documentation to repository from docs repository.: #258 in data-sources-app:

Remaining Questions

1. When you state "link to API docs from the gitbook docs left bar" and "link to data dictionaries from side bar of docs", you're discussing modifying the links in SUMMARY.md, correct?
2. What are the hidden properties in question that need to be migrated?
3. I'm unfamiliar with Docbox and would need more context as to why and how we would use it here; it may also be relevant to have as a follow-up issue in data-sources-app once the documents are moved, rather than within the same issue.

[josh-chamberlain]

@maxachis

1. Yes! they show up on the left bar here: https://docs.pdap.io

2. you moved them already, it's the hidden properties markdown file

3. docbox was suggested because API docs can be so complex and tedious to generate, yet they are repeatable in structure. in the screenshot attached, you can see that gitbook is doing some extra work to render tabs and things that are not supported in github flavored markdown. this is the source of the screenshot below. API docs are often hosted separately because of their specific requirements. The ideal case is that we have API docs generated from the app repo, so that PRs which change the API can change the docs at the same time. I'd rather not merge until they're rendering properly.

[maxachis]

So at least looking at the Python end, the best solution for automatic API documentation appears to be using Flask-RESTX. This is a popular module that provides decorators and other tools to describe the API and expose its documentation automatically.

The catch is that we will need to swap out uses of Flask's Api and Resource classes with flask_restx's implementations of the same class. Then, we'd need to add special decorators unique to the module. In theory, not too much, and once we do that, we can put the documentation right there in the code and see it replicate.

There may be a complicating factor with the Vue side of things, but I'm not sure yet. I may create a small sample Flask-RestX test repository on my own account to make sure I understand its functionality. I'd also want to keep @mbodeantor and @bonjarlow apprised of this, since these changes would affect code they've created.

[mbodeantor]

@josh-chamberlain This work could probably wait until we're further with V2 right?

[josh-chamberlain]

@mbodeantor yeah, I suppose this work seems much less urgent now that there's v2 stuff to do. As part of the v2 stuff we're going to be making a lot of API changes, but I guess it's about the same amount of work either way.

@maxachis I am open to Flask-RESTX and in-code documentation—are you good to wait on this a bit?

[maxachis]

@mbodeantor yeah, I suppose this work seems much less urgent now that there's v2 stuff to do. As part of the v2 stuff we're going to be making a lot of API changes, but I guess it's about the same amount of work either way.

@maxachis I am open to Flask-RESTX and in-code documentation—are you good to wait on this a bit?

I can, indeed! But there may be a third option!

I may be able to incrementally replace some of our components with flask_RESTx as I develop for v2. Flask-RESTx is a fork of Flask-RESTful and is designed to maintain backward compatibility -- it just allows for auto-generated Swagger documentation. So I may be able to swap out the flask_restful version of App with flask_restx's version, and do the same for any newResource` classes I instantiate. It may be that simple. If not, it'd be able to be identified when testing the PR, and I could easily swap them back.

The risk is this is a little extra effort with a chance of it coming to nothing (but not much). But if successful, it may reduce redundancy by not requiring us to change that documentation to a different schema later on.

[josh-chamberlain]

@maxachis It seems like more moving parts to me, but you'd be the one primarily managing that complexity—I don't really want to have API docs in two places. I wouldn't say starting this now is a good idea, but when we're further into v2 it might make a lot of sense