Unify docs site solution

[svdimchenko]

Problem statement: currently we must support 2 similar docs sites: readme file in dbt-athena repo and https://github.com/dbt-athena/dbt-athena.github.io, it's not so convenient and may lead to docs inconsistent state.

Possible options may be following:

1. Add docs directory to dbt-athena repo and push changes from there to dbt-athena.github.io on changes
2. Setup docs scraping from dbt-athena.github.io side somehow
3. Setup monorepo for dbt-athena and deprecate dbt-athena.github.io (It may be not the optimal solution due to site deployment extra work needed)

[Jrmyy]

I tend to be in favour of 1 or 3. 3 will be the hardest but the most future-proof solution while 1 is easier to put in place and keeps things ordered and separated. :)

[nicor88]

I believe that 1 and 3 are quite similar. I like the idea suggested on the 1, docs are centrally defined in dbt-athena with a github action that keeps dbt-athena.github.io in sync. Bare in mind that to use github pages you need a repo like dbt-athena.github.io, therefore 3 might not be really possible.

[Jrmyy]

Ok so let's go with solution 1:

Add docs directory to dbt-athena repo and push changes from there to dbt-athena.github.io on changes

[svdimchenko]

here is pipeline example I've just found which is pretty similar to what we need: https://github.com/marcosschroh/dc-avro/blob/master/.github/workflows/docs-publish.yaml

[svdimchenko]

@nicor88 @Jrmyy As we’ve chosen the way to push docs, I see 2 options:

1. commit directly to main branch in docs repo
2. create PR in docs repo and merge it manually for my point of view, the second option is better as it will allow us to deploy pr-review before merge. Once we finalise the vision, I can get it done asap.

[nicor88]

@svdimchenko I believe that option 2 is could be better. Now let's pick up a real case scenario, to highlight what needs to be done:

1. Contributor x open an PR in dbt-athena, where it update the docs (static markdown pages)
2. Thanks to our magic Github action, a pull request will be open in dbt-athena-docs repo
3. Someone review the PR in dbt-athena and ask for a change
4. Contributor x updated the his working branch, docs included
5. Our magic github action, in charge of updating the docs, should now update the PR with the latest docs change for the PR in dbt-athena
6. PR in dbt-athena is approved and merged
7. Some of us must merge also the PR in dbt-athena docs

Does the flow above make sense or I missed something? If so, can our magic docs update github action work as described above?