Please describe the feature you'd like to see
Request from Steven to add https://forum.astronomer.io/ as a data source.
Describe the solution you'd like
Add forum docs for ingest. Some older posts are inaccurate so a "recent" (ie. last 2 years) cutoff date will be needed. Also we would want to check tags for "software" and decide whether we want to ingest those.
[ ] Integration tests (if the feature relates to a new database or external service)
[ ] Example DAG
[ ] Docstrings in reStructuredText for each of methods, classes, functions and module-level attributes (including Example DAG on how it should be used)
[ ] Exception handling in case of errors
[ ] Logging (are we exposing useful information to the user? e.g. source and destination)
[ ] Improve the documentation (README, Sphinx, and any other relevant)
Please describe the feature you'd like to see Request from Steven to add https://forum.astronomer.io/ as a data source.
Describe the solution you'd like Add forum docs for ingest. Some older posts are inaccurate so a "recent" (ie. last 2 years) cutoff date will be needed. Also we would want to check tags for "software" and decide whether we want to ingest those.
Are there any alternatives to this feature? No
Additional context Forum is powered by Discourse. https://meta.discourse.org/t/does-discourse-support-export-conversations-as-an-organized-bulk-of-data/180537
Acceptance Criteria