search

WordPress / openverse

Openverse is a search engine for openly-licensed media. This monorepo includes all application code.
https://openverse.org
MIT License
218 stars 177 forks source link

Document Airflow variables #1251

Open sarayourfriend opened 1 year ago

sarayourfriend commented 1 year ago

Problem

@stacimc reports that the variables needed by Airflow are not sufficiently documented.

Description

A lot of these, we should audit the list to make sure we don't miss any. SILENCED_SLACK_NOTIFICATIONS, SKIPPED_INGESTION_ERRORS, INGESTION_LIMIT...

@stacimc Can you describe more about where this documentation should live? Does it need to live in code, somewhere in Airflow, or should it go in the documentation site?

Alternatives

Additional context

stacimc commented 1 year ago

These should be documented in the documentation site. Many of these, along with DagRun conf options, are utilities used for testing in the Catalog and should probably live in something like documentation/guides/catalog/test.md