Rename source column to path to be more consistent with our e.g. dataflatten tables, which also return a path column
Rename source to source_paths for clarity; additionally, make it an array rather than a single string to allow for matching against multiple patterns
Rename query to source_query to disambiguate from the query column we use elsewhere
Rename platform to filter to clarify that we may have other types of filters in the future
Update config structure to be a list of tables instead of a map, so that K2 is able to send down config definitions for all tables instead of filtering by platform; launcher handles the filtering instead
To support the above, move the table name inside the table config
These changes were made in roughly this order; it is possible to review commit-by-commit if that's easier.
This PR makes the following config updates:
source
column topath
to be more consistent with our e.g. dataflatten tables, which also return apath
columnsource
tosource_paths
for clarity; additionally, make it an array rather than a single string to allow for matching against multiple patternsquery
tosource_query
to disambiguate from thequery
column we use elsewhereplatform
tofilter
to clarify that we may have other types of filters in the futureThese changes were made in roughly this order; it is possible to review commit-by-commit if that's easier.