By using either command line replacement pairs or specifying a file with json defintions of replacements an arbitrary regexp ("pattern") can be detected and replaced with another string, including expanding captured groups in the pattern.
The replacement phase is taking place just before upsert, so all other textual manipulations are done by that time.
Replacements happen in a deterministic sequence. There are ample opportunities to get unexpected (but logically consistent) results by inadvertently result of a previous replacement.
The environment block is optional and used for very dynamic replacements. By specifying a python source file, it will be dynamically imported at run time. The new_value field can then specify a <module>.<func> that returns a string value. As an example, the following adds a replacement of "TODAY" to an iso-formatted datetime.
By using either command line replacement pairs or specifying a file with json defintions of replacements an arbitrary regexp ("pattern") can be detected and replaced with another string, including expanding captured groups in the pattern.
The replacement phase is taking place just before upsert, so all other textual manipulations are done by that time.
Replacements happen in a deterministic sequence. There are ample opportunities to get unexpected (but logically consistent) results by inadvertently result of a previous replacement.
Format of json file:
The
environment
block is optional and used for very dynamic replacements. By specifying a python source file, it will be dynamically imported at run time. Thenew_value
field can then specify a<module>.<func>
that returns a string value. As an example, the following adds a replacement of "TODAY" to an iso-formatted datetime.Funcs.py:
The parameter
term
is a Match object as per using https://docs.python.org/3/library/re.html#re.subn.