Closed stijnvanhoey closed 1 year ago
convert_to_source
and not the file name itself cf.vp()
The by default provided _convert_to_source
callback is effectively doing this for each of the file names provided to the vpts-input (as vpts uses a list/generator/... of file names as input, it need to to the translation for each of these files). Hence, by default (when he user is not providing a callable), this _convert_to_source
one is used:
The user can overwrite this default callable (e.g. the service for the aloft S3 bucket is doing this to include the s3 url)
See https://github.com/enram/vpts-csv/pull/42. Apart from removing the unit tests checks for the no-longer-required fields and the lat/lon-switch, the main focus is the 'source_file' field addition.
The adjustments enable to provide a (custom)
source_field
for thevp
andvpts
functionalities (main user API functions):vp
When using the
vp
function without 'source_file' parameter, the file name itself is used as source_file:It is however possible to overwrite the source_file default using either a str or a callable:
This last option is mainly to support custom 'source_file' fields when using
vpts
(and having multiple file paths as input).vpts
BY default it uses the file name itself (see vp section), but the customization using a callable to convert each of the file-paths provides the ability to use custom source_file constructions. By default the conversion to file name is used (default callable), but this can be overwritten, e.g. for the s3-service that runs daily:
For the daily service, also the source and the bucket name are abstracted (coming from the inventory file), but this introduces the implementation which can be used both within as outside the scope of the daily service