Currently we create the storage instance in SDK and use its configuration mechanisms to get the options with env var support. We need to simplify this in the SDK 3, so the handling of env vars should be moved to the storage implementation, somewhere here. Note that the env vars always take precedence.
The goal of this PR is to be able to remove this snippet from the SDK 3 and maintain similar functionality:
prefer APIFY_LOCAL_STORAGE_DIR and APIFY_LOCAL_STORAGE_ENABLE_WAL_MODE env vars if available
APIFY_LOCAL_STORAGE_ENABLE_WAL_MODE is boolean, but env vars are strings, so dont forget to handle falsy values property (see the Configuration class in SDK)
instead of the warning, we should use log.info(`Created a storage folder for your data at ${storageDir}. You can also use APIFY_LOCAL_STORAGE_DIR to override it.`) or something similar
this should be logged only if the directory does not exist
Currently we create the storage instance in SDK and use its configuration mechanisms to get the options with env var support. We need to simplify this in the SDK 3, so the handling of env vars should be moved to the storage implementation, somewhere here. Note that the env vars always take precedence.
The goal of this PR is to be able to remove this snippet from the SDK 3 and maintain similar functionality:
APIFY_LOCAL_STORAGE_DIR
andAPIFY_LOCAL_STORAGE_ENABLE_WAL_MODE
env vars if availableAPIFY_LOCAL_STORAGE_ENABLE_WAL_MODE
is boolean, but env vars are strings, so dont forget to handle falsy values property (see theConfiguration
class in SDK)log.info(`Created a storage folder for your data at ${storageDir}. You can also use APIFY_LOCAL_STORAGE_DIR to override it.`)
or something similar