BotoAdapter was starting to contain to much business logic, and so the opportunity was taken to refactor: a StorageService now sits between the SparkJobService and the BotoAdapter.
The StorageService creates the designated bucket if it does not already exist.
Due to issues with conflicting key sets on the Kubernetes master node, the access key is logged when run_piezo.py initialises the application. The secret key is not logged for obvious reasons.
System tests still run fine, except for the first checking Grafana. This is unrelated to the changes made here.
Stories covered: https://github.com/ukaea/piezo/issues/115
S3BucketNameis now a configuration item.BotoAdapterwas starting to contain to much business logic, and so the opportunity was taken to refactor: aStorageServicenow sits between theSparkJobServiceand theBotoAdapter.The
StorageServicecreates the designated bucket if it does not already exist.Due to issues with conflicting key sets on the Kubernetes master node, the access key is logged when
run_piezo.pyinitialises the application. The secret key is not logged for obvious reasons.System tests still run fine, except for the first checking Grafana. This is unrelated to the changes made here.