icenet-ai / icenet-etl

Infrastructure for storing IceNet predictions and importing them into a database
MIT License
1 stars 0 forks source link

Decouple application code from etl repository #40

Closed JimCircadian closed 10 months ago

JimCircadian commented 12 months ago

This will ultimately serve us well to decouple azure function app implementations from the infrastructure code, allowing devops services in azure to deploy from repositories when the perimeter is no longer directly accessible.

JimCircadian commented 11 months ago

For the apps in the geoapi and processor modules, local-exec of the az agent mean that decoupling is more complicated, as that invokes the build process within the container which we rely on. However, hopefully in all cases this can be farmed out to an action in the individual repositories. If so, we stand a much better chance of isolating the application deployments and thus securing the endpoints prior to needing to deploy, which can then be done via a bastion or VPN to satisfy #37

JimCircadian commented 11 months ago

Additional deployment over the top of a docker image doesn't seem to work, so abandoning that. The docker image was used thanks to the size of tensorflow binary setup and which wasn't working well with the oryx build process, so sticking with docker for the moment.

func azure functionapp publish [[REDACTED]] --python
Your functionapp is using a custom image DOCKER|registry.hub.docker.com/jimcircadian/iceneteventprocessor:latest.
Assuming that the image contains the correct framework.

Getting site publishing info...
Updating Application Settings for Remote build...
Timed out waiting for SCM to update the Environment Settings
JimCircadian commented 10 months ago

Sorted out a new repository for the geoapi-processing

JimCircadian commented 10 months ago

Confirmed that the new geoapi deployment from icenet-geoapi works using the secrets file output from terraform