microsoft / durabletask-mssql

Microsoft SQL storage provider for Durable Functions and the Durable Task Framework
MIT License
87 stars 32 forks source link

Sync AKS ScaledObject with host.json #42

Closed marcd123 closed 3 years ago

marcd123 commented 3 years ago

I have the following configuration in my host.json file to control my Function App's maxConcurrentOrchestratorFunctions and maxConcurrentActivityFunctions limits:

"extensions": {
    "durableTask": {
      "maxConcurrentOrchestratorFunctions": 2,
      "maxConcurrentActivityFunctions": 2,
      "storageProvider": {
        "type": "mssql",
        "connectionStringName": "SQLDB_Connection",
        "taskEventLockTimeout": "00:02:00",
        "partitionCount": 3
      }
    }
  }

When I generate my deployment using func kubernetes deploy it creates a ScaledObject for me with the GetScaleRecommendation function setup, however the function parameters are out of sync with my host.json file.

I would expect the parameters to line up with my host.json settings like SELECT dt.GetScaleRecommendation(2, 2) though instead I get this ScaledObject configuration:

apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name: bat-artwork-review-api
  labels: {}
spec:
  scaleTargetRef:
    name: bat-artwork-review-api
  triggers:
  - type: mssql
    metadata:
      query: SELECT dt.GetScaleRecommendation(10, 1)
      targetValue: "1"
      connectionStringFromEnv: SQLDB_Connection

Is there any way to sync these parameters currently?

marcd123 commented 3 years ago

Is there any solution for this?

Our current solution is to do a deployment using func kubernetes deploy which deploys the ScaledObject above with these settings SELECT dt.GetScaleRecommendation(10, 1)

Then we manually deploy another ScaledObject with the same name but different settings to overwrite the previous one:

apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name: bat-artwork-review-api
  labels: {}
spec:
  scaleTargetRef:
    name: bat-artwork-review-api
  triggers:
  - type: mssql
    metadata:
      query: SELECT dt.GetScaleRecommendation(2, 2)
      targetValue: "1"
      connectionStringFromEnv: SQLDB_Connection
cgillum commented 3 years ago

Just noting that the place where this needs to be fixed is here. This code already has the host.json data available. It just needs to be parsed out and used to update the query with the right values. I added the good first issue label to this issue if anyone is feeling adventurous enough to contribute a fix. :)

marcd123 commented 3 years ago

@cgillum Thanks! This would certainly simplify our CI/CD setup. Currently we have a static ScaledObject YAML file that we use to overwrite the one generated by func CLI

wsugarman commented 3 years ago

@cgillum I've created a PR over in the core tools repository: https://github.com/Azure/azure-functions-core-tools/pull/2771. Please let me know if it looks reasonable haha

wsugarman commented 3 years ago

@cgillum / @marcd123 - All of the relevant PRs have been checked in for the tooling, so I think we're good to close this issue.