Open conreaux opened 1 year ago
The error message seems to be coming from the constructor of System.Uri
. Can you try changing your WEBSITE_HOSTNAME
environment variable from localhost:80
to http://localhost:80
?
Same error. Here's the output of printenv
:
PS C:\Users\conreaux\source\repos\durabletask-mssql\test\PerformanceTests> kubectl exec durabletask-mssql-app-http-5c74887ffb-gbt57 -- printenv
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
HOSTNAME=durabletask-mssql-app-http-5c74887ffb-gbt57
AzureWebJobsKubernetesSecretName=secrets/func-keys-kube-secret-durabletask-mssql-app
WEBSITE_HOSTNAME=http://localhost:80
AzureFunctionsJobHost__functions__4=StartManySequences
AzureFunctionsJobHost__functions__1=StartLongHaul
AzureFunctionsJobHost__functions__2=StartManyEntities
AzureFunctionsJobHost__functions__3=StartManyMixedOrchestrations
AzureWebJobsSecretStorageType=kubernetes
SQLDB_Connection=Server=mssqlinst.mssql.svc.cluster.local;Database=DurableDB;User ID=sa;Password=Pass@word1;Persist Security Info=False;TrustServerCertificate=True;Encrypt=True;
AzureFunctionsJobHost__functions__0=PurgeOrchestrationData
KUBERNETES_SERVICE_PORT=443
KUBERNETES_PORT_443_TCP_ADDR=10.96.0.1
DURABLETASK_MSSQL_APP_HTTP_SERVICE_PORT=80
DURABLETASK_MSSQL_APP_HTTP_SERVICE_HOST=10.109.48.116
DURABLETASK_MSSQL_APP_HTTP_PORT=tcp://10.109.48.116:80
DURABLETASK_MSSQL_APP_HTTP_PORT_80_TCP=tcp://10.109.48.116:80
DURABLETASK_MSSQL_APP_HTTP_PORT_80_TCP_PORT=80
KUBERNETES_SERVICE_HOST=10.96.0.1
KUBERNETES_PORT=tcp://10.96.0.1:443
KUBERNETES_PORT_443_TCP=tcp://10.96.0.1:443
KUBERNETES_PORT_443_TCP_PORT=443
KUBERNETES_SERVICE_PORT_HTTPS=443
KUBERNETES_PORT_443_TCP_PROTO=tcp
DURABLETASK_MSSQL_APP_HTTP_PORT_80_TCP_PROTO=tcp
DURABLETASK_MSSQL_APP_HTTP_PORT_80_TCP_ADDR=10.109.48.116
ASPNETCORE_URLS=http://+:80
DOTNET_RUNNING_IN_CONTAINER=true
AzureWebJobsScriptRoot=/home/site/wwwroot
HOME=/home
FUNCTIONS_WORKER_RUNTIME=dotnet
DOTNET_USE_POLLING_FILE_WATCHER=true
HOST_VERSION=4.14.0
ASPNETCORE_CONTENTROOT=/azure-functions-host
AzureFunctionsJobHost__Logging__Console__IsEnabled=true
Oh, I was focused on this error message:
Invalid URI: The hostname could not be parsed.
But after re-reading your original post, I'm realizing that the more important error is this one:
System.InvalidOperationException: Unable to load metadata for function 'HelloCities'.
My understanding from speaking with other members of the Functions team is that there seems to be an issue with the Functions runtime where the splitting of HTTP and non-HTTP functions doesn't work correctly in some cases, causing orchestrations to get scheduled on the wrong pod. I've seen this issue occasionally, but not consistently (others have reported that its 100% reproducible).
I think one way to fix this is to ensure that the durabletask-mssql-app-http-*
pod has the orchestration and activity trigger functions enabled. I believe the AzureFunctionsJobHost__functions__N
environment variables control this, though I'm not completely confident about that. If you have the patience to do so, it might be worth experimenting with.
Ultimately, it seems like the func kubernetes deploy
mechanism is no longer working reliably. I'll probably need to look into changing the quick start instructions to do something more manual until the underlying Functions runtime issue gets resolved (and I don't know if there's any ETA on that).
I was able to root cause the URI exception and created an issue in the azure-functions-core-tools repo here.
For the other exception, it appears that the runtime doesn't respect the allow list controlled by the AzureFunctionsJobHost__functions__N
environment variables and will still attempt to schedule orchestrations on the HTTP pod.
By using func kubernetes deploy
with the --dry-run
option, I was able to dump the deployment to a YAML file.
It does appear that func kubernetes deploy
is working as intended since it produces the expected list of AzureFunctionsJobHost__functions__N
variables for the HTTP and non-HTTP deployments.
I also confirmed that manually adding AzureFunctionsJobHost__functions__N
variables to the deployment YAML to enable the orchestration/activity functions on the HTTP pod does eliminate the error. However, I don't believe this is a desirable solution.
Another workaround is to separate the HTTP and non-HTTP functions into separate projects each with its own Dockerfile, and therefore distinct images for client and server. For this to work, I also needed to set the ExternalClient
property to true
on the DurableClient
attribute in the client function.
While attempting the Kubernetes Quickstart, I've encountered the following error when deploying to a local k8s cluster using Docker Desktop for Windows. I had seen an SO thread about a possibly missing
WEBSITE_HOSTNAME
env var, but adding it aslocalhost:80
inmsssql-secrets.yml
did not resolve the issue.The
durabletask-mssql-app-http-*
pod starts, and I'm able to POST to theStartManySequences
endpoint, but get subsequent errors. I assume not everything has instantiated correctly due to the prior error. Any suggestions?