Open connor-napster opened 2 years ago
Couple things to try;
Can you specify the dir
as cf-template
rather than ./cf-template
? From the field's documentation I see This must be a relative path. If a step's dir is specified and is an absolute path, this value is ignored for that step's execution. eg. helloworld (no leading slash allowed)
. While you haven't specified an absolute path, I could see the API hitting a false positive here (and it should be pretty quick for you to verify!). Silently ignoring fields is... pretty frustrating.
Otherwise, can you collect debug logs, sanitize them, and share them? Folks will typically use a gist. If you're concerned about private information in 'em, let me know and we can find a secure way to share them. (Also, specifying a target will help keep the log size manageable if this is part of an existing config)
Couple things to try;
- Can you specify the
dir
ascf-template
rather than./cf-template
? From the field's documentation I seeThis must be a relative path. If a step's dir is specified and is an absolute path, this value is ignored for that step's execution. eg. helloworld (no leading slash allowed)
. While you haven't specified an absolute path, I could see the API hitting a false positive here (and it should be pretty quick for you to verify!). Silently ignoring fields is... pretty frustrating.- Otherwise, can you collect debug logs, sanitize them, and share them? Folks will typically use a gist. If you're concerned about private information in 'em, let me know and we can find a secure way to share them. (Also, specifying a target will help keep the log size manageable if this is part of an existing config)
Hi Riley
Apologies I copied the wrong snippet. I actually specified cf-template
to begin with and it didn't work.
The build logs are not very useful, do you have a working snippet at all I could use to help diagnose the issue?
-----------------------------------------------------: timestamp=2022-04-06T13:47:20.299+0100
2022-04-06T13:47:20.974+0100 [INFO] plugin.terraform-provider-google-beta_v4.15.0_x5: 2022/04/06 13:47:20 [DEBUG] Google API Response Details:
---[ RESPONSE ]--------------------------------------
HTTP/2.0 200 OK
Alt-Svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"
Cache-Control: private
Content-Type: application/json; charset=UTF-8
Date: Wed, 06 Apr 2022 12:47:20 GMT
Server: ESF
Vary: Origin
Vary: X-Origin
Vary: Referer
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
X-Xss-Protection: 0
{
"name": "projects/peerless-aria-310018/locations/us-central1/operations/operation-1649249196295-5dbfbc07dc399-eadc4fa7-64ddcc31",
"metadata": {
"@type": "type.googleapis.com/google.cloud.functions.v2beta.OperationMetadata",
"createTime": "2022-04-06T12:46:36.573505067Z",
"endTime": "2022-04-06T12:47:11.430250693Z",
"target": "projects/peerless-aria-310018/locations/us-central1/functions/test-function-2",
"verb": "create",
"cancelRequested": false,
"apiVersion": "v2beta",
"requestResource": {
"@type": "type.googleapis.com/google.cloud.functions.v2beta.Function",
"name": "projects/peerless-aria-310018/locations/us-central1/functions/test-function-2",
"description": "a new function",
"buildConfig": {
"runtime": "python39",
"entryPoint": "run",
"source": {
"repoSource": {
"projectId": "peerless-aria-310018",
"repoName": "github_rhapsody_data-applications",
"branchName": "development",
"dir": "cf-template"
}
}
},
"serviceConfig": {
"service": "projects/peerless-aria-310018/locations/us-central1/services/test-function-2",
"timeoutSeconds": 60,
"maxInstanceCount": 1,
"ingressSettings": "ALLOW_ALL",
"serviceAccountEmail": "938864285604-compute@developer.gserviceaccount.com",
"availableMemory": "256M",
"allTrafficOnLatestRevision": true
},
"environment": "GEN_2"
},
"stages": [
{
"name": "ARTIFACT_REGISTRY",
"message": "Deleting function artifacts in Artifact Registry",
"state": "COMPLETE"
},
{
"name": "SERVICE",
"message": "Deleting Cloud Run service",
"state": "COMPLETE",
"resource": "projects/peerless-aria-310018/locations/us-central1/services/test-function-2"
}
]
},
"done": true,
"error": {
"code": 3,
"message": "Build failed with status: FAILURE and message: missing main.py and GOOGLE_FUNCTION_SOURCE not specified. Either create the function in main.py or specify GOOGLE_FUNCTION_SOURCE to point to the file that contains the function. For more details see the logs at https://console.cloud.google.com/cloud-build/builds;region=us-central1/d254d0cf-666f-4625-9797-212ebf9153fc?project=938864285604."
}
}
We don't happen to have a test for repo mode, unfortunately- it's hard to set up + tear down one of 'em in a test.
@k-ezis: If you have a few space cycles, do you spot anything off in that response? Given the API silently rejects one field, I'm a little worried a value in another may be getting silently rejected.
@rileykarson understood, just was not sure if you had a working example
@rileykarson or @k-ezis any update on this?
The official Google documentation says "Deploying from Cloud Source Repositories is not currently supported in Cloud Functions (2nd gen)." So how was it supposed to work in the first place?
They have also specified that we have to use google-cloud-beta
as a provider, deployment fails using any of the google-cloud
providers
@burnier is correct, Gen2 currently doesn't support deploying from a Cloud Source Repository (docs: https://cloud.google.com/functions/docs/deploy#from-source-repo see note) For Gen2 you'd have to use a source bucket.
Ideally we'd have omitted the Cloud Source Repositories related fields and added them in later, in order to avoid confusion! Normally we'd consider deprecation for fields that don't currently work, but it sounds like these will start working the future- we could consider amending the docs to state that the feature is currently unavailalable.
The end-points are (partially) implemented from an API perspective, which is why it was written into the configuration. I'd be good with commenting that functionality out if we think that would clarify this issue. We are doing 2023 planning currently, so the roadmap towards this feature is uncertain at the moment.
It is pretty sad this wasn't documented because I just rewrote my module to use this after the archive way of handling it failed to work idempotently. I guess API in this case stands for Absolutely Positively Inoperable. :p
Hope it is added soon since it would be a much cleaner solution than a zip file in a bucket.
That google doc comment is odd, because deploying gen2 cloud functions with gcloud functions deploy
works just fine with --source=<repo_url>
and from the UI/web page.
@sharkymcdongles :- I was able to move the goal posts a bit by passing in the short name of the repo instead of the long qualified name. I don't get the invalid argument error now, though I'm not totally out of the woods yet as I think there might still be an issue with getting things properly staged/deployed -- the Cloud Run container for the CF has issues:
Ready condition status changed to False for Revision test-function2-00001-put with message: The user-provided container failed to start and listen on the port defined provided by the PORT=8080 environment variable. Logs for this revision might contain more information.
preceded by:
Provided module can't be loaded. Is there a syntax error in your code? Detailed stack trace: TypeError: Cannot read properties of undefined (reading 'replace') at formatLogName (/workspace/node_modules/@google-cloud/logging/build/src/utils/log-common.js:79:17) ... at Module.require (node:internal/modules/cjs/loader:1057:19) Could not load the function, shutting down. Container called exit(1).
which I suspect is maybe the result of source code not being pulled in, even though the create payloads between terraform and gcloud look identical if I pass the repo short name in:
gcloud functions deploy:
POST /v2/projects/<external>/locations/us-east4/functions?alt=json&functionId=gc-deploy-test2
{
"buildConfig": {
"entryPoint": "scale",
"environmentVariables": {},
"runtime": "nodejs16",
"source": {
"repoSource": {
"branchName": "master",
"projectId": <external>,
"repoName": "gcp-cf-webappserver-scaler"
}
}
},
"labels": {
"deployment-tool": "cli-gcloud"
},
"name": "projects/<external>/locations/us-east4/functions/gc-deploy-test2",
"serviceConfig": {
"environmentVariables": {
"GCP_PROJECT": <external>,
"GCP_REGION": "us-east4",
"GCP_VM_ZONE": "us-east4-a",
"LOG_NAME": "gcp-scaler"
},
"ingressSettings": "ALLOW_ALL",
"secretEnvironmentVariables": [
{
"key": "DB_SECRET",
"projectId": <internal>,
"secret": <secret>,
"version": "latest"
}
],
"serviceAccountEmail": <svc acct>,
"vpcConnector": <vpc connector>,
"vpcConnectorEgressSettings": "ALL_TRAFFIC"
}
}
vs.
terraform:
POST /v2/projects/<external>/locations/us-east4/functions?alt=json&functionId=tf-deploy-test1
{
"buildConfig": {
"entryPoint": "scale",
"runtime": "nodejs16",
"source": {
"repoSource": {
"branchName": "master",
"projectId": <external>,
"repoName": "gcp-cf-webappserver-scaler"
}
}
},
"description": "testing terraform deployment of cloud function",
"name": "projects/<external>/locations/us-east4/functions/tf-deploy-test1",
"serviceConfig": {
"allTrafficOnLatestRevision": true,
"environmentVariables": {
"GCP_LOG_NAME": "gcp-was-scaling",
"GCP_PROJECT": <external>,
"GCP_REGION": "us-east4",
"GCP_VM_ZONE": "us-east4-a"
},
"ingressSettings": "ALLOW_ALL",
"secretEnvironmentVariables": [
{
"key": "DB_SECRET",
"projectId": <external>,
"secret": <secret>,
"version": "latest"
}
],
"serviceAccountEmail": <svc acct>,
"vpcConnector": <vpc connector>,
"vpcConnectorEgressSettings": "ALL_TRAFFIC"
}
}
so I don't know why there would be a difference in behavior (though I did create a unique user for terraform, so maybe there is a permission breakdown that isn't readily apparent).
anyway, wanted to offer the advice of trying the short name to see if things work out better for you.
I encountered the same issue. However, I am able to successfully deploy it after moving main.py
and requirements.txt
to the root directory of the repository (Cloud Source Repository).
This is my folder structure when error occurs:
/cloud_functions/main.py
/cloud_functions/requirements.txt
This arrangement allows the build to complete and the function works as intended, despite dir
is set as cloud_functions
.
/main.py
/requirements.txt
The partial debug log is as follows:
POST /v2/projects/<project_id>/locations/asia-southeast1/functions?alt=json&functionId=<function_id> HTTP/1.1
Host: cloudfunctions.googleapis.com
User-Agent: Terraform/1.3.6 (+https://www.terraform.io) Terraform-Plugin-SDK/2.10.1 terraform-provider-google/dev
Content-Length: 1128
Content-Type: application/json
Accept-Encoding: gzip
{
"buildConfig": {
"entryPoint": "object_finalize",
"environmentVariables": {
"BUILD_CONFIG_TEST": "build_test"
},
"runtime": "python310",
"source": {
"repoSource": {
"branchName": "main",
"dir": "cloud_functions",
"projectId": "<project_id>",
"repoName": "sample_repo"
}
}
},
...
}
@kylase "dir" parameter doesn't work at all
@kylase
"dir" parameter doesn't work at all
Yes, I was pointing out that it doesn't work if we are using monorepo, unless we can put main.py
at the root.
Note: marking as very large because the situation appears fairly complicated w/ the resource
I discussed this with an API team member and support is blocked on an API change. No known ETA at the moment, but marking upstream until further news.
"dir" parameter doesn't work at all
This is also true for Node.js runtimes. Hope they can grab some resource and get it to work π .
Description
It seems its not possible to deploy cloud functions gen 2 using source repositories, there is an error. I am able to deploy a cloud function gen1 using the source repository link but not gen2
Community Note
modular-magician
user, it is either in the process of being autogenerated, or is planned to be autogenerated soon. If an issue is assigned to a user, that user is claiming responsibility for the issue. If an issue is assigned tohashibot
, a community member has claimed the issue already.Terraform Version
Affected Resource(s)
Terraform Configuration Files
Debug Output
Panic Output
Expected Behavior
Function should deploy as normal
Actual Behavior
Error when deploying cloud function
Steps to Reproduce
terraform apply
Important Factoids
References
0000
b/308248099