search

va-big-data-genomics / trellis-mvp-functions

Trellis serverless data management framework for variant calling of VA MVP whole-genome sequencing data.
6 stars 1 forks source link

Cloud Build is failing but build details show "Successful" #33

Closed pbilling closed 1 year ago

pbilling commented 2 years ago

From the Build log:

ERROR: (gcloud.beta.functions.deploy) OperationError: code=3, message=Function failed on loading user code. This is likely due to a bug in the user code. Error message: Error: please examine your function logs to see the error cause

But when I click on the link to the Cloud Build logs it shows Successful.

pbilling commented 2 years ago

I was defining a global variable using using os.environ and that value which should have been defined in the "Substitution variables" section of the Cloud Build trigger configuration for this function, but was not. I think this could have been causing the issue.

pbilling commented 1 year ago

Also, the variable was _ENABLE_JOB_LAUNCH, which you didn't fix, but it's fine. Thrilled to have been able to successfully reproduce this issue.

pbilling commented 1 year ago

Updating the environment variables didn't resolve the issue. I updated to a recommended build image (gcloud => gcr.io/google.com/cloudsdktool/cloud-sdk) and got an updated error message that recommended checking the function logs. Function logs revealed a "Module not found" error: I hadn't added "trellisdata" to the requirements.txt file.

pbilling commented 1 year ago

Follow-up error seemingly from trying to import pubsub library.

Descriptors cannot not be created directly. If this call came from a _pb2.py file, your generated code is out of date and must be regenerated with protoc >= 3.19.0. If you cannot immediately regenerate your protos, some other possible workarounds are: 1. Downgrade the protobuf package to 3.20.x or lower. 2. Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python (but this will use pure-Python parsing and will be much slower). More information: https://developers.google.com/protocol-buffers/docs/news/2022-05-06#python-updates

I had been using the '==' operator to specify specific versions of dependencies in the requirements.txt file; going to change to the '>=' for all reqs and see if that resolves the issue.

pbilling commented 1 year ago

Update: It did 🙂 .