Unable to create Cloud Run Job resource - 404 Requested entity was not found

[1oglop1]

What happened?

I'm trying to deploy Cloud Run Job (currently in public preview) but I keep getting 404s.

Steps to reproduce

Create a ts project wi

import * as gcp from "@pulumi/google-native"

const region = "europe-west3"
const project = "my-gcp-project"

const provider = new gcp.Provider(
  "jan-provider",
  {
    project: project,
    region: region, // Frankfurt, Germany
  }
)

const reg = new gcp.artifactregistry.v1.Repository(
  "repo",
  {
    description: "Hello repo world",
    format: "DOCKER",
    repositoryId: "rid"
  },
  {provider}
)

const job = new gcp.run.v2.Job(
  "job",
  {
    jobId: 'myjob',
    template: {
      taskCount: 1,
      template: {
        maxRetries: 0,
        timeout: "600s",
        containers: [
          {
            image: "us-docker.pkg.dev/cloudrun/container/job:latest",
          }
        ]
      }
    },
  },
  {provider}
)

Expected Behavior

everything is deployed

Actual Behavior

Only artifactregistry is deployed correctly.

Error:

google-native:run/v2:Job (job):
error: error sending request: googleapi: Error 404: Requested entity was not found.:
 "https://run.googleapis.com/v2/projects/my-gcp-project/locations/europe-west3/jobs?jobId=myjob" 
 map[__autonamed:true jobId:myjob location:europe-west3 name:projects/my-gcp-project/locations/europe-west3/jobs/job-570f641 
 project:my-gcp-project template:map[taskCount:1 template:map[containers:[map[image:us-docker.pkg.dev/cloudrun/container/job:latest]] maxRetries:0 timeout:600s]]]

Output of pulumi about

CLI          
Version      3.36.0
Go Version   go1.17.11
Go Compiler  gc

Plugins
NAME    VERSION
nodejs  unknown

Host     
OS       darwin
Version  12.5
Arch     arm64

This project is written in nodejs: executable='/Users/user/.asdf/shims/node' version='v18.0.0'

Current Stack: 1oglop1/dev

TYPE                                          URN
pulumi:pulumi:Stack                           urn:pulumi:dev::analytics::pulumi:pulumi:Stack::analytics-dev
pulumi:providers:google-native                urn:pulumi:dev::analytics::pulumi:providers:google-native::user-provider
google-native:artifactregistry/v1:Repository  urn:pulumi:dev::analytics::google-native:artifactregistry/v1:Repository::repo

Found no pending operations associated with 1oglop1/dev

Backend        
Name           pulumi.com
URL            https://app.pulumi.com/1oglop1
User           1oglop1
Organizations  1oglop1

Pulumi locates its logs in /var/folders/79/bvt24ns10_n58z24g_klmhmm0000gn/T/ by default
warning: Failed to get information about the Pulumi program's plugins: Could not find either /Users/user/work/bluecode/pulumi-utils/pulumi/projects/analytics/yarn.lock or /Users/user/work/bluecode/pulumi-utils/pulumi/projects/analytics/package-lock.json

Additional context

No response

Contributing

Vote on this issue by adding a 👍 reaction. To contribute a fix for this issue, leave a comment (and link to your pull request, if you've opened one already).

[viveklak]

Thanks for opening the issue @1oglop1. I am struggling to understand why the api is throwing the above error. This is a resource the provider is erroneously trying to set the autoname for which I have disabled locally. Yet I continue to see the same error as you. The Google native provider is correctly following the API specification for this resource outside of the above issue. Even running a simple example using the Google Go SDK is running into the same issue. I don't see a lot of material on creating Cloud Run Jobs outside of the console or CLI. I believe the Terraform provider doesn't support it either. I will bring this up with our Google contact.

package main

import (
    "context"
    "fmt"

    "google.golang.org/api/run/v2"
)

func main() {
    ctx := context.Background()
    err := doRun(ctx)
    if err != nil {
        panic(err)
    }
}

func doRun(ctx context.Context) error {
    runService, err := run.NewService(ctx)
    if err != nil {
        return err
    }
    jobsService := run.NewProjectsLocationsJobsService(runService)
    createCall := jobsService.Create("projects/XXXX/locations/us-central1",
        &run.GoogleCloudRunV2Job{
            Template: &run.GoogleCloudRunV2ExecutionTemplate{
                TaskCount: 1,
                Template: &run.GoogleCloudRunV2TaskTemplate{
                    MaxRetries: 0,
                    Timeout:    "600s",
                    Containers: []*run.GoogleCloudRunV2Container{
                        {
                            Image: "us-docker.pkg.dev/cloudrun/container/job:latest",
                        },
                    },
                    ServiceAccount: "XXXX",
                },
            },
        })
    createCall.JobId("test")
    op, err := createCall.Do()
    if err != nil {
        return err
    }
    fmt.Printf("operation result: %+v\n", string(op.Response))
    return nil
}

[1oglop1]

Thank you for the detailed info about the problem! I used CLI to create the job and update the image in CI/CD as well. For the reference this is the command gcloud beta run jobs create, I tried debug verbosity but beta group did not log the requests.

I also wonder if v2 API can be the source of the problem, since docs mention create endpoint in v1 https://cloud.google.com/run/docs/reference/rest/v1/namespaces.jobs/create

[viveklak]

Thank you for the detailed info about the problem! I used CLI to create the job and update the image in CI/CD as well. For the reference this is the command gcloud beta run jobs create, I tried debug verbosity but beta group did not log the requests.

I also wonder if v2 API can be the source of the problem, since docs mention create endpoint in v1 https://cloud.google.com/run/docs/reference/rest/v1/namespaces.jobs/create

Yes I am suspecting something subtle missing from the v2 apis. The discussion here on adding jobs support also seems to discuss the v1 api: https://github.com/hashicorp/terraform-provider-google/issues/11743

[SimonMacIntyre]

Any chance of traction here? Seems the issue is upstream with Googles docs :(

[1oglop1]

I wonder if this fix helps https://github.com/hashicorp/terraform-provider-google/releases/tag/v4.46.0

[pinalbaldha]

Does the issue still persist?

few observations from error logs

google-native:run/v2:Job (job):
error: error sending request: googleapi: Error 404: Requested entity was not found.:
 "https://run.googleapis.com/v2/projects/my-gcp-project/locations/europe-west3/jobs?jobId=myjob" 

The reason could be {jobId} is expected as path param. https://run.googleapis.com/v2/projects/my-gcp-project/locations/europe-west3/jobs/myJob might succeed.

API Reference doc : https://cloud.google.com/run/docs/reference/rest/v2/projects.locations.jobs/get

map[__autonamed:true jobId:myjob location:europe-west3 name:projects/my-gcp-project/locations/europe-west3/jobs/job-570f641 
 project:my-gcp-project template:map[taskCount:1 template:map[containers:[map[image:us-docker.pkg.dev/cloudrun/container/job:latest]] maxRetries:0 timeout:600s]]]

Also not sure if this is expected, jobId: myjob and as per this it should be converted to name:projects/my-gcp-project/locations/europe-west3/jobs/myJob

@mikhailshilkov & team More specifically can you pl mention what error/unexpected behaviour from run.googleapis.com endpoint side.