search

GoogleCloudPlatform / terraform-provider-cdap

Custom Terraform Provider for CDAP
https://registry.terraform.io/providers/GoogleCloudPlatform/cdap/
Apache License 2.0
12 stars 12 forks source link

cdap_streaming_program_run error 404 #100

Closed pradeepchc closed 3 years ago

pradeepchc commented 3 years ago

We are facing an error while running the streaming pipeline in datafusion instance. Terraform is not able to recognise the app name.

Code :

data "google_client_config" "current" {}

terraform { required_providers { cdap = { source = "GoogleCloudPlatform/cdap" version = "0.9.0" } } }

provider "cdap" {

.

host = "https://xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx.googleusercontent.com/api/" token = data.google_client_config.current.access_token }

resource "cdap_streaming_program_run" "test" { app = "testpoc" ( the app name is referred from deployed pipeline name) type = "spark"

runtime_arguments = { "system.profile.name" = "SYSTEM:dataproc" } }

-------error------------

Error: 404: │ <!DOCTYPE html> │ │ │ │ Error 404 (Not Found)!!1 │ │ │

404. That’s an error. │

That’s all we know. │ │ │ with cdap_streaming_program_run.test, │ on main.tf line 18, in resource "cdap_streaming_program_run" "test": │ 18: resource "cdap_streaming_program_run" "test" {

umairidris commented 3 years ago

You may wish to create the run in UI and then list the runs through the REST API to see what the right values to set should be

https://cloud.google.com/data-fusion/docs/reference/cdap-reference

Endpoints: https://github.com/GoogleCloudPlatform/terraform-provider-cdap/blob/master/cdap/resource_streaming_program_run.go#L306

pradeepchc commented 3 years ago

dear @umairidris thanks for your reply. We are trying to deploying the pipeline using terraform script. when we are trying to execute the script we are facing the below error, We tried also with the REST API and ended with same result. Initially we created the cdap application using the export json file and later we are referring it (application name) in streaming run module.

umairidris commented 3 years ago

Hi @pradeepchc,

This TF provider is making REST calls under the hood, so please continue working through the REST API calls until it works.

See https://cloud.google.com/data-fusion/docs/reference/cdap-reference#batch_pipeline_run_records for more documentation on this.

If there is a bug in the provider we are happy to accept contributions.

If you are unable to get the REST API working, I would encourage you to raise this issue with cloud support who can get you in touch with the Data Fusion team themselves as they don't monitor this repo.

pradeepchc commented 3 years ago

@umairidris : Yes we are now only working on the REST API These rest api also gives the same error as below, are we missing something while deploying pipeline?

<!DOCTYPE html>

Error 404 (Not Found)!!1

404. That’s an error.

That’s all we know.

umairidris commented 3 years ago

The data fusion team will be better able to support you, consider filing a ticket with cloud support: https://cloud.google.com/support-hub