GoogleCloudDataproc / initialization-actions

Run in all nodes of your cluster before the cluster starts - lets you customize your cluster
https://cloud.google.com/dataproc/init-actions
Apache License 2.0
588 stars 512 forks source link

Added OpenTelemetry Initialization action for Dataproc clusters (along with integration tests) #1075

Closed akshatbhargava123 closed 1 year ago

akshatbhargava123 commented 1 year ago

PR credits: @shambhavirai16 Original author could not continue contribution.

akshatbhargava123 commented 1 year ago

/gcbrun

akshatbhargava123 commented 1 year ago

Added my official email in GH account, checking again if it works.

akshatbhargava123 commented 1 year ago

/gcbrun

jayadeep-jayaraman commented 1 year ago

/gcbrun

jayadeep-jayaraman commented 1 year ago

BUILD file is missing in the otel folder

2023-08-04T13:35:45.114355556Z ERROR: no such package 'otel': BUILD file not found in any of the following directories. Add a BUILD file to a directory to mark it as a package.
2023-08-04T13:35:45.114416017Z  - /init-actions/otel
akshatbhargava123 commented 1 year ago

BUILD file is missing in the otel folder

2023-08-04T13:35:45.114355556Z ERROR: no such package 'otel': BUILD file not found in any of the following directories. Add a BUILD file to a directory to mark it as a package.
2023-08-04T13:35:45.114416017Z  - /init-actions/otel

Got it. I initially followed cloud-sql-proxy as an example where the BUILD instructions were specified in the project scoped (global) file. Updated it to have local BUILD file.

jayadeep-jayaraman commented 1 year ago

/gcbrun

akshatbhargava123 commented 1 year ago

/gcbrun

akshatbhargava123 commented 1 year ago

/gcbrun

akshatbhargava123 commented 1 year ago

There's now environment related issue because systemd is not installed/initialised in the test image:

2023-08-10T16:27:42.686337019Z STDERR:
2023-08-10T16:27:42.686343412Z System has not been booted with systemd as init system (PID 1). Can't operate.
2023-08-10T16:27:42.686349622Z Failed to connect to bus: Host is down
akshatbhargava123 commented 1 year ago

/gcbrun

akshatbhargava123 commented 1 year ago

Now only rocky-8 tests are failing due to some permission issue, it might be flaky so I'm re-running it.

akshatbhargava123 commented 1 year ago

/gcbrun

akshatbhargava123 commented 1 year ago

/gcbrun

akshatbhargava123 commented 1 year ago

/gcbrun

akshatbhargava123 commented 1 year ago

/gcbrun

akshatbhargava123 commented 1 year ago

/gcbrun