Add PrometheusRemoteWrite Exporter

[benjaminguttmann-avtq]

Adds prometheusremotewriteexporter to the otel collector

[ctlong]

The failure is unrelated to your change.

The pipeline is failing to upload a windows2019 stemcell, which it needs to do in order to deploy a windows cell to test the otel-collector-windows job. For some reason CF-D thinks the 2019.74 windows2019 stemcell is available, but it's not in bosh.io.

[ctlong]

~I'll try re-running it through the pipeline tomorrow.~ Re-running it now, bosh.io appears to be working again.

[ctlong]

There was what appears to be a legitimate CI error with this change:

Task 52 | 22:52:23 | Compiling packages: otel-collector-windows/4d8cdf0c5e8010b06f18e6ab610ae09a033ad4654062d7e2c987724d2d0b8b2f (00:07:45)
                   L Error: Action Failed get_task: Task 6e20247d-4343-4266-54ba-412c7a81905d result: Compiling package otel-collector-windows: Running packaging script: Running packaging script: Command exited with 1; Stdout: , Stderr: 2024-06-12T22:51:54.634Z   INFO    internal/command.go:159 Using config file   {"path": "otel-collector\\config.yaml"}
2024-06-12T22:51:54.634Z    INFO    builder/config.go:109   Using go    {"go-executable": "C:\\var\\vcap\\packages\\golang-1.21-windows\\go\\bin\\go.exe"}
2024-06-12T22:51:54.635Z    INFO    builder/main.go:62  Skipping generating source codes.
2024-06-12T22:51:54.635Z    INFO    builder/main.go:129 Generating source codes only, will not update go.mod and retrieve Go modules.
2024-06-12T22:51:54.635Z    INFO    builder/main.go:102 Compiling
2024-06-12T22:51:54.635Z    INFO    builder/main.go:25  Running go subcommand.  {"arguments": ["build", "-trimpath", "-o", "cf-otel-collector", "-ldflags=-s -w"]}
Error: failed to compile the OpenTelemetry Collector distribution: go subcommand failed with args '[build -trimpath -o cf-otel-collector -ldflags=-s -w]': exit status 1. Output:
vendor\github.com\open-telemetry\opentelemetry-collector-contrib\exporter\prometheusremotewriteexporter\exporter.go:31:2: cannot find module providing package github.com/open-telemetry/opentelemetry-collector-contrib/pkg/translator/prometheusremotewrite: import lookup disabled by -mod=vendor
    (Go version in go.mod is at least 1.14 and vendor directory exists.)

iex (get-content -raw packaging) : Error building otel-collector
At line:22 char:3
+   Write-Error "Error building otel-collector"
+   ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (:) [Write-Error], WriteErrorException
    + FullyQualifiedErrorId : Microsoft.PowerShell.Commands.WriteErrorException

@benjaminguttmann-avtq can you please have a look

[benjaminguttmann-avtq]

@ctlong do I see it correctly that the issue is related to windows stemcells?

[acrmp]

@benjaminguttmann-avtq The error Carson shared did occur when compiling the package on a Windows VM but doesn't have to be specific to Windows.

It looks like not all of the paths referenced in the generated go mod are vendored.

Can you also remove the cf-otel-collector binary please.

Thanks!

[benjaminguttmann-avtq]

@acrmp @ctlong tests look good now I would say

[acrmp]

Hi @benjaminguttmann-avtq,

Thanks for making these changes. Unfortunately it looks like one of the new packages required by the prometheus remote write exporter has not been included in vendor in the commit, so it won't compile:

$ git log --name-only 9a00189e..2790e0df | grep pkg/translator/prometheusremotewrite | wc -l
0

We've also just cut v0.5.0 which will cause your current PR to have merge conflicts as the version of otel-collector has been bumped. Sorry about that.

Please let us know if you're ok to rework the PR against main (this time pulling in v0.103.0 of the new exporter in the config) and add the missing package or if you would prefer us to do so.

Thanks,

Andrew.

[benjaminguttmann-avtq]

Hi @acrmp I will rework the PR and update accordingly.

[benjaminguttmann-avtq]

Hi @acrmp ,

hmmm prepared it again, but the file you are talking about is still missing... maybe you know what I am doing wrong here.

I added the prometheusremotewrite to

src/otel-collector/config.yaml

then I ran this script

https://github.com/cloudfoundry/otel-collector-release/blob/main/scripts/regenerate-otel-collector-distribution

and ran git add and git commit.

I also created a dev release and could use the binary on the VM, so a bit confused right now.

Thank you

[acrmp]

If the file exists on the filesystem but is not staged when you run git add then it may be because you have a gitignore pattern set that is excluding them?

In this case I would guess that you might have gitignore set to exclude paths with pkg.

[benjaminguttmann-avtq]

@acrmp you have been totally right, found a global ignore that was ignoring pkg

git log --name-only 16054f17d6768360eb74ea8a277686d1c2aa3da1 | grep pkg/translator/prometheusremotewrite | wc -l
       8

Now it looks way better