open-telemetry / opentelemetry-collector-contrib

Contrib repository for the OpenTelemetry Collector
https://opentelemetry.io
Apache License 2.0
3.07k stars 2.37k forks source link

Flaky test - prometheusremotewriteexporter - Test_PushMetrics/WAL #9124

Closed jpkrohling closed 1 year ago

jpkrohling commented 2 years ago

Seen here: https://github.com/open-telemetry/opentelemetry-collector-contrib/runs/5884764908?check_suite_focus=true

2022-04-08T12:16:18.3480658Z === RUN   Test_PushMetrics
2022-04-08T12:16:18.3480983Z === RUN   Test_PushMetrics/WAL
2022-04-08T12:16:18.3481310Z === RUN   Test_PushMetrics/WAL/invalid_type_case
2022-04-08T12:16:18.3481690Z === PAUSE Test_PushMetrics/WAL/invalid_type_case
2022-04-08T12:16:18.3482052Z === RUN   Test_PushMetrics/WAL/intSum_case
2022-04-08T12:16:18.3507241Z === PAUSE Test_PushMetrics/WAL/intSum_case
2022-04-08T12:16:18.3507709Z === RUN   Test_PushMetrics/WAL/doubleSum_case
2022-04-08T12:16:18.3508092Z === PAUSE Test_PushMetrics/WAL/doubleSum_case
2022-04-08T12:16:18.3508475Z === RUN   Test_PushMetrics/WAL/doubleGauge_case
2022-04-08T12:16:18.3508824Z === PAUSE Test_PushMetrics/WAL/doubleGauge_case
2022-04-08T12:16:18.3509195Z === RUN   Test_PushMetrics/WAL/intGauge_case
2022-04-08T12:16:18.3509564Z === PAUSE Test_PushMetrics/WAL/intGauge_case
2022-04-08T12:16:18.3509924Z === RUN   Test_PushMetrics/WAL/histogram_case
2022-04-08T12:16:18.3510285Z === PAUSE Test_PushMetrics/WAL/histogram_case
2022-04-08T12:16:18.3510646Z === RUN   Test_PushMetrics/WAL/summary_case
2022-04-08T12:16:18.3511005Z === PAUSE Test_PushMetrics/WAL/summary_case
2022-04-08T12:16:18.3511383Z === RUN   Test_PushMetrics/WAL/unmatchedBoundBucketHist_case
2022-04-08T12:16:18.3511809Z === PAUSE Test_PushMetrics/WAL/unmatchedBoundBucketHist_case
2022-04-08T12:16:18.3512190Z === RUN   Test_PushMetrics/WAL/5xx_case
2022-04-08T12:16:18.3512531Z === PAUSE Test_PushMetrics/WAL/5xx_case
2022-04-08T12:16:18.3512888Z === RUN   Test_PushMetrics/WAL/emptyGauge_case
2022-04-08T12:16:18.3513251Z === PAUSE Test_PushMetrics/WAL/emptyGauge_case
2022-04-08T12:16:18.3513641Z === RUN   Test_PushMetrics/WAL/emptyCumulativeSum_case
2022-04-08T12:16:18.3514014Z === PAUSE Test_PushMetrics/WAL/emptyCumulativeSum_case
2022-04-08T12:16:18.3514429Z === RUN   Test_PushMetrics/WAL/emptyCumulativeHistogram_case
2022-04-08T12:16:18.3516746Z === PAUSE Test_PushMetrics/WAL/emptyCumulativeHistogram_case
2022-04-08T12:16:18.3518089Z === RUN   Test_PushMetrics/WAL/emptySummary_case
2022-04-08T12:16:18.3520411Z === PAUSE Test_PushMetrics/WAL/emptySummary_case
2022-04-08T12:16:18.3521538Z === RUN   Test_PushMetrics/WAL/staleNaNIntGauge_case
2022-04-08T12:16:18.3523534Z === PAUSE Test_PushMetrics/WAL/staleNaNIntGauge_case
2022-04-08T12:16:18.3524517Z === RUN   Test_PushMetrics/WAL/staleNaNDoubleGauge_case
2022-04-08T12:16:18.3527591Z === PAUSE Test_PushMetrics/WAL/staleNaNDoubleGauge_case
2022-04-08T12:16:18.3527994Z === RUN   Test_PushMetrics/WAL/staleNaNIntSum_case
2022-04-08T12:16:18.3529621Z === PAUSE Test_PushMetrics/WAL/staleNaNIntSum_case
2022-04-08T12:16:18.3531011Z === RUN   Test_PushMetrics/WAL/staleNaNSum_case
2022-04-08T12:16:18.3532812Z === PAUSE Test_PushMetrics/WAL/staleNaNSum_case
2022-04-08T12:16:18.3554785Z === RUN   Test_PushMetrics/WAL/staleNaNHistogram_case
2022-04-08T12:16:18.3555277Z === PAUSE Test_PushMetrics/WAL/staleNaNHistogram_case
2022-04-08T12:16:18.3555705Z === RUN   Test_PushMetrics/WAL/staleNaNEmptyHistogram_case
2022-04-08T12:16:18.3556163Z === PAUSE Test_PushMetrics/WAL/staleNaNEmptyHistogram_case
2022-04-08T12:16:18.3556564Z === RUN   Test_PushMetrics/WAL/staleNaNSummary_case
2022-04-08T12:16:18.3556950Z === PAUSE Test_PushMetrics/WAL/staleNaNSummary_case
2022-04-08T12:16:18.3557893Z === CONT  Test_PushMetrics/WAL/invalid_type_case
2022-04-08T12:16:18.3558316Z === CONT  Test_PushMetrics/WAL/emptyCumulativeSum_case
2022-04-08T12:16:22.4700997Z === CONT  Test_PushMetrics/WAL/5xx_case
2022-04-08T12:16:26.5700241Z === CONT  Test_PushMetrics/WAL/unmatchedBoundBucketHist_case
2022-04-08T12:16:26.5734642Z === CONT  Test_PushMetrics/WAL/summary_case
2022-04-08T12:16:30.6756277Z === CONT  Test_PushMetrics/WAL/unmatchedBoundBucketHist_case
2022-04-08T12:16:30.6757417Z     testing.go:967: TempDir RemoveAll cleanup: unlinkat /tmp/Test_PushMetricsWALunmatchedBoundBucketHist_case1954787114/001: directory not empty
2022-04-08T12:16:30.6758030Z === CONT  Test_PushMetrics/WAL/histogram_case
2022-04-08T12:16:34.7776159Z === CONT  Test_PushMetrics/WAL/intGauge_case
2022-04-08T12:16:38.8782752Z === CONT  Test_PushMetrics/WAL/emptyGauge_case
2022-04-08T12:16:42.9812311Z === CONT  Test_PushMetrics/WAL/doubleGauge_case
2022-04-08T12:16:47.0825569Z === CONT  Test_PushMetrics/WAL/intSum_case
2022-04-08T12:16:51.1866769Z === CONT  Test_PushMetrics/WAL/doubleSum_case
2022-04-08T12:16:55.2890986Z === CONT  Test_PushMetrics/WAL/staleNaNSummary_case
2022-04-08T12:16:59.3915275Z === CONT  Test_PushMetrics/WAL/staleNaNEmptyHistogram_case
2022-04-08T12:17:03.4936243Z === CONT  Test_PushMetrics/WAL/staleNaNHistogram_case
2022-04-08T12:17:07.5961395Z === CONT  Test_PushMetrics/WAL/staleNaNIntSum_case
2022-04-08T12:17:11.7010754Z === CONT  Test_PushMetrics/WAL/staleNaNSum_case
2022-04-08T12:17:15.8029494Z === CONT  Test_PushMetrics/WAL/staleNaNIntGauge_case
2022-04-08T12:17:19.9059623Z === CONT  Test_PushMetrics/WAL/staleNaNDoubleGauge_case
2022-04-08T12:17:24.0073006Z === CONT  Test_PushMetrics/WAL/emptySummary_case
2022-04-08T12:17:28.1103885Z === CONT  Test_PushMetrics/WAL/emptyCumulativeHistogram_case
2022-04-08T12:17:36.3139815Z === RUN   Test_PushMetrics/NoWAL
2022-04-08T12:17:36.3140832Z === RUN   Test_PushMetrics/NoWAL/invalid_type_case
2022-04-08T12:17:36.3141793Z === PAUSE Test_PushMetrics/NoWAL/invalid_type_case
2022-04-08T12:17:36.3142703Z === RUN   Test_PushMetrics/NoWAL/intSum_case
2022-04-08T12:17:36.3143569Z === PAUSE Test_PushMetrics/NoWAL/intSum_case
2022-04-08T12:17:36.3144462Z === RUN   Test_PushMetrics/NoWAL/doubleSum_case
2022-04-08T12:17:36.3145366Z === PAUSE Test_PushMetrics/NoWAL/doubleSum_case
2022-04-08T12:17:36.3179731Z === RUN   Test_PushMetrics/NoWAL/doubleGauge_case
2022-04-08T12:17:36.3180230Z === PAUSE Test_PushMetrics/NoWAL/doubleGauge_case
2022-04-08T12:17:36.3180642Z === RUN   Test_PushMetrics/NoWAL/intGauge_case
2022-04-08T12:17:36.3181010Z === PAUSE Test_PushMetrics/NoWAL/intGauge_case
2022-04-08T12:17:36.3181397Z === RUN   Test_PushMetrics/NoWAL/histogram_case
2022-04-08T12:17:36.3181791Z === PAUSE Test_PushMetrics/NoWAL/histogram_case
2022-04-08T12:17:36.3182532Z === RUN   Test_PushMetrics/NoWAL/summary_case
2022-04-08T12:17:36.3182924Z === PAUSE Test_PushMetrics/NoWAL/summary_case
2022-04-08T12:17:36.3183369Z === RUN   Test_PushMetrics/NoWAL/unmatchedBoundBucketHist_case
2022-04-08T12:17:36.3183835Z === PAUSE Test_PushMetrics/NoWAL/unmatchedBoundBucketHist_case
2022-04-08T12:17:36.3184253Z === RUN   Test_PushMetrics/NoWAL/5xx_case
2022-04-08T12:17:36.3184616Z === PAUSE Test_PushMetrics/NoWAL/5xx_case
2022-04-08T12:17:36.3185004Z === RUN   Test_PushMetrics/NoWAL/emptyGauge_case
2022-04-08T12:17:36.3185405Z === PAUSE Test_PushMetrics/NoWAL/emptyGauge_case
2022-04-08T12:17:36.3185931Z === RUN   Test_PushMetrics/NoWAL/emptyCumulativeSum_case
2022-04-08T12:17:36.3186375Z === PAUSE Test_PushMetrics/NoWAL/emptyCumulativeSum_case
2022-04-08T12:17:36.3186835Z === RUN   Test_PushMetrics/NoWAL/emptyCumulativeHistogram_case
2022-04-08T12:17:36.3187321Z === PAUSE Test_PushMetrics/NoWAL/emptyCumulativeHistogram_case
2022-04-08T12:17:36.3187764Z === RUN   Test_PushMetrics/NoWAL/emptySummary_case
2022-04-08T12:17:36.3188180Z === PAUSE Test_PushMetrics/NoWAL/emptySummary_case
2022-04-08T12:17:36.3188577Z === RUN   Test_PushMetrics/NoWAL/staleNaNIntGauge_case
2022-04-08T12:17:36.3189005Z === PAUSE Test_PushMetrics/NoWAL/staleNaNIntGauge_case
2022-04-08T12:17:36.3189443Z === RUN   Test_PushMetrics/NoWAL/staleNaNDoubleGauge_case
2022-04-08T12:17:36.3189893Z === PAUSE Test_PushMetrics/NoWAL/staleNaNDoubleGauge_case
2022-04-08T12:17:36.3190316Z === RUN   Test_PushMetrics/NoWAL/staleNaNIntSum_case
2022-04-08T12:17:36.3190730Z === PAUSE Test_PushMetrics/NoWAL/staleNaNIntSum_case
2022-04-08T12:17:36.3191118Z === RUN   Test_PushMetrics/NoWAL/staleNaNSum_case
2022-04-08T12:17:36.3191519Z === PAUSE Test_PushMetrics/NoWAL/staleNaNSum_case
2022-04-08T12:17:36.3191933Z === RUN   Test_PushMetrics/NoWAL/staleNaNHistogram_case
2022-04-08T12:17:36.3192358Z === PAUSE Test_PushMetrics/NoWAL/staleNaNHistogram_case
2022-04-08T12:17:36.3192802Z === RUN   Test_PushMetrics/NoWAL/staleNaNEmptyHistogram_case
2022-04-08T12:17:36.3193252Z === PAUSE Test_PushMetrics/NoWAL/staleNaNEmptyHistogram_case
2022-04-08T12:17:36.3193697Z === RUN   Test_PushMetrics/NoWAL/staleNaNSummary_case
2022-04-08T12:17:36.3194455Z === PAUSE Test_PushMetrics/NoWAL/staleNaNSummary_case
2022-04-08T12:17:36.3194917Z === CONT  Test_PushMetrics/NoWAL/invalid_type_case
2022-04-08T12:17:36.3195341Z === CONT  Test_PushMetrics/NoWAL/staleNaNSummary_case
2022-04-08T12:17:36.3211911Z === CONT  Test_PushMetrics/NoWAL/doubleSum_case
2022-04-08T12:17:36.3242469Z === CONT  Test_PushMetrics/NoWAL/emptyGauge_case
2022-04-08T12:17:36.3293833Z === CONT  Test_PushMetrics/NoWAL/staleNaNEmptyHistogram_case
2022-04-08T12:17:36.3305283Z === CONT  Test_PushMetrics/NoWAL/staleNaNHistogram_case
2022-04-08T12:17:36.3329633Z === CONT  Test_PushMetrics/NoWAL/staleNaNSum_case
2022-04-08T12:17:36.3351280Z === CONT  Test_PushMetrics/NoWAL/staleNaNIntSum_case
2022-04-08T12:17:36.3372656Z === CONT  Test_PushMetrics/NoWAL/staleNaNDoubleGauge_case
2022-04-08T12:17:36.3407080Z === CONT  Test_PushMetrics/NoWAL/staleNaNIntGauge_case
2022-04-08T12:17:36.3426058Z === CONT  Test_PushMetrics/NoWAL/emptyCumulativeHistogram_case
2022-04-08T12:17:36.3476128Z === CONT  Test_PushMetrics/NoWAL/emptyCumulativeSum_case
2022-04-08T12:17:36.3483507Z === CONT  Test_PushMetrics/NoWAL/intGauge_case
2022-04-08T12:17:36.3505688Z === CONT  Test_PushMetrics/NoWAL/emptySummary_case
2022-04-08T12:17:36.3529680Z === CONT  Test_PushMetrics/NoWAL/5xx_case
2022-04-08T12:17:36.3559418Z === CONT  Test_PushMetrics/NoWAL/unmatchedBoundBucketHist_case
2022-04-08T12:17:36.3587772Z === CONT  Test_PushMetrics/NoWAL/histogram_case
2022-04-08T12:17:36.3612171Z === CONT  Test_PushMetrics/NoWAL/doubleGauge_case
2022-04-08T12:17:36.3617018Z === CONT  Test_PushMetrics/NoWAL/summary_case
2022-04-08T12:17:36.3690255Z === CONT  Test_PushMetrics/NoWAL/intSum_case
2022-04-08T12:17:36.3749433Z --- FAIL: Test_PushMetrics (78.03s)
2022-04-08T12:17:36.3750735Z     --- FAIL: Test_PushMetrics/WAL (0.01s)
2022-04-08T12:17:36.3752979Z         --- PASS: Test_PushMetrics/WAL/invalid_type_case (4.11s)
2022-04-08T12:17:36.3754960Z         --- PASS: Test_PushMetrics/WAL/emptyCumulativeSum_case (8.22s)
2022-04-08T12:17:36.3756820Z         --- PASS: Test_PushMetrics/WAL/5xx_case (4.10s)
2022-04-08T12:17:36.3759027Z         --- FAIL: Test_PushMetrics/WAL/unmatchedBoundBucketHist_case (4.11s)
2022-04-08T12:17:36.3760749Z         --- PASS: Test_PushMetrics/WAL/summary_case (8.20s)
2022-04-08T12:17:36.3762337Z         --- PASS: Test_PushMetrics/WAL/histogram_case (8.20s)
2022-04-08T12:17:36.3763917Z         --- PASS: Test_PushMetrics/WAL/intGauge_case (8.20s)
2022-04-08T12:17:36.3765799Z         --- PASS: Test_PushMetrics/WAL/emptyGauge_case (8.20s)
2022-04-08T12:17:36.3767422Z         --- PASS: Test_PushMetrics/WAL/doubleGauge_case (8.21s)
2022-04-08T12:17:36.3768999Z         --- PASS: Test_PushMetrics/WAL/intSum_case (8.21s)
2022-04-08T12:17:36.3770655Z         --- PASS: Test_PushMetrics/WAL/doubleSum_case (8.20s)
2022-04-08T12:17:36.3772330Z         --- PASS: Test_PushMetrics/WAL/staleNaNSummary_case (8.20s)
2022-04-08T12:17:36.3774169Z         --- PASS: Test_PushMetrics/WAL/staleNaNEmptyHistogram_case (8.20s)
2022-04-08T12:17:36.3775932Z         --- PASS: Test_PushMetrics/WAL/staleNaNHistogram_case (8.21s)
2022-04-08T12:17:36.3777653Z         --- PASS: Test_PushMetrics/WAL/staleNaNIntSum_case (8.21s)
2022-04-08T12:17:36.3779299Z         --- PASS: Test_PushMetrics/WAL/staleNaNSum_case (8.20s)
2022-04-08T12:17:36.3781021Z         --- PASS: Test_PushMetrics/WAL/staleNaNIntGauge_case (8.20s)
2022-04-08T12:17:36.3782837Z         --- PASS: Test_PushMetrics/WAL/staleNaNDoubleGauge_case (8.20s)
2022-04-08T12:17:36.3784537Z         --- PASS: Test_PushMetrics/WAL/emptySummary_case (8.20s)
2022-04-08T12:17:36.3786353Z         --- PASS: Test_PushMetrics/WAL/emptyCumulativeHistogram_case (8.20s)
2022-04-08T12:17:36.3787742Z     --- PASS: Test_PushMetrics/NoWAL (0.00s)
2022-04-08T12:17:36.3789374Z         --- PASS: Test_PushMetrics/NoWAL/staleNaNSummary_case (0.01s)
2022-04-08T12:17:36.3791084Z         --- PASS: Test_PushMetrics/NoWAL/invalid_type_case (0.01s)
2022-04-08T12:17:36.3792741Z         --- PASS: Test_PushMetrics/NoWAL/emptyGauge_case (0.00s)
2022-04-08T12:17:36.3794363Z         --- PASS: Test_PushMetrics/NoWAL/doubleSum_case (0.01s)
2022-04-08T12:17:36.3796087Z         --- PASS: Test_PushMetrics/NoWAL/staleNaNHistogram_case (0.00s)
2022-04-08T12:17:36.3798362Z         --- PASS: Test_PushMetrics/NoWAL/staleNaNEmptyHistogram_case (0.01s)
2022-04-08T12:17:36.3800133Z         --- PASS: Test_PushMetrics/NoWAL/staleNaNSum_case (0.00s)
2022-04-08T12:17:36.3801951Z         --- PASS: Test_PushMetrics/NoWAL/staleNaNDoubleGauge_case (0.00s)
2022-04-08T12:17:36.3803741Z         --- PASS: Test_PushMetrics/NoWAL/staleNaNIntSum_case (0.01s)
2022-04-08T12:17:36.3805651Z         --- PASS: Test_PushMetrics/NoWAL/emptyCumulativeHistogram_case (0.00s)
2022-04-08T12:17:36.3807451Z         --- PASS: Test_PushMetrics/NoWAL/staleNaNIntGauge_case (0.01s)
2022-04-08T12:17:36.3809273Z         --- PASS: Test_PushMetrics/NoWAL/emptyCumulativeSum_case (0.00s)
2022-04-08T12:17:36.3811012Z         --- PASS: Test_PushMetrics/NoWAL/emptySummary_case (0.00s)
2022-04-08T12:17:36.3812638Z         --- PASS: Test_PushMetrics/NoWAL/intGauge_case (0.01s)
2022-04-08T12:17:36.3814493Z         --- PASS: Test_PushMetrics/NoWAL/unmatchedBoundBucketHist_case (0.00s)
2022-04-08T12:17:36.3816157Z         --- PASS: Test_PushMetrics/NoWAL/5xx_case (0.01s)
2022-04-08T12:17:36.3817735Z         --- PASS: Test_PushMetrics/NoWAL/histogram_case (0.00s)
2022-04-08T12:17:36.3819355Z         --- PASS: Test_PushMetrics/NoWAL/summary_case (0.01s)
2022-04-08T12:17:36.3820994Z         --- PASS: Test_PushMetrics/NoWAL/doubleGauge_case (0.01s)
2022-04-08T12:17:36.3822586Z         --- PASS: Test_PushMetrics/NoWAL/intSum_case (0.00s)
2022-04-08T12:17:36.3823652Z === RUN   Test_validateAndSanitizeExternalLabels
2022-04-08T12:17:36.3824857Z === RUN   Test_validateAndSanitizeExternalLabels/success_case_no_labels
2022-04-08T12:17:36.3826514Z === RUN   Test_validateAndSanitizeExternalLabels/success_case_with_labels
2022-04-08T12:17:36.3827872Z === RUN   Test_validateAndSanitizeExternalLabels/success_case_2_with_labels
2022-04-08T12:17:36.3829255Z === RUN   Test_validateAndSanitizeExternalLabels/success_case_with_sanitized_labels
2022-04-08T12:17:36.3830656Z === RUN   Test_validateAndSanitizeExternalLabels/labels_that_start_with_underscore
2022-04-08T12:17:36.3832024Z === RUN   Test_validateAndSanitizeExternalLabels/labels_that_start_with_digit
2022-04-08T12:17:36.3833346Z === RUN   Test_validateAndSanitizeExternalLabels/fail_case_empty_label
2022-04-08T12:17:36.3834848Z === RUN   Test_validateAndSanitizeExternalLabels/success_case_no_labels#01
2022-04-08T12:17:36.3836156Z === RUN   Test_validateAndSanitizeExternalLabels/success_case_with_labels#01
2022-04-08T12:17:36.3839150Z === RUN   Test_validateAndSanitizeExternalLabels/success_case_2_with_labels#01
2022-04-08T12:17:36.3840633Z === RUN   Test_validateAndSanitizeExternalLabels/success_case_with_sanitized_labels#01
2022-04-08T12:17:36.3842066Z === RUN   Test_validateAndSanitizeExternalLabels/labels_that_start_with_underscore#01
2022-04-08T12:17:36.3843467Z === RUN   Test_validateAndSanitizeExternalLabels/labels_that_start_with_digit#01
2022-04-08T12:17:36.3844797Z === RUN   Test_validateAndSanitizeExternalLabels/fail_case_empty_label#01
2022-04-08T12:17:36.3846392Z --- PASS: Test_validateAndSanitizeExternalLabels (0.00s)
2022-04-08T12:17:36.3848280Z     --- PASS: Test_validateAndSanitizeExternalLabels/success_case_no_labels (0.00s)
2022-04-08T12:17:36.3850530Z     --- PASS: Test_validateAndSanitizeExternalLabels/success_case_with_labels (0.00s)
2022-04-08T12:17:36.3852524Z     --- PASS: Test_validateAndSanitizeExternalLabels/success_case_2_with_labels (0.00s)
2022-04-08T12:17:36.3854578Z     --- PASS: Test_validateAndSanitizeExternalLabels/success_case_with_sanitized_labels (0.00s)
2022-04-08T12:17:36.3856681Z     --- PASS: Test_validateAndSanitizeExternalLabels/labels_that_start_with_underscore (0.00s)
2022-04-08T12:17:36.3858654Z     --- PASS: Test_validateAndSanitizeExternalLabels/labels_that_start_with_digit (0.00s)
2022-04-08T12:17:36.3860593Z     --- PASS: Test_validateAndSanitizeExternalLabels/fail_case_empty_label (0.00s)
2022-04-08T12:17:36.3862534Z     --- PASS: Test_validateAndSanitizeExternalLabels/success_case_no_labels#01 (0.00s)
2022-04-08T12:17:36.3864491Z     --- PASS: Test_validateAndSanitizeExternalLabels/success_case_with_labels#01 (0.00s)
2022-04-08T12:17:36.3866466Z     --- PASS: Test_validateAndSanitizeExternalLabels/success_case_2_with_labels#01 (0.00s)
2022-04-08T12:17:36.3868557Z     --- PASS: Test_validateAndSanitizeExternalLabels/success_case_with_sanitized_labels#01 (0.00s)
2022-04-08T12:17:36.3870729Z     --- PASS: Test_validateAndSanitizeExternalLabels/labels_that_start_with_underscore#01 (0.00s)
2022-04-08T12:17:36.3872763Z     --- PASS: Test_validateAndSanitizeExternalLabels/labels_that_start_with_digit#01 (0.00s)
2022-04-08T12:17:36.3874720Z     --- PASS: Test_validateAndSanitizeExternalLabels/fail_case_empty_label#01 (0.00s)
2022-04-08T12:17:36.3875869Z === RUN   TestWALOnExporterRoundTrip
2022-04-08T12:17:40.4885034Z --- PASS: TestWALOnExporterRoundTrip (4.11s)
2022-04-08T12:17:40.4886374Z === RUN   Test_createDefaultConfig
2022-04-08T12:17:40.4888340Z --- PASS: Test_createDefaultConfig (0.00s)
2022-04-08T12:17:40.4889214Z === RUN   Test_createMetricsExporter
2022-04-08T12:17:40.4889921Z === RUN   Test_createMetricsExporter/success_case
2022-04-08T12:17:40.4935982Z === RUN   Test_createMetricsExporter/fail_case
2022-04-08T12:17:40.4937308Z === RUN   Test_createMetricsExporter/invalid_config_case
2022-04-08T12:17:40.4938131Z === RUN   Test_createMetricsExporter/invalid_tls_config_case
2022-04-08T12:17:40.4975658Z --- PASS: Test_createMetricsExporter (0.01s)
2022-04-08T12:17:40.4977020Z     --- PASS: Test_createMetricsExporter/success_case (0.00s)
2022-04-08T12:17:40.5069298Z     --- PASS: Test_createMetricsExporter/fail_case (0.00s)
2022-04-08T12:17:40.5070206Z     --- PASS: Test_createMetricsExporter/invalid_config_case (0.00s)
2022-04-08T12:17:40.5070798Z     --- PASS: Test_createMetricsExporter/invalid_tls_config_case (0.00s)
2022-04-08T12:17:40.5071169Z === RUN   Test_batchTimeSeries
2022-04-08T12:17:40.5071495Z === RUN   Test_batchTimeSeries/no_timeseries
2022-04-08T12:17:40.5077567Z === RUN   Test_batchTimeSeries/normal_case
2022-04-08T12:17:40.5078076Z === RUN   Test_batchTimeSeries/two_requests
2022-04-08T12:17:40.5078711Z --- PASS: Test_batchTimeSeries (0.00s)
2022-04-08T12:17:40.5079131Z     --- PASS: Test_batchTimeSeries/no_timeseries (0.00s)
2022-04-08T12:17:40.5079932Z     --- PASS: Test_batchTimeSeries/normal_case (0.00s)
2022-04-08T12:17:40.5080500Z     --- PASS: Test_batchTimeSeries/two_requests (0.00s)
2022-04-08T12:17:40.5080972Z === RUN   TestEnsureTimeseriesPointsAreSortedByTimestamp
2022-04-08T12:17:40.5081581Z --- PASS: TestEnsureTimeseriesPointsAreSortedByTimestamp (0.00s)
2022-04-08T12:17:40.5081930Z === RUN   TestWALCreation_nilConfig
2022-04-08T12:17:40.5082387Z --- PASS: TestWALCreation_nilConfig (0.00s)
2022-04-08T12:17:40.5082688Z === RUN   TestWALCreation_nonNilConfig
2022-04-08T12:17:40.5083161Z --- PASS: TestWALCreation_nonNilConfig (0.00s)
2022-04-08T12:17:40.5083464Z === RUN   TestWALStopManyTimes
2022-04-08T12:17:40.5083904Z --- PASS: TestWALStopManyTimes (0.00s)
2022-04-08T12:17:40.5084154Z === RUN   TestWAL_persist
2022-04-08T12:17:40.5084468Z --- PASS: TestWAL_persist (0.00s)
2022-04-08T12:17:40.5084800Z FAIL
2022-04-08T12:17:40.5085951Z FAIL   github.com/open-telemetry/opentelemetry-collector-contrib/exporter/prometheusremotewriteexporter    82.230s
2022-04-08T12:17:40.5111278Z FAIL
2022-04-08T12:17:40.5231951Z make[2]: *** [../../Makefile.Common:46: test] Error 1
2022-04-08T12:17:40.5233363Z make[2]: Leaving directory '/home/runner/work/opentelemetry-collector-contrib/opentelemetry-collector-contrib/exporter/prometheusremotewriteexporter'
2022-04-08T12:17:40.5234400Z make[1]: *** [Makefile:162: for-all-target-./exporter/prometheusremotewriteexporter] Error 2
2022-04-08T12:17:40.5238879Z make[1]: Leaving directory '/home/runner/work/opentelemetry-collector-contrib/opentelemetry-collector-contrib'
2022-04-08T12:17:40.5241353Z make: *** [Makefile:72: gotest] Error 2
2022-04-08T12:17:40.5259942Z ##[error]Process completed with exit code 2.
jpkrohling commented 2 years ago

cc code owners: @anuraaga @Aneurysm9 @alolita

jpkrohling commented 2 years ago

And here: https://github.com/open-telemetry/opentelemetry-collector-contrib/runs/5889572864?check_suite_focus=true

pmm-sumo commented 2 years ago

This seems to be another occurrence of that here: https://github.com/open-telemetry/opentelemetry-collector-contrib/runs/5973225036?check_suite_focus=true

jpkrohling commented 2 years ago

Another one: https://github.com/open-telemetry/opentelemetry-collector-contrib/runs/5976410911?check_suite_focus=true

gouthamve commented 2 years ago

So I'm looking into this and running this on a M1 Mac:

go test -v . -run=Test_PushMetrics/WAL/intGauge_case -count=10

The above command fails almost always (1-2 failures in the 10 count) with:

    testing.go:1090: TempDir RemoveAll cleanup: unlinkat /var/folders/6n/rn9lc80129z68tqqzd4tyrxm0000gn/T/Test_PushMetricsWALintGauge_case518192279/001/prom_remotewrite: directory not empty

From this thread https://github.com/golang/go/issues/43547, it could be because a background goroutine is writing to the directory even after the end of the test. I tried using goleak to check:

diff --git a/exporter/prometheusremotewriteexporter/exporter_test.go b/exporter/prometheusremotewriteexporter/exporter_test.go
index 0d14b3ed6..d21752045 100644
--- a/exporter/prometheusremotewriteexporter/exporter_test.go
+++ b/exporter/prometheusremotewriteexporter/exporter_test.go
@@ -29,6 +29,8 @@ import (
    "github.com/prometheus/prometheus/prompb"
    "github.com/stretchr/testify/assert"
    "github.com/stretchr/testify/require"
+   "go.uber.org/goleak"
+
    "go.opentelemetry.io/collector/component"
    "go.opentelemetry.io/collector/component/componenttest"
    "go.opentelemetry.io/collector/config"
@@ -618,7 +620,9 @@ func Test_PushMetrics(t *testing.T) {
        t.Run(name, func(t *testing.T) {
            for _, tt := range tests {
                t.Run(tt.name, func(t *testing.T) {
+                   opt := goleak.IgnoreCurrent()
+                   defer goleak.VerifyNone(t, opt)
                    server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
                        if tt.reqTestFunc != nil {
                            tt.reqTestFunc(t, r, tt.expectedTimeSeries, tt.isStaleMarker)

And the failures ALL stopped, even with count=100 🙃

I'm a little lost as to what to do next.

gouthamve commented 2 years ago

(ignore the email sent through this comment, there is a t.Parallel() which caused me to detect extra routines. Removing t.Parallel() doesn't detect any leaks 🤦 )

djaglowski commented 2 years ago

Here again: https://github.com/open-telemetry/opentelemetry-collector-contrib/runs/6028883054?check_suite_focus=true#step:8:3832

jpkrohling commented 2 years ago

And here: https://github.com/open-telemetry/opentelemetry-collector-contrib/runs/6032198410?check_suite_focus=true

github-actions[bot] commented 2 years ago

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

jpkrohling commented 1 year ago

@kovrus, would you like to take a look at this one?

github-actions[bot] commented 1 year ago

Pinging code owners: @Aneurysm9. See Adding Labels via Comments if you do not have permissions to add labels yourself.

github-actions[bot] commented 1 year ago

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

Pinging code owners:

See Adding Labels via Comments if you do not have permissions to add labels yourself.

github-actions[bot] commented 1 year ago

This issue has been closed as inactive because it has been stale for 120 days with no activity.