childe / gohangout

使用 golang 模仿的 Logstash。用于消费 Kafka 数据,处理后写入 ES、Clickhouse 等。
MIT License
1.01k stars 234 forks source link

add opentelemetry output #234

Closed evanzhang87 closed 10 months ago

evanzhang87 commented 10 months ago

use add_fields to build formatted fields, then send to OLPT server, just support log now

const (
    resourcePrefix       = "oltp_resource_"
    scopePrefix          = "oltp_scope_"
    scopeName            = "oltp_scope_name"
    scopeVersion         = "oltp_scope_version"
    scopeSchemaUrl       = "oltp_scope_schemaurl"
    recordPrefix         = "oltp_logrecords_"
    recordTime           = "oltp_logrecords_time"
    recordSeverity       = "oltp_logrecords_severity"
    recordSeverityNumber = "oltp_logrecords_severitynumber"
    recordFlags          = "oltp_logrecords_flags"
    recordTraceId        = "oltp_logrecords_traceid"
    recordSpanId         = "oltp_logrecords_spanid"
    recordBody           = "oltp_logrecords_body"
)
type OLTPOutputConfig struct {
    ServiceAddress string `json:"service_address"` // default localhost:4317
    ClientAuthConfig
    Timeout     int               `json:"timeout"` // default 5s
    Compression string            `json:"compression"` // default none
    Headers     map[string]string `json:"headers"` // grpc headers
}

type ClientAuthConfig struct {
    TLSCA               string `json:"tls_ca"`
    TLSCert             string `json:"tls_cert"`
    TLSKey              string `json:"tls_key"`
    TLSKeyPwd           string `json:"tls_key_pwd"`
    TLSMinVersion       string `json:"tls_min_version"`
    InsecureSkipVerify  bool   `json:"insecure_skip_verify"`
    ServerName          string `json:"tls_server_name"`
    RenegotiationMethod string `json:"tls_renegotiation_method"`
    Enable              *bool  `json:"tls_enable"`
}

update GoVersion to 1.19, it's needed by opentelemetry, and 1.13 was too old, it's time to try new GO!

childe commented 10 months ago

Thanks very much for your PR, but according to my current plan, I'm not intending to add any more OUTPUTs. If you could provide a plugin-style codec & output, I'm pleased to add its link to Readme.

PS: refer to https://go.dev/ref/mod#go-mod-file-go:

The go directive sets the minimum version of Go required to use this module.

codecov-commenter commented 10 months ago

Codecov Report

Attention: 188 lines in your changes are missing coverage. Please review.

Comparison is base (5c04854) 32.62% compared to head (566c3ba) 31.96%.

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## master #234 +/- ## ========================================== - Coverage 32.62% 31.96% -0.66% ========================================== Files 53 55 +2 Lines 4377 4611 +234 ========================================== + Hits 1428 1474 +46 - Misses 2794 2973 +179 - Partials 155 164 +9 ``` | [Flag](https://app.codecov.io/gh/childe/gohangout/pull/234/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None) | Coverage Δ | | |---|---|---| | [unittests](https://app.codecov.io/gh/childe/gohangout/pull/234/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None) | `31.96% <19.65%> (-0.66%)` | :arrow_down: | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None#carryforward-flags-in-the-pull-request-comment) to find out more. | [Files](https://app.codecov.io/gh/childe/gohangout/pull/234?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None) | Coverage Δ | | |---|---|---| | [codec/oltp\_encoder.go](https://app.codecov.io/gh/childe/gohangout/pull/234?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None#diff-Y29kZWMvb2x0cF9lbmNvZGVyLmdv) | `0.00% <0.00%> (ø)` | | | [output/oltp\_output.go](https://app.codecov.io/gh/childe/gohangout/pull/234?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None#diff-b3V0cHV0L29sdHBfb3V0cHV0Lmdv) | `29.11% <29.11%> (ø)` | |

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

evanzhang87 commented 10 months ago

That's ok, learn from you!