raystack / raccoon

Raccoon is a high-throughput, low-latency service to collect events in real-time from your web, mobile apps, and services using multiple network protocols.
https://raystack.github.io/raccoon/
Apache License 2.0
199 stars 29 forks source link

Support for serialised data in raccoon client #100

Open punit-kulal opened 1 month ago

punit-kulal commented 1 month ago

Summary Currently, the raccoon, client supports in built serialization for either PROTO(ProtoMessage Object), or JSON. This means for already serialised data like in case for users of Stencil, they have to explicitly parse the message into a DynamicMessage for it to be of serializer Proto. We need to add support for a Serializer.RAW which forwards the message as it is without explicitly parsing the message they have serialised.

Proposed solution Introduce Serialiser.RAW which gives the input as it is after checking for type safety


// SerializerFunc defines a conversion for raccoon message to byte sequence.
type SerializerFunc func(interface{}) ([]byte, error)

var (
    // json raccoon message serializer
    JSON = json.Marshal

    // proto raccoon message serializer
    PROTO = func(m interface{}) ([]byte, error) {
        msg, ok := m.(proto.Message)
        if !ok {
            return nil, errors.New("unable to marshal non proto")
        }
        return proto.Marshal(msg)
    }

      // New serialiser type proposed.
       RAW = func(m interface{}) ([]byte, error) {
            typedM, ok := m.([]byte)
            if !ok {
                return nil, errors.New("failed to convert to byte array")
            }
           return typedM, nil
       } 

)

Additional context Currently for stencil users,

        stencilUrl := fmt.Sprintf(stencilSchemaTemplate, namespace, schemaName)
    client, err := stencil.NewClient([]string{stencilUrl}, stencil.Options{})
    if err != nil {
        return nil, err
    }
    serialisedData, err := client.Serialize(packageName, data) // is of type []byte
    protoMessage, err := client.Parse(packageName, serialisedData) // additional step for converting the data to a protoMessage.
    if err != nil {
        return nil, err
    }
    return protoMessage, err

instead of directly returning the serialisedData.