Failed to transform Event without Data

[AdrianM]

There are some exceptional events in our EventStore without the 'Data' attribute, which results in a failure of the replicator. The solution would be, to create a function 'ParseData' similar to 'ParseMeta' in the file 'JsTransform.cs' (around line 81, 'parser.Parse(original.Data.AsUtf8String())'). I guess it is discussable if there should be events without Data, but as long as the eventstore allows to store them, it would be nice if the replicator could handle them (it would work for me if the replicator gives the event to the JS transform section an there I will add an empty Data object to each such event).

Setup: Replicator 0.4.5, EventStore reader 20.10, EventStore sink 21.10, on Kubernetes with HELM)

The stacktrace is:

{"@t":"2022-04-07T18:23:09.6255774Z","@m":"Error occured in the \"PrepareContext\" pipe: \"One or more errors occurred. (Line 1: Unexpected end of input)\"","@i":"c9acd0a5","@l":"Error","@x":"System.AggregateException: One or more errors occurred. (Line 1: Unexpected end of input)
    ---> Esprima.ParserException: Line 1: Unexpected end of input
    at Jint.Native.Json.JsonParser.ThrowError(Token token, String messageFormat, Object[] arguments)
    at Jint.Native.Json.JsonParser.ThrowUnexpected(Token token)
    at Jint.Native.Json.JsonParser.ParseJsonValue()
    at Jint.Native.Json.JsonParser.Parse(String code, ParserOptions options)
    at EventStore.Replicator.JavaScript.JsTransform.Transform(OriginalEvent original, CancellationToken cancellationToken) in /app/src/EventStore.Replicator.JavaScript/JsTransform.cs:line 95
    at EventStore.Replicator.Prepare.TransformFilter.<>c__DisplayClass2_0.<<Send>g__Transform|2>d.MoveNext() in /app/src/EventStore.Replicator/Prepare/TransformFilter.cs:line 39
    --- End of inner exception stack trace ---
    at System.Threading.Tasks.Task.ThrowIfExceptional(Boolean includeTaskCanceledExceptions)
    at System.Threading.Tasks.Task`1.GetResultCore(Boolean waitCompletionNotification)
    at Ubiquitous.Metrics.Metrics.MeasureTask[T](Func`1 action, IHistogramMetric metric, ICountMetric errorCount, String[] labels, Int32 count)
    at EventStore.Replicator.Prepare.TransformFilter.Send(PrepareContext context, IPipe`1 next) in /app/src/EventStore.Replicator/Prepare/TransformFilter.cs:line 40
    at EventStore.Replicator.Prepare.EventFilterFilter.Send(PrepareContext context, IPipe`1 next) in /app/src/EventStore.Replicator/Prepare/EventFilterFilter.cs:line 25
    at GreenPipes.Filters.ConcurrencyLimitFilter`1.Send(TContext context, IPipe`1 next)
    at EventStore.Replicator.LoggingFilter`1.Send(T context, IPipe`1 next) in /app/src/EventStore.Replicator/Logging.cs:line 17","Type":"PrepareContext","Message":"One or more errors occurred. (Line 1: Unexpected end of input)","SourceContext":"EventStore.Replicator.LoggingFilter`1[T]"}

Regards, Adrian

[alexeyzimarev]

It's strange that you got events with empty data accepted by ESDB. As far as I know, the minimal requirement for an event is to have some valid JSON. Of course, it might have been introduced in some later version, after you got those events appended. I will try to fix the issue. I'll check if I can append an empty event to 20.10.

[AdrianM]

I checked it with the colleagues, the reason for the empty events were some problems with EventStore GUI in the beginning. In some cases, when an event was manually added in the UI, the JSON was somehow not taken and the result was an empty event added on the stream.

Just some additional inputs to this topic: It would be nice if there is an error message with the stream name in it as soon as something failed (e.g. at the moment, I got only the error about the parsing exception, but not on which stream or event it occurred). And it would be helpful if the revision number of the event would be accessible in the JS transform method.

[alexeyzimarev]

Ok, I'll see what I can do. I agree that some additional diagnostics would be nice.

[AdrianM]

Thanks for the fix! As soon as the Helm chart is available, I will use the new Version for our production replication