AdrianStrugala / AvroConvert

Rapid Avro serializer for C# .NET
Other
102 stars 27 forks source link

AvroConvert.Avro2Json() only getting partial avro as json output #132

Closed RanjithBabu31 closed 7 months ago

RanjithBabu31 commented 11 months ago

I am trying to convert a avro into json. when the input avro contains less then 400 entries, i am able to get the exact json output but when the avro content is higher only 415 records are converted to json.

StringBuilder jsonLineBuilder = new();
                using (MemoryStream ms = new())
                {
                    blobStream.CopyTo(ms);
                    var events = ms.ToArray();
                    var resultJson = AvroConvert.Avro2Json(events);
                    _logger.LogInformation($"{resultJson} : {resultJson.Length}");
                    dynamic serializedObject = JsonConvert.DeserializeObject(resultJson);
                    if (serializedObject.GetType() == typeof(JArray))
                    {
                        dynamic jsonEvents = serializedObject as JArray;
                        _logger.LogInformation($"{serializedObject.Count} : {jsonEvents.Count}");
                        foreach (dynamic evnt in jsonEvents)
                        {
                            var body = Convert.FromBase64String((string)evnt.Body);
                            var jsonStr = Encoding.UTF8.GetString(body);
                            jsonStr = jsonStr.Replace("\r\n", "");
                            jsonLineBuilder.AppendLine(jsonStr);
                        }
                    }
                    else
                    {
                        dynamic jsonEvent = serializedObject as JObject;
                        var body = Convert.FromBase64String((string)jsonEvent.Body);
                        var jsonStr = Encoding.UTF8.GetString(body);
                        jsonStr = jsonStr.Replace("\r\n", "");
                        jsonLineBuilder.AppendLine(jsonStr);
                    }
                }
AdrianStrugala commented 11 months ago

Hello, Thank you for reporting this problem. Looks like a bug, I will take a look for sure. Adrian

AdrianStrugala commented 10 months ago

Hey @RanjithBabu31 , I've run a test for converting 200k objects to avro, then to json and deserializing - the count remained the same. Could you provide more details on that? Best, Adrian

github-actions[bot] commented 7 months ago

Closed due to inactivity.