Hitesh031194 / ports_processor

0 stars 0 forks source link

Json stream #2

Open johnnywidth opened 2 years ago

johnnywidth commented 2 years ago

using of interface type for json stream is overload data serialization, also, potentially lead to fail if json schema out of strict model

Hitesh031194 commented 2 years ago

The reason I used interface was to make it a generic type. so that it can be used as a package. I will notify if I can make it any better.

Hitesh031194 commented 2 years ago

using of interface type for json stream is overload data serialization, also, potentially lead to fail if json schema out of strict model

since I am marshalling and unmarshalling a single port data coming out from a generic json stream, I do not see a potential of failure due to json schema not adhering to model.

https://github.com/Hitesh031194/ports_processor/blob/89777cd45ad31729c9801d120abccbbb6b0caea6/internal/app/app.go#L48

Also if you take a look at the integration test: https://github.com/Hitesh031194/ports_processor/blob/89777cd45ad31729c9801d120abccbbb6b0caea6/test/integration/ports_processor/port_processor_test.go#L58

Both the ports were inserted despite some missing data in the json schema.

Potential data loss might come up if the golang model uses strict json tags.

Hitesh031194 commented 2 years ago

using of interface type for json stream is overload data serialization, also, potentially lead to fail if json schema out of strict model

Pardon, I did not understand 'overload' data serialization.

johnnywidth commented 2 years ago

I would say that function handlePortData could be a part of interface and this interface says "I want to receive specific model, please implement it". interface type not so obvious and potentially might lead to incorrect result, and require double serialization as a result on the caller side

Hitesh031194 commented 2 years ago

I would say that function handlePortData could be a part of interface and this interface says "I want to receive specific model, please implement it". interface type not so obvious and potentially might lead to incorrect result, and require double serialization as a result on the caller side

I understand the 'double seriallization' issue. But handling of data depends from caller to caller. Handling cannot be generic and hence may not be part of 'pkg'. I am not so sure about encountering incorrect result because of using interface. Maybe we can write a test case to check if that can happen. Also 'double seriallization' wont add to latency here much as we are handling small chunk of data at a time.