search

openaire / iis

Information Inference Service of the OpenAIRE system
Apache License 2.0
20 stars 11 forks source link

Closes #1417: Introduce missing integration test for avro to protobuf workflow #1425

Closed marekhorst closed 1 year ago

marekhorst commented 1 year ago

This PR introduces missing test verifying correctness of agro -> protobuf transformation workflow.

The major difficulty was to implement a testing consumer verifying whether produced protobuf outcome is built according to predefined expectations. Usually we rely on a simple java class (Process in the IIS nomenclature) but it turned out to be rather cumbersome to veriify protocol buffer record this way because a lot of new protobuf related code seemed to be required to be written in order to introduce protobuf-compatible TestingConsumer. So I decided to implement less generic, spark based but very simple PersonProtoConsumer which does the job.

marekhorst commented 1 year ago

I have introduced the proposed changes, extended the commit message with PR description and merged with the master branch with the following commit: https://github.com/openaire/iis/commit/e807df7bd5f0c5127f9c295243558e9b33884e63