aggregator & data alignment of offline and online processes

[mmagaldi-eng]

In order to align the structure data of the JSON documents stored into fdg_article by both offline and online processes (see also #87), we need to know the EXACT and ACTUAL format of the JSON document that should be stored into ES to eventually fix the aggregator (if needed).

We just know that has been added a new field (Keywords) and I know that we're using two different "score" structures in offline and online processes, but we ALL need the complete structure (see #87).

P.S. Actually, I've also asked by email to update/create a shared document where these data structures can be quickly defined, but I didn't receive any answers,

@neilpbyrne, please forward this issue to Felipe.

[pstalidis]

After the check we detected that:

• the online process was producing a "keywords" field which is not necessary (the values are included in the "about" field from the topic analyzer)
• the offline process was producing the media identifier in the "contains" field while the online was putting media in "images" and "videos" according to the type. We have decided that the distinction is necessary, therefore the output should be split in the offline process to these 2 fields.
• "calculatedRating" and "calculatedRatingDetails" should not be produced by the online process

The final form of the fdg-article index was included in: https://docs.google.com/spreadsheets/d/15cWJwX9D0XuY7zficapz55AoXkC2NfYDowfLSPZSpwE/edit?pli=1#gid=1207208562

Everyone needs to validate that the output of their modules is correct

[dmgutierrez]

@pstalidis @macagari which analyser is going to produce the final document of the article in the Online with the updated format? is it going to be throughout the backend of the UI as before? What about the offline?

[macagari]

@pstalidis I'm going to update the image today with the modifications you suggested.