pstalidis
commented
3 years ago the mapping that I used to create the fusion score index in elasticsearch is:
"mappings": { "properties": { "identifier": {"type": "keyword"}, "analyzer": {"type": "text"}, "calculatedRating": {"type": "float"}, "calculatedRatingDetail": {"type": "text"}, "timestamp": {"type": "date"} } }
which defines calculatedRatingDetail as text. This is the format used for storing the object from the analyser and what the online service returns. When was this changed?
In any case, let's decide on if we change the mapping or the object.
@neilpbyrne is there any reason to have this field as an object?
We should check that services are returning the right structures for the data model on elastic
For example, the mapping on elastic is this one: "calculatedRating" : { "type" : "float" }, "calculatedRatingDetail" : { "properties" : { "authorRating" : { "properties" : { "ratings" : { "type" : "text", "fields" : { "keyword" : { "type" : "keyword", "ignore_above" : 256 } } } } }, "publisherRating" : { "properties" : { "ratings" : { "type" : "text", "fields" : { "keyword" : { "type" : "keyword", "ignore_above" : 256 } } } } }, "textRating" : { "type" : "float" } }
fusionscore serivece return this, so but in the fields there not text, there is a object so I'm afraid that the offline and the online process are inconsistent in terms of what the final version of the document is on elastic
{'data': {'analyzer': 'Overall Score', 'calculatedRating': 0.0,
'calculatedRatingDetail':
'{"textRating": {"trustworthiness": 0.0, "relevance": 0.0},
"authorRating": {"trustworthiness": 0.0, "relevance": 0.0},
"publisherRating": {"trustworthiness": 0.0, "relevance": 0.0}}',
'identifier': '5c03fbd2cdd66f1c306416f19895aab5f48d0faf7b2447c4dc0290b35da15542c5ad2682f65124d61502c61d3ee5e6e4e8343ca74ed98ffa88a15c38865fb6df', 'timestamp': 'Fri, 16 Oct 2020 13:57:51 GMT'},
'message': 'Successful Operation', 'status': 200}
Let me know what you think, maybe I'm just missing something.