MAAP-Project / maap-hec-aws

2 stars 1 forks source link

R1: updates to figaro faceted search UI to support showing which ADES cluster each job is from #14

Closed marjo-luc closed 2 years ago

marjo-luc commented 2 years ago

job.endpoint_id describes the ADES cluster on which the given job was executed. This is the field to expose. Screen Shot 2022-05-05 at 2.31.59 PM.png

marjo-luc commented 2 years ago

The hysds_ui needed to be updated to retrieve documents from the test index that was created. This was done by adding the following to the hysds_ui config.js:

exports.MOZART_ES_INDICES = "job_status,new_job_status";

It looks like the index is being accessed, but Figaro does not show the data from the new index. Looking at the network traffic shows this elasticsearch error:

"reason": { "type": "illegal_argument_exception", "reason": "Fielddata is disabled on text fields by default. Set fielddata=true on [resource] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory. Alternatively use a keyword field instead." }

marjo-luc commented 2 years ago

The core issue wound up being improper mapping on the ES index. I created a new index called new_job_status and used the same mapping the job_status index uses except I added a field for the ades endpoint. Users can now facet jobs on the ades endpoint they were run on, if that information is available within the document.

Relevant links: https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-put-mapping.html#updating-field-mappings