Certain overlayVariables give a 500 response for Histogram and Scatter visualisations

[bobular]

I have a histogram viz on GEMS->information of process->Anthropometry->Weight (using bleeding edge dfalke-b back end - and no subsetting filters)

I can use Participant->Demo->Sex or Participant->Case or control as overlayVariable

But with overlayVariable = Participant->Feeding->Given untreated water There is a 500 error

500 Internal Server Error {"status":"server-error","message":"Unable to fetch all required data","requestId":"3mqAOuLMXcYqF85KaiBOkI"}

I noted previously in subsetting that only around 3000 participants have a value for Given untreated water

The same happens for a scatter plot of GEMS->information of process->Anthropometry->Weight vs Height

[d-callan]

i see an error in the merge service: ESC[mESC[32m2021-06-02 13:06:23.421 [rid: ] INFO ClientUtil:51 - Will send following POST request to http://subsetting:8080/studies/GEMSCC0003-1/entities/EUPATH_0000738/tabular {"filters":[],"outputVariableIds":["IAO_0000414"]} ESC[mESC[32m2021-06-02 13:06:23.427 [rid: ] INFO ClientUtil:51 - Will send following POST request to http://subsetting:8080/studies/GEMSCC0003-1/entities/EUPATH_0000096/tabular {"filters":[],"outputVariableIds":["EUPATH_0010151"]} ESC[mESC[32m2021-06-02 13:06:23.546 [rid: ] INFO MergeRequestProcessor:109 - All requested streams (2) ready for consumption ESC[mESC[32m2021-06-02 13:06:23.546 [rid: ] INFO EntityStream:43 - Instantiated TargetEntityStream for entity EUPATH_0000738 ESC[mESC[32m2021-06-02 13:06:23.547 [rid: ] INFO EntityStream:43 - Instantiated RootEntityStream for entity EUPATH_0000096 ESC[mESC[32m2021-06-02 13:06:23.547 [rid: ] INFO MergeRequestProcessor:122 - Writing header row:EUPATH_0000738.InformationOfProcess_stable_id EUPATH_0000096.Participant_stable_id PCO_0000024.Household_stable_id EUPATH_0000738.IAO_0000414 EUPATH_0000096.EUPATH_0010151 ESC[mESC[33m2021-06-02 13:06:23.576 [rid: ] WARN ErrorMapper:79 - Caught Exception: ESC[m java.lang.RuntimeException: Ancestor stream 'EUPATH_0000096' could not provide a row matching 'EUPATH_0000096.Participant_stable_id' with value '1010000081'. at org.veupathdb.service.eda.ms.core.stream.TargetEntityStream.next(TargetEntityStream.java:74) ~[service.jar:1.0.0] at org.veupathdb.service.eda.ms.core.stream.TargetEntityStream.next(TargetEntityStream.java:17) ~[service.jar:1.0.0] at org.veupathdb.service.eda.ms.core.MergeRequestProcessor.writeMergedStream(MergeRequestProcessor.java:127) ~[service.jar:1.0.0] at org.veupathdb.service.eda.ms.core.MergeRequestProcessor.lambda$createMergedResponseSupplier$2(MergeRequestProcessor.java:88) ~[service.jar:1.0.0]

[bobular]

Possibly related:

barplot for GEMS->Information of process->Hospitalization->Outcome leaving hospital with overlay variable Study participant->Feeding->Given stored water for drinking as below - works fine

{
    "config": {
        "outputEntityId": "EUPATH_0000738",
        "overlayVariable": {
            "entityId": "EUPATH_0000096",
            "variableId": "EUPATH_0010141"
        },
        "valueSpec": "count",
        "xAxisVariable": {
            "entityId": "EUPATH_0000738",
            "variableId": "EUPATH_0010353"
        }
    },
    "filters": [],
    "studyId": "GEMSCC0002-1"
}

When changing overlay variable to its sibling "given untreated water"

{
    "config": {
        "outputEntityId": "EUPATH_0000738",
        "overlayVariable": {
            "entityId": "EUPATH_0000096",
            "variableId": "EUPATH_0010151"
        },
        "valueSpec": "count",
        "xAxisVariable": {
            "entityId": "EUPATH_0000738",
            "variableId": "EUPATH_0010353"
        }
    },
    "filters": [],
    "studyId": "GEMSCC0002-1"
}

it gives a 500

500 Internal Server Error {"status":"server-error","message":"Unable to fetch all required data","requestId":"ZrtpeusRUptvtU1mkzfqJ"}

Both are yes/no variables. The latter has more missing data though (25% has data).

[ryanrdoherty]

I believe the original issue here is a duplicate of an issue that has already been fixed. However, there is a current problem on histogram submits. When fixed, this seems to work. I wrote an issue for web-eda to fix: https://github.com/VEuPathDB/web-eda/issues/251