Drug warnings widget not showing records due to API grouping

[d0choa]

@ireneisdoomed identified that for the drug CHEMBL418995 the platform is displaying 2 records when there are 3 records in the data:

ETL output:

In [64]: dw_process.withColumn("chemblId", f.explode("chemblIds")).filter(f.col("chemblId") == "CHEMBL418995").show()
+--------------------+--------------+--------------------+--------------------+----+--------------------+-----------+----+--------------------+-------------+------------------------+-------------+------------+
|       toxicityClass|     chemblIds|             country|         description|  id|          references|warningType|year|            efo_term|       efo_id|efo_id_for_warning_class|meddraSocCode|    chemblId|
+--------------------+--------------+--------------------+--------------------+----+--------------------+-----------+----+--------------------+-------------+------------------------+-------------+------------+
|              misuse|[CHEMBL418995]|France; Thailand;...|Potential for abu...|2874|[{EDM-QSM-2001.3,...|  Withdrawn|1999|     substance abuse|MONDO:0002491|             EFO:0011049|         null|CHEMBL418995|
|dermatological to...|[CHEMBL418995]|France; Thailand;...|Abuse; Dependence...|3502|[{10.1177/0092861...|  Withdrawn|1999|                acne|  EFO:0003894|             EFO:0011048|         null|CHEMBL418995|
|              misuse|[CHEMBL418995]|France; Thailand;...|Potential for abu...|3383|[{EDM-QSM-2001.3,...|  Withdrawn|1999|substance dependence|MONDO:0004938|             EFO:0011049|         null|CHEMBL418995|
+--------------------+--------------+--------------------+--------------------+----+--------------------+-----------+----+--------------------+-------------+------------------------+-------------+------------+

API query:

query q{
  drug(chemblId: "CHEMBL418995"){
    id
    name
    drugWarnings{
      id
      toxicityClass
      warningType
      description
    }
  }
}

API response:

{
  "data": {
    "drug": {
      "id": "CHEMBL418995",
      "name": "AMINEPTINE",
      "drugWarnings": [
        {
          "id": 3502,
          "toxicityClass": "dermatological toxicity",
          "warningType": "Withdrawn",
          "description": "Abuse; Dependence; Severe Acne"
        },
        {
          "id": 3383,
          "toxicityClass": "misuse",
          "warningType": "Withdrawn",
          "description": "Potential for abuse and risk of dependence; Follows the decision taken by France to suspend amineptine on the basis of abuse and dependency potential"
        }
      ]
    }
  }
}

This is the consequence of an incorrect grouping strategy agreed here: opentargets/platform#1506.

We believe the issue would be fixed in the API code here if we use as unique field the column id instead of the combination of warningType and toxicityClass.

[remo87]

There're 3 elements in elastic search but the API is aggregating by warningtype and toxicityclass. If we query the API including the references we'll see that misuse has 2 references:

query:

query q{
  drug(chemblId: "CHEMBL418995"){
    id
    name
    drugWarnings{
      id
      toxicityClass
      warningType
      description
      references {
        id
        source
        url
      }
    }
  }
}

reponse:

{
  "data": {
    "drug": {
      "id": "CHEMBL418995",
      "name": "AMINEPTINE",
      "drugWarnings": [
        {
          "id": 3502,
          "toxicityClass": "dermatological toxicity",
          "warningType": "Withdrawn",
          "description": "Abuse; Dependence; Severe Acne",
          "references": [
            {
              "id": "10.1177/009286150103500134",
              "source": "DOI",
              "url": "https://doi.org/10.1177/009286150103500134"
            }
          ]
        },
        {
          "id": 3383,
          "toxicityClass": "misuse",
          "warningType": "Withdrawn",
          "description": "Potential for abuse and risk of dependence; Follows the decision taken by France to suspend amineptine on the basis of abuse and dependency potential",
          "references": [
            {
              "id": "EDM-QSM-2001.3",
              "source": "WHO",
              "url": "https://www.who.int/publications/i/item/EDM-QSM-2001.3"
            },
            {
              "id": "EDM-QSM-2001.3",
              "source": "WHO",
              "url": "https://www.who.int/publications/i/item/EDM-QSM-2001.3"
            }
          ]
        }
      ]
    }
  }
}

[d0choa]

Resolved in dev