Proposed benchmarks

[IritR]

This is a thread for proposing benchmarks in response to Data Modeling 8/24/2023

I will add more as I go along, and anyone is invited to contribute.

Here are several FDA-approved drugs for treating Gaucher Disease type1: Eliglustat, Imiglucerase, Taliglucerase alfa, Velaglucerase-alfa, Miglustat. They should all score very high.

Approved for Niemann-Pick type C: miglustat

Approved for Familial hypercholesterolemia homozygous (LDLR): Rosuvastatin Lomitapide Mipomersen Evolocumab

Approved for Familial hypercholesterolemia heterozygous (LDLR): Rosuvastatin Evolocumab

Approved for Familial defective apolipoprotein B (APOB): Rosuvastatin Evolocumab

[andrewsu]

On the subject of benchmark examples, just want to note that we have several benchmarks related to diseases and treatments that have been systematically harvested in the Benchmarks repo. Of particular note are the following:

• DrugCentral: 8675 indications harvested from DrugCentral, mostly for common diseases
• OOPD: 3915 indications harvested from the Orphan Drug Designations and Approvals database created by FDA's Office of Orphan Products Development, mostly for rare diseases
• GTRx: 695 indications from https://gtrx.radygenomiclab.com/ and this publication focused on rare genetic diseases
• IRDiRC: 120 indications from Table 1 of this publication focused on rare diseases

Spot checking the examples above against these sources (though note that some of the "not found" examples may be found under alternate names):

• Gaucher Disease type1

  • Eliglustat: found in OOPD
  • Imiglucerase: found in IRDiRC
  • Taliglucerase alfa: not found ("Taliglucerase" is not found for any indication, so perhaps another name?)
  • Velaglucerase-alfa: not found ("Velaglucerase" is not found for any indication, so perhaps another name?)
  • Miglustat: found in OOPD and IRDiRC

• Niemann-Pick type C

  • miglustat: found in OOPD and GTRx

• Familial hypercholesterolemia homozygous (LDLR)

  • Rosuvastatin: found in DrugCentral, IRDiRC
  • Lomitapide: found in DrugCentral, IRDiRC
  • Mipomersen: found in OOPD for parent disease "Familial hypercholesterolemia"
  • Evolocumab: found in IRDiRC

• Familial hypercholesterolemia heterozygous (LDLR)

  • Rosuvastatin: found in DrugCentral
  • Evolocumab: not found

• Familial defective apolipoprotein B (APOB) -- this disease is not found in any of the benchmarks, so perhaps under a different name

  • Rosuvastatin: not found
  • Evolocumab: not found

[sandrine-muller-research]

Thank you @IritR ! I have added these to the test assets sheet.

[cbizon]

Current benchmarking work is happening in the repo linked above.