Speeds up queries by checking field grammar at runtime

[adkron]

Often, the queries we make are not coming from a user's input and are hardcoded by developers. Using a macro to build those queries has two advantages. The first is correctness checking and developer joy. The second is throughput. Third is a small memory decrease.

When grammars are turned into indices, they are parsed from a string into a list. The parsing of the string can fail if the string is not properly formatted. The sigil checks the string format at compile time instead of runtime. That catches errors early and reduces developer mistakes.

Throughput when using precompiled grammars was increased on multiple platforms. I used Benchee on an M1 and M2 Mac. The M1 with a String was 1.29x slower. On the M2, the increase was even better, with the string version being 3.65 times slower.

Memory consumption had a similar benefit. M1 Mac was 1.05x larger on M1 and 2.15x larger on the M2. I was surprised but ran the benchmarks multiple times with similar results.

Further gains could be had by removing the ability to use Strings so that no extra function heads need to be checked.

[adkron]

import HL7.FieldGrammar, only: :macros

hl7 =     
  """
  MSH|^~\\&|MegaReg|XYZHospC|SuperOE|XYZImgCtr|20060529090131-0500||ADT^A01^ADT_A01|01052901|P|2.5
  EVN||200605290901||||200605290900
  PID|||56782445^^^UAReg^PI||KLEINSAMPLE^BARRY^Q^JR||19620910|M||2028-9^^HL70005^RA99113^^XYZ|260 GOODWIN CREST DRIVE^^BIRMINGHAM^AL^35209^^M~NICKELL’S PICKLES^10000 W 100TH AVE^BIRMINGHAM^AL^35200^^O|||||||0105I30001^^^99DEF^AN
  PV1||I|W^389^1^UABH^^^^3||||12345^MORGAN^REX^J^^^MD^0010^UAMC^L||67890^GRAINGER^LUCY^X^^^MD^0010^UAMC^L|MED|||||A0||13579^POTTER^SHERMAN^T^^^MD^0010^UAMC^L|||||||||||||||||||||||||||200605290900
  OBX|1|N^K&M|^Body Height||1.80|m^Meter^ISO+|||||F
  OBX|2|NM|^Body Weight||79|kg^Kilogram^ISO+|||||F
  AL1|1||^ASPIRIN
  DG1|1||786.50^CHEST PAIN, UNSPECIFIED^I9|||A
  """
  |> String.replace("\n", "\r")

indicies = ~g{PID-3.1}
string = "PID-3.1"

Benchee.run(
  %{
    "with sigil" => fn ->
      HL7.Query.get_part(hl7, indicies)
    end,
    "with string" => fn->
      HL7.Query.get_part(hl7, string)
    end    
  },
  time: 20,
  memory_time: 2
)

This is the benchmark I used.

[adkron]

I am going to make a change here that will still support Strings coming in and compile them at runtime so that we won't have to use the sigil if we don't want to do that. It will still be able to take variables at runtime, but anything hard coded with automatically be checked and converted.

[adkron]

Closing in favor of #29