Same HF set message being processed multiple times

[zabeen]

Same message being processed multiple times causes import failures ("Cannot insert duplicate key row in object 'MatchPrediction.HaplotypeFrequencySets' with unique index 'IX_RegistryCode_And_EthnicityCode'...") which eats up space in the db.

I suspect this may be something to do with lock expiry on messages for long-running HF set imports, which makes the request message available again to other function instances.

[zabeen]

host.json

    "serviceBus": {
      "messageHandlerOptions": {
        "maxConcurrentCalls": 1,
        "maxAutoRenewDuration": "02:00:00"
      }
    }
  },

[zabeen]

Testing

• manually send multiple identical messages to HF import topic (haplotype-frequency-file-uploads) for the same HF set file in quick succession
• outcomes:
  • no alerts should be published during HF set import, i.e., all messages should complete successfully
  • should see multiple rows in MatchPrediction.HaplotypeFrequencySets table for the same file name, all rows should be inactive except for one

[DmitriyShcherbina]

@zabeen Testing status: Ok