Response's classification results ordering by span behaving inconsistently

[pmcjr]

Describe the bug

When calling /api/v1/task/classification-with-text-generation with multiple models present in guardrail_config.input objects in the payload, response's token_classification_results.input ordering by span are not consistent.

Platform

Please provide details about the environment you are using, including the following:

fms-orchestrator image version: fms-guardrails-orchestr8:ee4dd81

Observed behavior

Request body for both calls

{
  "inputs": "I hate cats they are stupid. I love dogs love@dogs.com. Rabbits are pretty but",
  "model_id": "bloom-560m",
  "guardrail_config": {
    "input": {
      "models": {
        "en_syntax_slate.38m.hap": {
          "threshold": 0.5
        },
        "en_syntax_rbr_pii": {
          "threshold": 0.8
        }
      }
    },
    "output": {
      "models": {}
    }
  },
  "text_gen_parameters": {
    "preserve_input_text": true,
    "max_new_tokens": 99,
    "min_new_tokens": 1,
    "truncate_input_tokens": 500,
    "decoding_method": "SAMPLING",
    "top_k": 2,
    "top_p": 0.9,
    "typical_p": 0.5,
    "temperature": 0.8,
    "seed": 42,
    "repetition_penalty": 2,
    "max_time": 0,
    "stop_sequences": [
      "42"
    ]
  }
}

Response

Response 1

Response 2

[evaline-ju]

Since the detectors are called in parallel, some detectors may return detections first before others (potentially also depending on how loaded each detector service is) leading to this "inconsistency"- we should consider if there should be sorting after all detections, such as based on start