search

msberends / AMR

Functions to simplify and standardise antimicrobial resistance (AMR) data analysis and to work with microbial and antimicrobial properties by using evidence-based methods, as described in https://doi.org/10.18637/jss.v104.i03.
https://msberends.github.io/AMR/
Other
83 stars 12 forks source link

Veterinary breakpoint substitutions incorrect #142

Closed samueljpearce closed 1 week ago

samueljpearce commented 6 months ago

I've been working on creating a complete antibiogram with breakpoint substitutions as per CLSI VET09. Now the current method for substitution is not in line with the VET09 but that is not the problem I am referring to. I highlighted part of this issue in issue #141, but have discovered that was only part of a wider problem with how this package substitutes breakpoints where host-specific breakpoints are not available.

As referred to in #141, I discovered that for UTI isolates that did not have host-specific breakpoints, the package would substitute soft tissue breakpoints instead of urinary tract breakpoints from another species. Perhaps this is the correct thing to do as we can't guarantee UTI breakpoints to be the same between host species, but it then blanket applies this breakpoint to all UTI isolates, even where host specific UTI breakpoints exist, incorrectly identifying them as resistant.

The second part of this issue that I have discovered is that if there is not a host species-specific breakpoint for a particular bug-drug combination, but there are genus level breakpoints that can be extrapolated, instead of substituting those genus level breakpoints for that host species, it often uses a different host species breakpoint. The odd thing is that this is not consistent for all antimicrobials for some reason. For example, using the minimal example data from #141, the following two lines of code evaluate a feline S. aureus isolate and canine S. pseudintermedius isolate. Neither has a host species-specific breakpoint for that Staphylococcus species, but both have generic genus-wide Staphylococcus spp. breakpoints.

Oddly, sometimes it substitutes the genus breakpoint correctly for that host species; sometimes it says they're not available so substitutes in a genus breakpoint from a different host species; and sometimes it returns ℹ Animal hosts not set in host, assuming host = "dogs", since these have the highest breakpoint availability., even though it clearly has interpreted host species for other antimicrobials breakpoints.

samueljpearce commented 6 months ago

This issue for staph/strep also seems to be related to the uti issue.

For the second line: as.sir(minimal_example_data[49, ], guideline = "CLSI", breakpoint_type = "animal", host = "host"))

There is a 'skin, urinary tract infection' pradofloxacin breakpoint for S. pseudintermedius in dogs, but it is not being converted to SIR unless UTI = TRUE. The same goes for the genus breakpoints - they will only work if, the UTI columns match that of the clinical breakpoints table. For example, there is no marbofloxacin breakpoint for S. pseudintermedius but there is a Staphylococcus spp. breakpoint for 'Skin, soft tissue, uncomplicated urinary tract infection', but samples are only interpreted if UTI = TRUE, like that of the breakpoint file, even though they should be applied to skin and soft tissue also.

msberends commented 6 months ago

Thanks so much for diving into this, that’s really helpful.

Yes, definitely some bugs to fix. I’ll look into it this or next week and then I’ll get back to you.

Thanks again!

samueljpearce commented 6 months ago

Just to add to this UTI substitution problem. Clindamycin with Staph. pseudintermedius is a good example where there are only canine breakpoints for soft tissue sites. In theory it should substitute in these breakpoints for urinary tract infections, but it does not. It seems that the function is looking for substitutions from other host species first, and because their are none with a breakpoint, it doesn't interpret the MIC at all.

msberends commented 3 months ago

Could you please check this again with the latest beta version? I updated the code for veterinary interpretation a lot. Curious for your thoughts.

samueljpearce commented 2 months ago

Will be coming back to this in the next few weeks so will take a look. Thanks again for your work on this!

samueljpearce commented 1 month ago

So it's better, although the output log is a little confusing and it's quite confusing to get my head around where it is going wrong. To summarise my thoughts on it now:

  1. It is correctly selecting a genus/family/order level breakpoint where a species specific one does not exist.
  2. Using the same examples and code above for isolates where UTI is specified as false and a bacterial species specific breakpoint does not exist, I get this message (for various antibiotics) Breakpoints for enrofloxacin (ENR) in Staphylococcus pseudintermedius are only available for (uncomplicated) urinary tract infections (UTI); assuming uti = TRUE.. Now this doesn't apply incorrect breakpoints because that genus level breakpoint is for 'skin, soft tissue and uncomplicated UTI' but given that the isolate specifies UTI = FALSE, it seems illogical to assume UTI=TRUE to find and apply a breakpoint (and incorrectly state there is only one for UTIs). Would there ever be a situation where there is a separate "skin, soft tissue" and "UTI" breakpoints for a genus and therefore the UTI breakpoint is picked preferentially, applying less appropriate breakpoints?
  3. The behaviour mentioned in https://github.com/msberends/AMR/issues/141#issuecomment-2000427647 is arguably more strange than before. I think (not 100% sure) that at least the patient species works properly now and cat/dog isolates are interpreted with their own breakpoints - I did try to force this by removing feline breakpoints like the previously missing breakpoint issue and it didn't seem to repeat the previous problem. However, using that same minimal dataset, it seems that isolates of the same bacterial species are not being interpreted with different breakpoints (e.g. soft tissue vs UTI) within the same patient species. A reproducible example would be to compare the difference between as.sir(minimal_reproducible_example[1:16, ], guideline = "CLSI 2023", breakpoint_type = "animal", verbose = TRUE) and as.sir(minimal_reproducible_example[1:17, ], guideline = "CLSI 2023", breakpoint_type = "animal", verbose = TRUE). As soon as that 17th row, a non-UTI E. coli isolate, is included, all the previous UTI E. coli isolates which were identified correctly as 'S' are now identified as 'R'. This is not elaborated on in the logs either to figure out why this is.

Also, unrelated to these issues, when I specify host = "host" it does not identify that as a column name and thinks that is an incorrect species. Runs fine and identified the correct column when it is not specified but I assume that is not intended behaviour.

That's as much as I can look into for now, but obviously that's quite a serious issue, but easy to workaround by interpreting UTI and other samples separately before rejoining the data.

samueljpearce commented 1 month ago

Just noticed that I think my second point is related to #163

samueljpearce commented 3 weeks ago

Sorry, I've found another problem to add:

Breakpoint substitutions seem to be coming from the latest dataset, not the specific breakpoint dataset. For example, I'm still using CLSI 2023 breakpoints and specifying that in as.sir. However, I get this below. Thing is, these don't exist for cats (and there are not genus/order breakpoints for Enterobacterales) in the CLSI 2023 dataset. I think it is drawing these from the Enterobacterales order breakpoints in the 2024 breakpoints. Interpreting MIC values: column 'AMP' (ampicillin), CLSI 2023... NOTE • Using cats breakpoints since dogs for ampicillin (AMP) in Proteus mirabilis are not available. • Using cats breakpoints since dogs for ampicillin (AMP) in Klebsiella pneumoniae pneumoniae are not available. • Using cats breakpoints since dogs for ampicillin (AMP) in Staphylococcus pseudintermedius are not available. • Using cats breakpoints since dogs for ampicillin (AMP) in Enterobacter cloacae complex are not available.

msberends commented 1 week ago

This is now fixed in the latest beta. Specifically, we moved from using the most prevalent breakpoints to support CLSI VET09.