Bug-Batch Update failing to update correct batch

[ChrisGrinter]

Since our last pull of BioKIC for Monarch, Batch Update tool is not accurately updating batches. For example I run a search for sciname = Alobates pensylvanicus for 64 results. Batch Update Family = Tenebrionidae and it warns me that I'm about to change 79 results. I have no idea what those unlisted 15 records are, it does not correspond to all records without families in the database. It doesn't correspond to all Alobates without family either. This error has completely disabled Batch Update since we have no idea what records are being updated.

[dbettman]

Can confirm, I'm hitting the same problem (I'm using the same BioKIC/Monarch version as @ChrisGrinter). In the example screenshot below, the search I ran was "Family = Notodontidae AND Scientific Name starts with Cerura". You can see that the bulk edit should only be changing 16 records, but it was instead trying to change 215 records. I have no idea what those 199 other records could be (there are 1448 total records where "Family = Notodontidae" and 17 total records where "Scientific Name starts with Cerura" in our database):

Note that there was only a single Monarch tab open in my browser, so this wasn't caused by anything like e.g. having different Monarch tabs with different searches open at the same time. I am not yet able to find any pattern that gives insight into what is causing the bulk edit feature to misbehave.

[themerekat]

@ChrisGrinter and @dbettman , I'm looking into this issue but am currently unable to replicate it on our portals. Does this happen when you are trying to change any field or just some of them?

[dbettman]

@themerekat thanks for looking into this! I did some additional testing this morning and can now reliably characterize and reproduce the problem; it happens when trying to change any field, but only when the field you are trying to change is not the field you searched on. So e.g. in my previous screenshot, where the search results of the boolean "Family = Notodontidae AND Scientific Name starts with Cerura" were displayed, it showed 16 results. However, the bulk edit tried to change 215 results. It turns out that the simple search "ID Qualifier = Notodontinae" yields 215 results (i.e., the Current Value that we are asking the bulk editor to change). So the bulk editor is ignoring the search constraints of "Family = Notodontidae AND Scientific Name starts with Cerura" and is only looking at how many records in the entire database have the current ID Qualifier value of "Notodontinae".

So suppose I just do a simple search of "ID Qualifier = Notodontinae". I get the expected 215 results, and if I try to bulk edit the ID Qualifier field, then everything works as expected. However, if I try to bulk edit any field other than the field I searched on (in this case, the ID Qualifier field) then the bulk edit tries to change the wrong number of results again, because it is completely ignoring the search parameters and is only considering how many records in the database have the Current Value you've provided to the bulk editor, given the Field Name you've provided to the bulk editor. So, for example, in this simple search of "ID Qualifier = Notodontinae" that gives 215 results, if I open the bulk editor and set the Field Name to "Family" and try to change the 215 records in the search results from the Current Value of "Notodontidae" to anything else, the bulk editor will try to change 1448 records, which is how many records in the entire database have the current Family value of "Notodontidae". If I set the field name to "Country" and try to change the 215 records in the search results from the current value of "Costa Rica" to anything else, the bulk editor will try to change 5289 records, which is how many records in the entire database have the current Country value of "Costa Rica". Etc. etc. etc.

One possible complication is that the above explanation contradicts what @ChrisGrinter says in his report above, since he says the 79 records the bulk editor tried to change do not correspond to the number of records in the entire database where Family = NULL.. I am hoping he was simply mistaken about that, but if that's not the case, then the explanation is not as simple as what I wrote above. (I just redid his search and the results I get are consistent with my explanation, i.e. the records the bulk editor tried to change are precisely the number of records in the entire database where "Family = NULL"; so I'm reasonably confident my explanation is correct.)

I'm happy to provide screenshots of any of the above, or additional information if needed. Hopefully you can reproduce this on your end!

[themerekat]

@dbettman thanks for the additional information! When was the last time Monarch's code was updated? I'm wondering if this is an old problem that was fixed in more recent updates. Have you contacted Jon Fong? I believe he is at least one person who maintains your codebase.

[dbettman]

@themerekat Monarch was last updated to current BioKIC on August 10, 2022. Were there changes to the BioKIC batch updating code since then? Updating to latest BioKIC is somewhat painful for us until issue 226 is implemented--that's probably our biggest delta from upstream BioKIC.

[ChrisGrinter]

@themerekat Just wondering if there was an update after August that may be necessary for us to update to? Or if you've been able to replicate on BioKIC?

[themerekat]

@ChrisGrinter , I haven't been able to replicate this issue on any of our Symbiota instances, so I'm still scratching my head!

[ChrisGrinter]

@themerekat could you set us up with a dummy account to test things somewhere? That might help us isolate issues that are on our particular version (and our developer Joe has built in some tools to get around tasks we do not like, so this bug must lie in our tables somewhere).

[themerekat]

@ChrisGrinter just sent you a message via Slack