Uploading an image to a taxa without assigning an Occurrence Record breaks deltaImport

[LordFlashmeow]

The solr deltaQuery gets the ids of updated records since the last import and expects all images to have an occid associated and breaks if one of them doesn't.

As you can see here, the query gets a list of the occid for new images and the occid for new occurrences.

https://github.com/Symbiota/Symbiota/blob/eac45700beddf1d84e2930da1054fd558e0ed4da/config/solr/data-config.xml#L36-L37

However, an image's occid is allowed to be null. When dataImport tries to find the primary key associated with the occid, it fails because the value is null, raising this error:

o.a.s.h.d.DataImporter Delta Import Failed => java.lang.RuntimeException: java.lang.IllegalArgumentException: deltaQuery returned a row with null for primary key occid

https://github.com/Symbiota/Symbiota/blob/eac45700beddf1d84e2930da1054fd558e0ed4da/config/schema-1.0/utf8/db_schema-1.0.sql#L1035-L1067

Steps to reproduce

1. Go to the taxa page for a taxon: /taxa/index.php?taxon=...
2. Click the pencil to edit taxon data
3. Go to the Add Image tab
4. Choose a file, add no other data
5. Upload image
6. The upload will succeed, and is visible and and has an imgid on the taxon page
7. Go to the solr instance at localhost:8983/solr/#/coreName/dataimport and run a delta-import. It will make one request and stop.
8. localhost/solr/#/~logging will report the error
9. The imgid is not searchable on the solr page, despite existing in the database.

If you connect to the sql database, you can verify that the occid is null.

mysql> select occid, imgid FROM images i where i.occid is null;
+-------+--------+
| occid | imgid  |
+-------+--------+
|  NULL | 463111 |
+-------+--------+
1 row in set (0.00 sec)

If you then edit the image and add it to an occurrence record, the dataimport will work fine.

Possible solutions:

• Require an occurrence record when adding images to a taxa
• Skip images will null occid values with something like SELECT occid FROM images i WHERE (i.occid IS NOT NULL AND i.InitialTimeStamp > CONVERT_TZ('${dih.last_index_time}','+00:00','-06:00'))

[evindunn]

Hi @LordFlashmeow

Thanks for pointing this out. We are 2-3 months from the initial public beta release of Symbiota2, so most of our development efforts are currently focused there. Symbiota2 will likely use ElasticSearch rather than SOLR for indexing.

I actually ran into this issue as well and updated the SOLR config for scan-bugs.org with something similar. Hopefully this can fix the issue for now?

Unfortunately we don't have the bandwidth right now to support both Symbiota & the development of Symbiota2. But thanks again for your interest!

-Evin

[neilcobb]

@LordFlashmeow what Symbiota portal are you working on?

-Neil

[LordFlashmeow]

@neilcobb monarch.calacademy.org

[bdbrandt]

@LordFlashmeow I just committed a patch to the solr deltaQuery that should fix the issue. Unfortunately you'll have to transfer the patched code into the active data-config.xml file in your solr core, but once done this issue should be gone.

[LordFlashmeow]

Thanks! We're running solr as a docker service so it's pretty easy to update it.