Error when label is missing `pds:Time_Coordinates/pds:start_date_time` attribute

[jimmie]

💡 Description

During upgrades to 1.1.11, a NPE was observed when a /products?limit=15 was issued. Log file attached contains the stack trace and the record which created it (which needs to be identified amongst the result set from Opensearch). Unfortunately, after the registry index was re-sharded, this error no longer appeared - likely because the 'default' document order prevented it from appearing in the results.

We need to determine if the NPE was created due to a shortcoming in the code or if the document is lacking in some way. If the latter, we need to try to figure out if it is in fact an invalid document or if some data migration needs to be performed.

Log file: sbnpsi-log-events-20221213.csv

[jimmie]

This is a task: need to create bug or enhancement tickets as a result of this investigation.

[jimmie]

@tloubrieu-jpl also observed the NPE from a /bundles request to SBNUMD. Log attached.

sbnumd-log-events-20221215.csv

[tloubrieu-jpl]

The bug might come of this bundle (and others) which don't have a pds:Time_Coordinates/pds:start_date_time.

The record in opensearch is: { "_index": "registry", "_type": "_doc", "_id": "urn:nasa:pds:soho::1.0", "_score": 7.247081, "_source": { "pds:Citation_Information/pds:description": "This archive bundle contains collections of comet observations and derived results from the SOHO data archives, with related documentation.", "ops:Label_File_Info/ops:creation_date_time": "2020-02-07T15:22:56Z", "pds:Modification_Detail/pds:version_id": "1.0", "lid": "urn:nasa:pds:soho", "description": [ "This archive bundle contains collections of comet observations and derived results from the SOHO data archives, with related documentation.", "A.C.Raugh: Creation for delivery of first data collection. Some of the existing documentation from the PDS3 SOHO/LASCO data set has been included in support of the initial SWAN derived results collection." ], "pds:Bundle_Member_Entry/pds:reference_type": [ "bundle_has_document_collection", "bundle_has_data_collection" ], "title": "Comet Data from the Solar and Heliospheric Observatory (SOHO)", "_package_id": "0131dc16-8479-45e0-a0da-1d0dbd61ed1d", "ops:Harvest_Info/ops:node_name": "PDS_SBN", "ref_lid_collection": [ "urn:nasa:pds:soho:document", "urn:nasa:pds:soho:swan_derived" ], "pds:Bundle_Member_Entry/pds:lid_reference": [ "urn:nasa:pds:soho:document", "urn:nasa:pds:soho:swan_derived" ], "vid": "1.0", "product_class": "Product_Bundle", "pds:Identification_Area/pds:logical_identifier": "urn:nasa:pds:soho", "ops:Label_File_Info/ops:blob": "eJy9VV1v0zAUfd+vsPoEEo1ThgaqvKDSgYa0sYmC4C1y49vGkmNHttNu/57rJFuSpuNLCD9ESe65n+fYZm/vCkV2YJ00+nwyi+IJAZ0ZIfX2fPL1y4fpm8nb5IQF2LQwAhTJLWzOJ7n35ZzSUrhIc8ejrdmFj1fhQXczenuxepXiI52dxXHksnxC8AEF9/claNcFqKyKTKayyNgtFU4o2uKs0SH1Cbu1RlSZT99VWiggWEnf/4kCJicEV42d3zn5iN/v99H+tE72Mo5n9Pv11arON5Xaea4zaD2dnDeFXJmM+3o6v0hJ/mwmd05MkjpX/WAfBWgvN7LJli4s8MZcW5XZokWlskWBTSqr5yHPHMPPnckNo0dQXYiWZTQmyDOjve8O5KVXkCxNAZ5ccM/JxpqC+BzIyihuCdeCXIKSxpU5WJmRm7UDu+Pe2HvybHVzefOc0SZIF1TqjbFF01ctorTNjYWcRXEo5mlIF6ZshZAp7lwylAWjQ+ujV+e+lL6J/7HL1eFqCAiJjaRKOp985tU2f0EW0TJitG8YupTVWj1wdg/cJi/j2Wus5vD30EuAy6wsxyWE9SWXjnCb5XIHZN2oPjPac5QovigFWXB0xGzwMzBlWhLqv4EigdzsQBALrlLekVGOjlakjIjAdJvRvSB76XN0VdxjCGGyqkA51dGjYRv0aB+M/nzUHfAaD5pHxV/icFFFB5MaQC4Ah6DGE2NFH4XdQKDhbBq/mcanjI6t4wi/szt+j76wgmpqAc3JEjdyABKcBPKicMS4VZC6jbTON6PvSI0IbrQCgj2wA3c4FDyLjybprQFJHbl42JzWDNOrxWrZ8uxQMDl3ZA2gidSZqgTSLPVYJAfLVWVprH+oTWrpJVdk9W3xaSS4Xkfj8R2XTWP6Kd8H5oFiGH3yBG3szUHRC9ZsrTTcSMmiET+j/Z9t3L5jP1R6DcUabPpe+75smZIixUsSLN6kMD6m5w9k4Xk9QHYRiiYwXki+CiedLLi9Rx0PfnfwxxBN2W0LyHH6kCrt+GD0AD7o8khP/6Jlt+e48xqN/I+2Ued/2zKjw5sl+QEuT9+Q", "pds:Citation_Information/pds:publication_year": "2017", "pds:Bundle/pds:bundle_type": "Archive", "ops:Label_File_Info/ops:file_ref": "/bx/sbnarch04/PDS4/soho/bundle.xml", "pds:Bundle_Member_Entry/pds:member_status": [ "Primary", "Primary" ], "ops:Label_File_Info/ops:md5_checksum": "7b23f2e87e464548dfae5ccf3ec0e0c8", "pds:Identification_Area/pds:information_model_version": "1.6.0.0", "lidvid": "urn:nasa:pds:soho::1.0", "pds:Citation_Information/pds:editor_list": "Raugh, A.C.", "pds:Modification_Detail/pds:modification_date": "2016-08-03T00:00:00Z", "ops:Label_File_Info/ops:file_size": "2340", "pds:Modification_Detail/pds:description": "A.C.Raugh: Creation for delivery of first data collection. Some of the existing documentation from the PDS3 SOHO/LASCO data set has been included in support of the initial SWAN derived results collection.", "ops:Label_File_Info/ops:json_blob": "eJydU2Fr2zAQ/StCnzZIbact3fC3LB200C5lKWywDaFIl1ggS0aSk4aS/74722madqxj/mDsk+7eu/fuHvld8LpVSXxqnbbAy0d+rcElszRKJuOdmASQFG6Gi8rKGHn5MnHEb70+ZF2ZmHzYUuJR/BKSNJbC9fOwlgmx+WkxvjgpPp4UZ1hPQ1TBNHSOR5Nsmn2V7aoq2RQZUZQtfWAarFlD2DK/ZEsTYmJYSzLlrQVFtzLG5r4GOk8VMHhAZsatfjr290d71daoxAAVfN3l313Oz9h8djXLbybz6axHi5BYJSNbADhmnLKtBo0f7E2U2DaND2nPzjiTjLRs/m3yBVsL2JpmAWJrU3zeE8qDTUeSzmhejne7ETcO9ah7PVFcsGK4guqNs4usyApMs36FoltM612GgMdtcKWTUZaNjmX0lX9Zf8SnpldCXB9gurFoF3bv4hYkVkMTP7x0774ykcmgKmyILbqJwXZQXOOe9xVJB4VuoSCLCGEt+6h0+pUar6V98ojs6Y0ZIOOIbUyqMNfioOljb0lM0AbHVVicDWTbzdmI0chxFDaZRJvBpx2xS6p7gPJWho7fFQ6ij02FNBWbDexxBdg7ovOeCj3U1tHqVCk1ZZ6j2Bmpnq38mn7O6ZWvx8jnsI29WCJtG6Iw6fuhYv0VcQv1AoL47BJt249Hbo0WAZYQwCn4k7flvnvEqfvsiEq0/VKbWmKhEX8qsYceiOCYi30BcXAOGf0LdNxIXPbeyf+HRweOoH+htNGUUVVQyxuvhuF8S2f2xnmOu34u8CXGF0WRPUSi3HlYItyh/mazyTZnmQ+r/LQoxvn325t5R+UEpztJ0mK3+w0z0ton", "pds:Identification_Area/pds:product_class": "Product_Bundle", "pds:Identification_Area/pds:title": "Comet Data from the Solar and Heliospheric Observatory (SOHO)", "ops:Harvest_Info/ops:harvest_date_time": "2022-02-18T20:53:52.601004Z", "ops:Label_File_Info/ops:file_name": "bundle.xml", "ops:Tracking_Meta/ops:archive_status": "archived", "pds:Identification_Area/pds:version_id": "1.0" }

[tloubrieu-jpl]

I am seeing 2 options:

• we make the entityProduct which reads these data more flexible to accept missing properties
• we skip the products which cannot be read. For robustness purpose, we might want to do both.