edsu
commented
1 year ago We noticed that one JSON object in the Transform log was getting split over multiple lines?
{"cho_title":{"en":["Luxor Temple"]},"cho_creator":{"en":["Abdullah Frères"]},"cho_date":{"en":["1890-1899"]},"cho_date_range_hijri":[1307,1308,1309,1310,1311,1312,1313,1314,1315,1316,1317],"cho_date_range_norm":[1890,1891,1892,1893,1894,1895,1896,1897,1898,1899],"cho_dc_rights":{"en":["To inquire about permissions or reproductions, contact the Rare Books and Special Collections Library, The American University in Cairo at +20.2.2615.3676 or rbscl-ref@aucegypt.edu."]},"cho_description":{"en":["Temple de Luxor. 109A"]},"cho_edm_type":{"en":["Image"],"ar-Arab":["صورة"]},"cho_format":{"en":["image/jpg"]},"cho_has_type":{"en":["Other Images"],"ar-Arab":["صور أخرى"]},"cho_is_part_of":{"en":["19th Century photographs"]},"cho_medium":{"en":["photographic prints"]},"cho_type":{"en":["Still Image"]},"agg_data_provider":{"en":["American University in Cairo"],"ar-Arab":["الجامعة الأمريكية في القاهرة"]},"agg_provider":{"en":["American Uni
[2023-03-16, 17:22:21 UTC] {docker.py:373} INFO - versity in Cairo
[2023-03-16, 17:22:21 UTC] {docker.py:373} INFO - "],"ar-Arab":["الجامعة الأمريكية في القاهرة"]},"agg_provider_country":{"en":["Egypt"],"ar-Arab":["مصر"]},"agg_data_provider_country":{"en":["Egypt"],"ar-Arab":["مصر"]},"cho_type_facet":{"en":["Image","Image:Other Images"],"ar-Arab":["صورة","صورة:صور أخرى"]},"id":"p15795coll38:91","transform_version":"923910e","transform_timestamp":"2023-03-16 17:22:21 +0000","agg_data_provider_collection_id":"auc-abdullah-freres","dlme_source_file":"/auc/abdullah_freres/data.csv","agg_is_shown_at":{"wr_dc_rights":["To inquire about permissions or reproductions, contact the Rare Books and Special Collections Library, The American University in Cairo at +20.2.2615.3676 or rbscl-ref@aucegypt.edu."],"wr_format":["image/jpeg"],"wr_is_referenced_by":["https://cdm15795.contentdm.oclc.org/iiif/p15795coll38:91/manifest.json"],"wr_id":"https://cdm15795.contentdm.oclc.org/iiif/2/p15795coll38:91/full/full/0/default.jpg"},"agg_preview":{"wr_dc_rights":["To inquire about permissions or reproductions, contact the Rare Books and Special Collections Library, The American University in Cairo at +20.2.2615.3676 or rbscl-ref@aucegypt.edu."],"wr_format":["image/jpeg"],"wr_is_referenced_by":["https://cdm15795.contentdm.oclc.org/iiif/p15795coll38:91/manifest.json"],"wr_id":"https://cdm15795.contentdm.oclc.org/iiif/2/p15795coll38:91/full/400,400/0/default.jpg"}}
Exception: ERROR: failed to transform all harvested records: harvested record count (48) != transformed record count (51)I confirmed 48 records harvested by running harvest again locally. I don't know why the transformed record count is 51. This is strange behavior. I only expected the the transformed record count could be lower. Bad data or errors in the config might prevent some records from being transformed. This is adding 3 records and I have no idea where they would come from. I suspect traject or another ariflow task might be adding blank lines?
auc_ahmed_toughan, auc_coptic, auc_maps, etc. worked