The DwCACreator is failing to parse the syntactically valid output from the image-service export.csv service, other than the two bugs identified in the image-service tracker for incorrect URLs and the occurrenceID field being duplicated.
aws-bstore-1b 2019-09-20 12:47:09,290 INFO : [Cassandra3PersistenceManager] - All threads have completed paging
aws-bstore-1b 2019-09-20 12:47:09,831 INFO : [DwCACreator] - Downloading images archive extract....
aws-bstore-1b 2019-09-20 12:48:06,338 INFO : [DwCACreator] - Downloaded images archive extract to /data/tmp/images-export/images-export.csv.gz
aws-bstore-1b 2019-09-20 12:48:06,338 INFO : [DwCACreator] - Extracting Gzip....
aws-bstore-1b 2019-09-20 12:48:13,147 INFO : [DwCACreator] - Splitting into separate files to..../data/tmp/images-export/split
aws-bstore-1b 2019-09-20 13:28:47,017 ERROR: [DwCACreator] - Unterminated quoted field at end of CSV line. Beginning of lost text: [ Henry Cook-3973.jpg,,jpg,CC BY-NC-SA 3.0,,43eb346f65da49215544390d4774312b,369431,1624,1080,5,e8...]
java.io.IOException: Unterminated quoted field at end of CSV line. Beginning of lost text: [ Henry Cook-3973.jpg,,jpg,CC BY-NC-SA 3.0,,43eb346f65da49215544390d4774312b,369431,1624,1080,5,e8...]
at com.opencsv.CSVReader.readNext(CSVReader.java:334)
at au.org.ala.biocache.export.DwCACreator.addImageExportsToArchives(DwCACreator.scala:450)
at au.org.ala.biocache.export.DwCACreator$.main(DwCACreator.scala:178)
at au.org.ala.biocache.cmd.CMD2$.main(CMD2.scala:130)
at au.org.ala.biocache.cmd.CMD2.main(CMD2.scala)
Exception in thread "main" java.lang.RuntimeException: java.io.IOException: Unterminated quoted field at end of CSV line. Beginning of lost text: [ Henry Cook-3973.jpg,,jpg,CC BY-NC-SA 3.0,,43eb346f65da49215544390d4774312b,369431,1624,1080,5,e8...]
at au.org.ala.biocache.export.DwCACreator$.main(DwCACreator.scala:182)
at au.org.ala.biocache.cmd.CMD2$.main(CMD2.scala:130)
at au.org.ala.biocache.cmd.CMD2.main(CMD2.scala)
Caused by: java.io.IOException: Unterminated quoted field at end of CSV line. Beginning of lost text: [ Henry Cook-3973.jpg,,jpg,CC BY-NC-SA 3.0,,43eb346f65da49215544390d4774312b,369431,1624,1080,5,e8...]
at com.opencsv.CSVReader.readNext(CSVReader.java:334)
at au.org.ala.biocache.export.DwCACreator.addImageExportsToArchives(DwCACreator.scala:450)
at au.org.ala.biocache.export.DwCACreator$.main(DwCACreator.scala:178)
... 2 more
Parsing it with a standards-compliant CSV parser succeeds, so the issue is either in the configuration or the operation of the opencsv parser.
The DwCACreator is failing to parse the syntactically valid output from the image-service export.csv service, other than the two bugs identified in the image-service tracker for incorrect URLs and the
occurrenceID
field being duplicated.Parsing it with a standards-compliant CSV parser succeeds, so the issue is either in the configuration or the operation of the opencsv parser.