search

PASTAplus / PASTA

Repository for the Provenance Aware Synthesis Tracking Architecture (PASTA) project.
12 stars 3 forks source link

Quoted newline in table record causes exception during ECC database load #29

Open servilla opened 4 years ago

servilla commented 4 years ago

From Sven Bohm:

On Tue, Jun 16, 2020 at 9:02 AM sven bohm bohms@ideomatic.io wrote: Hi Mark,

Hope you are doing well. I'm not sure who to send this to, but I noticed that the congruency checker seems to prioritize line breaks over quotes. That is if a quoted string includes a line break it complains "There is a un-closed quote in data file". Here is an example:

https://portal-s.edirepository.org/nis/reportviewer?packageid=knb-lter-kbs.195.20&localPath=%2Fhome%2Fpasta%2Flocal%2Fharvester%2FLTER-ecoinformatics-org%2FKBS-evaluate-2020-06-16-1592317526799%2Fknb-lter-kbs.195.20%2FqualityReport.xml on entity: /datatables/640

I can file a issue if you'd like.

Thanks

Sven Bohm -.- ..-. ---.. .-

servilla commented 4 years ago

This problem is likely in "PASTA/DataPackageManager/src/edu/lternet/pasta/dml/database/DelimitedReader.java".

servilla commented 4 years ago

Ignoring the quoted newline or carriage return should be handled in this method: see edu.lternet.pasta.dml.database.DelimitedReader.processQuotedCharacterOneRowData in the DataPackageManager for a possible entry point for this issue.