"Collection is missing pid for item 0."

[egpalazzolo]

Have been trying to rerun wax tasks (on the branch "new") on the expanded version of the document set for the new site with .csv generated in Airtable and cleaned through OpenRefine. It keeps giving the error:

"WaxTasks::Error::MissingPid: Collection is missing pid for item 0. Hint: review common .csv formatting issues (such as hidden characters) in the documentation: https://minicomp.github.io/wiki/wax/preparing-your-collection-data/metadata/"

when i try to run for new pages :(

[egpalazzolo]

After uploading the .csv to Google sheets from Airtable and then re-downloading, it seemed to work just fine...

[jabrah]

Update: Looks like either the Airtable export or OpenRefine steps are adding BOM encoding data to the beginning of the CSV (more likely Airtable). The BOM (byte order mark) are a couple of bytes of data that appear at the very beginning of the file just before the pid header in a way that is tripping up how the Wax tasks are parsing the data.

Solutions:

Any of these options may work for this issue

• Maybe Airtable has a way to not export this BOM data in the CSV. Airtable may not have this capability
• If OpenRefine is or will be part of your data workflow (getting data into the site), then maybe it would be able to clean the pid header
• We can make a script to strip these extra bytes from the beginning of the CSV to restore Wax functionality
  • Jon Baynes had a proof of concept in Python
  • A simple Bash script would also work
• We could update the Wax scripts to work correctly if a CSV has BOM data