search

dag-hammarskjold-library / dlx-rest

UNDHL Files and Metadata Manager
1 stars 1 forks source link

Certain fields aren't captured in CSV export #1551

Open viola-v opened 5 days ago

viola-v commented 5 days ago

When we try to export results from authority searches in CSV and limit the out to individual fields, such as 110, 111, 130 and 150, these fields aren't captured in the export file. It works when we don't limit it to certain fields. It also works for 100, 190 and 191.

viola-v commented 5 days ago

For this query, some 110 fields display, others don't: https://metadata.un.org/editor/records/auths/search?q=998%3A1998*+AND+110%3A*+AND+UNEP&start=1&limit=250&sort=updated&direction=desc&search=998%3A1998*+AND+110%3A*+AND+unesco&engine=community&format=brief

aaronhelton commented 5 days ago

The issue seems to stem, in part, from fields that have commas in them. This doesn't explain all cases, but in the query above, most 110s display in the resulting CSV; some 110s are blank; and an unknown number are truncated because of bare commas that arrived in the export from the API.

Sample export: export(39).csv

aaronhelton commented 5 days ago

Okay, scratch that. On further inspection, the fields that appear to be truncated are fields that have multiple subfields that aren't being concatenated.

We'll have to dig in further to see what's going wrong here.

At least for auth exports, the current workaround is to export all fields instead of limiting fields. This seems to work.