When NCPI export is attempted on large query results (many matching files), the export service reports a query timeout. The same export would succeed as a BDBag output.
This is due to the relatively slow export transformer code generating the NCPI manifest file. If we precompute an NCPI manifest table during the ETL phase when other portal tables are constructed, we can produce the same CSV output during export with the cheaper method used in BDBag exports.
When NCPI export is attempted on large query results (many matching files), the export service reports a query timeout. The same export would succeed as a BDBag output.
This is due to the relatively slow export transformer code generating the NCPI manifest file. If we precompute an NCPI manifest table during the ETL phase when other portal tables are constructed, we can produce the same CSV output during export with the cheaper method used in BDBag exports.