search

lanl / dsi

LANL Data Science Infrastructure Project
https://lanl.github.io/dsi
5 stars 4 forks source link

Slowness when importing data #5

Open qwofford opened 1 year ago

qwofford commented 1 year ago

I got a report today from @rstyd about what appears to be a fixed cost slowness when importing CSV files. This is particularly expensive when the column name is empty, as is the case when importing a Pandas dataframe with an un-named index column.

This is a common pattern for Pandas users, so we should take a look!

qwofford commented 1 year ago

Have we made progress on this issue or have we decided to not pursue?