search

lanl / dsi

LANL Data Science Infrastructure Project
https://lanl.github.io/dsi
5 stars 3 forks source link

Bueno plugin: handle multiple files #64

Closed qwofford closed 1 year ago

qwofford commented 1 year ago

taking advantage of some of Daniel's work on new plugin types for files

qwofford commented 1 year ago

forgive the 'a' commit messages...we will squash them on merge :grimacing:

qwofford commented 1 year ago

@DanielRJohnson Can you take a look at this PR and take a shot at the sampling problem described in the most recent commit message. This is the highest priority DSI issue at the moment and I'm down an arm...let me know if you have questions about how it should work.

DanielRJohnson commented 1 year ago

@qwofford Of course, take it easy!

DanielRJohnson commented 1 year ago

The reason that it was overwriting was that pack_header was being called every add_rows call, which initializes every column to an empty list.

DanielRJohnson commented 1 year ago

Tests and pylama pass, ready to merge I think