jazzband / tablib

Python Module for Tabular Datasets in XLS, CSV, JSON, YAML, &c.
https://tablib.readthedocs.io/
MIT License
4.56k stars 588 forks source link

appending rows when there are dynamic columns #572

Closed LucOevel closed 6 months ago

LucOevel commented 7 months ago

At the moment we cannot add rows (using append, rpush, lpush) when there is already a dynamic column: a function object can be added at the appropriate place in the row, but unlike a normal dynamic column it will not be executed.

claudep commented 7 months ago

Thanks for the report, are you planning a pull request?

LucOevel commented 7 months ago

no... (I would if i could)

claudep commented 7 months ago

It would be great if you could test (or make test) the suggested patch above.

LucOevel commented 6 months ago
#------short test ------
print(tablib.__version__)

data = tablib.Dataset()

txt='''Greg Thorton
Dave Coutts
Joshua Ourisman'''

data.extend([item.split() for item in txt.splitlines()])

def dynamicColumn(row):
    return row[-1].upper()+'!'

data.append_col(dynamicColumn,'Calculated')  

data.append(('Brad','Montgomery', dynamicColumn))

#----------result: as before, dynamicColumn is not evaluated:
# 3.5.0
# Greg  |Thorton   |THORTON!
# Dave  |Coutts    |COUTTS!
# Joshua|Ourisman  |OURISMAN!
# Brad  |Montgomery|<function dynamicColumn at 0x000001A071042940>
claudep commented 6 months ago

Indeed, I worked on a similar but different use case in this patch. After adding a dynamic column, you can only provide the static values and the dynamic ones will be inserted automatically.

The use case in your latest example is to be able to add a dynamic value in any row, regardless of any previous data.append_col(func) call. I'll see if I can also integrate that use case.

claudep commented 6 months ago

@hugovk do you think that adding dynamic values in rows is worth implementing? We could argue that when inserting rows, you should be able to compute the values yourself.

hugovk commented 6 months ago

I don't need it and am generally for less maintenance but not against it :)

claudep commented 6 months ago

@LucOevel, I committed the current patch which matches this issue description. Now feel free to create a new ticket for adding support to appending rows including callable values. However, please describe also the use case for that functionality, as you could as well prepare the values you are going to insert before adding the row.

LucOevel commented 6 months ago

As I understood it, the main point of dynamic columns is that their content automatically depends on other columns. So when other columns change, the dynamic values follow dynamically. Is that not what the name means? Having to pre-compute values when inserting a new row defeats that behavior, and those values will not be dynamic anymore. If the idea is that values can just as well be pre-computed instead of dynamic then why have dynamic values at all, right?
Why have dynamic elements when adding columns but not when adding rows?

BTW: @claudep can you provide an example of the use case you were thinking about? I seem to miss that point.