Avaiga / taipy-core

A Python library to build powerful and customized data-driven back-end applications.

Apache License 2.0

38 stars 17 forks source link

Feature/#336 - Filter numpy array exposed type #768

Closed trgiangdo closed 1 year ago

trgiangdo commented 1 year ago

https://github.com/Avaiga/taipy-core/issues/336

A few things we need to consider:

We should probably also consider to expose the Operator and JoinOperator at the taipy init level.
If the exposed type is numpy, then the key is the index of the column. If the user wants to filter by row value, we need to find a way to allow that.

github-actions[bot] commented 1 year ago

☂️ Python Cov

current status: ✅

Overall Coverage

Lines Covered Coverage Threshold Status

8175 7760 95% 85% 🟢

New Files

No new covered files...

Modified Files

File Coverage Status

src/taipy/core/_entity/_migrate/_utils.py 85% 🟢

src/taipy/core/data/_abstract_sql.py 92% 🟢

src/taipy/core/data/_filter.py 91% 🟢

src/taipy/core/data/csv.py 95% 🟢

src/taipy/core/data/data_node.py 99% 🟢

src/taipy/core/data/parquet.py 98% 🟢

src/taipy/core/data/sql.py 97% 🟢

src/taipy/core/data/sql_table.py 98% 🟢

TOTAL 94% 🟢

updated for commit: bb54978 by action🐍

Lines	Covered	Coverage	Threshold	Status
8175	7760	95%	85%	🟢

File	Coverage	Status
src/taipy/core/_entity/_migrate/_utils.py	85%	🟢
src/taipy/core/data/_abstract_sql.py	92%	🟢
src/taipy/core/data/_filter.py	91%	🟢
src/taipy/core/data/csv.py	95%	🟢
src/taipy/core/data/data_node.py	99%	🟢
src/taipy/core/data/parquet.py	98%	🟢
src/taipy/core/data/sql.py	97%	🟢
src/taipy/core/data/sql_table.py	98%	🟢
TOTAL	94%	🟢

trgiangdo commented 1 year ago

That method of excel can also return a dict of array, I should update it as well.

2.1. Me neither 🥲 I think we can remove it since it is not documented at all.

2.2. I agree with moving the logic to a dedicate file. And if I understand correctly, the filter() method should still return the filtered data directly, not returning the Filter class right?

trgiangdo commented 1 year ago

In this PR:

New feature:

Ability to filter numpy exposed_type

Refactoring:

Refactor the _FilterDataNode class. Move the logic of the DataNode.filter() method to the _FilterDataNode class.
Simplify the getitem method of DataNode, make it consistent with DataNode.filter() method.
Add unit tests for filtering all data node types.

Bug fixes:

Fix unable to to filter excel data node (because excel data node return a dictionary of exposed_type, not the direct exposed_type).
Fix unable to filter JSONDataNode when the data is a list of dictionaries.
Fix unable to filter a list of objects (custom exposed type) when some object has an attribute that others don't.
Fix unable to write to SQLDataNode due to not converting the string query into SQL text query (raised by issue https://github.com/Avaiga/taipy-core/issues/816).