Open cosmicBboy opened 2 years ago
cc @cosmicBboy we can't use non-dataclass type in dataclass. However, you could change the type hint to data: StrucutredDataset
, and it should work.
we can't use non-dataclass type in dataclass
@pingsutw is this a technical issue or a philosophical one? surely we can update the dataclass transformer to be able to handle pd.DataFrame
annotations as StructuredDataset
s under the hood?
An alternate solution here would be to introspect on the dataclass
definition and raise an informative error pointing the user to StructuredDataset
.
this is a technical issue and a limitation of python dataclass/dataclass_json, not on the flyte side. but yeah, flytekit should introspect and raise a friendly error.
Hello 👋, This issue has been inactive for over 9 months. To help maintain a clean and focused backlog, we'll be marking this issue as stale and will close the issue if we detect no activity in the next 7 days. Thank you for your contribution and understanding! 🙏
Hello 👋, This issue has been inactive for over 9 months and hasn't received any updates since it was marked as stale. We'll be closing this issue for now, but if you believe this issue is still relevant, please feel free to reopen it. Thank you for your contribution and understanding! 🙏
Hi, seems like this issue is already solved by the dataclass source code, please refer to https://github.com/lidatong/dataclasses-json/pull/389#issue-1446332429
@zychen5186, you can use strucutredDataset inside the dataclass.
@dataclass_json
@dataclass
class TrainArgs:
hyperparameters: dict
data: StructuredDataset
Describe the bug
using
pd.DataFrame
with dataclasses raises an error:Expected behavior
this should work like structured dataset
Additional context to reproduce
Using this type in a task
will lead to the error above
Screenshots
No response
Are you sure this issue hasn't been raised already?
Have you read the Code of Conduct?