ibis-project / ibis-ml

IbisML is a library for building scalable ML pipelines using Ibis.
https://ibis-project.github.io/ibis-ml/
Apache License 2.0
96 stars 13 forks source link

bug(recipe): should we not transform the last step in recipe during recipe.fit() #148

Closed jitingxu1 closed 2 months ago

jitingxu1 commented 2 months ago

should we not transform the last step during fit(): https://github.com/ibis-project/ibis-ml/blob/main/ibis_ml/core.py#L512

    def _fit_table(
        self, table: ir.Table, targets: tuple[str, ...] = (), index: str | None = None
    ) -> None:
        metadata = Metadata(targets=targets)

        if index is not None:
            table = table.drop(index)

        for step in self.steps:
            step.fit_table(table, metadata)
            table = step.transform_table(table)

        self.metadata_ = metadata
jitingxu1 commented 2 months ago

if it used in sklearn or other pipeline, I think it is needed. But it looks weird, if we use recipe independently...it already does all the transformation work in fit()