ibis-project / ibis

the portable Python dataframe library
https://ibis-project.org
Apache License 2.0
5.25k stars 592 forks source link

ux: `SignatureValidationError` leaks details of ibis's internals to users #10093

Open jcrist opened 1 month ago

jcrist commented 1 month ago

Ibis expression system has two layers:

Most user-facing methods in the former are implemented by creating an instance of the latter (followed by a .to_expr() call). In cases where an invalid type or value is passed, ibis's validation layer raises a SignatureValidationError.

These errors can be a bit verbose (see #10092), but they also leak details about the operation layer to the user who won't know anything about those types.

For example:

Hidden for readability ```python In [1]: import ibis In [2]: t = ibis.examples.diamonds.fetch() In [3]: t.color + t.price --------------------------------------------------------------------------- SignatureValidationError Traceback (most recent call last) Cell In[3], line 1 ----> 1 t.color + t.price File ~/Code/ibis/ibis/expr/types/strings.py:1619, in StringValue.__add__(self, other) 1570 def __add__(self, other: str | StringValue) -> StringValue: 1571 """Concatenate strings. 1572 1573 Parameters (...) 1617 └──────────────────────┘ 1618 """ -> 1619 return self.concat(other) File ~/Code/ibis/ibis/expr/types/strings.py:1568, in StringValue.concat(self, other, *args) 1527 def concat(self, other: str | StringValue, *args: str | StringValue) -> StringValue: 1528 """Concatenate strings. 1529 1530 NULLs are propagated. This methods is equivalent to using the `+` operator. (...) 1566 └──────────────────────────┘ 1567 """ -> 1568 return ops.StringConcat((self, other, *args)).to_expr() File ~/Code/ibis/ibis/common/bases.py:72, in AbstractMeta.__call__(cls, *args, **kwargs) 52 def __call__(cls, *args, **kwargs): 53 """Create a new instance of the class. 54 55 The subclass may override the `__create__` classmethod to change the (...) 70 71 """ ---> 72 return cls.__create__(*args, **kwargs) File ~/Code/ibis/ibis/common/grounds.py:119, in Annotable.__create__(cls, *args, **kwargs) 116 @classmethod 117 def __create__(cls, *args: Any, **kwargs: Any) -> Self: 118 # construct the instance by passing only validated keyword arguments --> 119 kwargs = cls.__signature__.validate(cls, args, kwargs) 120 return super().__create__(**kwargs) File ~/Code/ibis/ibis/common/annotations.py:501, in Signature.validate(self, func, args, kwargs) 498 this[name] = result 500 if errors: --> 501 raise SignatureValidationError( 502 "{call} has failed due to the following errors:{errors}\n\nExpected signature: {sig}", 503 sig=self, 504 func=func, 505 args=args, 506 kwargs=kwargs, 507 errors=errors, 508 ) 510 return this SignatureValidationError: StringConcat((r0 := DatabaseTable: diamonds carat float64 cut string color string clarity string depth float64 table float64 price int64 x float64 y float64 z float64 color: r0.color, r0 := DatabaseTable: diamonds carat float64 cut string color string clarity string depth float64 table float64 price int64 x float64 y float64 z float64 price: r0.price)) has failed due to the following errors: `arg`: (r0 := DatabaseTable: diamonds carat float64 cut string color string clarity string depth float64 table float64 price int64 x float64 y float64 z float64 color: r0.color, r0 := DatabaseTable: diamonds carat float64 cut string color string clarity string depth float64 table float64 price int64 x float64 y float64 z float64 price: r0.price) is not a tuple of coercibles to a Value[String, DataShape] Expected signature: StringConcat(arg: tuple[Value[String, DataShape], ...]) ```

Truncating a bit, the lines to focus on are:

... is not a tuple of coercibles to a Value[String, DataShape]

Expected signature: StringConcat(arg: tuple[Value[String, DataShape], ...])

I'm not sure I have a good proposed solution here, mostly opening this for discussion. I think ideally:

gforsyth commented 1 month ago

One thing that might be helpful for us to do here is to do a survey of which API calls don't point to a single Op, but that are comprised of many operations