The user-facing ibis.expr.types layer. This is types like Table, IntegerColumn, StringScalar, ...
The internal ibis.expr.operations layer. This is types like ApproxMultiQuantile, ExtractEpochSeconds, DayOfWeekIndex, ...
Most user-facing methods in the former are implemented by creating an instance of the latter (followed by a .to_expr() call). In cases where an invalid type or value is passed, ibis's validation layer raises a SignatureValidationError.
These errors can be a bit verbose (see #10092), but they also leak details about the operation layer to the user who won't know anything about those types.
For example:
Hidden for readability
```python
In [1]: import ibis
In [2]: t = ibis.examples.diamonds.fetch()
In [3]: t.color + t.price
---------------------------------------------------------------------------
SignatureValidationError Traceback (most recent call last)
Cell In[3], line 1
----> 1 t.color + t.price
File ~/Code/ibis/ibis/expr/types/strings.py:1619, in StringValue.__add__(self, other)
1570 def __add__(self, other: str | StringValue) -> StringValue:
1571 """Concatenate strings.
1572
1573 Parameters
(...)
1617 └──────────────────────┘
1618 """
-> 1619 return self.concat(other)
File ~/Code/ibis/ibis/expr/types/strings.py:1568, in StringValue.concat(self, other, *args)
1527 def concat(self, other: str | StringValue, *args: str | StringValue) -> StringValue:
1528 """Concatenate strings.
1529
1530 NULLs are propagated. This methods is equivalent to using the `+` operator.
(...)
1566 └──────────────────────────┘
1567 """
-> 1568 return ops.StringConcat((self, other, *args)).to_expr()
File ~/Code/ibis/ibis/common/bases.py:72, in AbstractMeta.__call__(cls, *args, **kwargs)
52 def __call__(cls, *args, **kwargs):
53 """Create a new instance of the class.
54
55 The subclass may override the `__create__` classmethod to change the
(...)
70
71 """
---> 72 return cls.__create__(*args, **kwargs)
File ~/Code/ibis/ibis/common/grounds.py:119, in Annotable.__create__(cls, *args, **kwargs)
116 @classmethod
117 def __create__(cls, *args: Any, **kwargs: Any) -> Self:
118 # construct the instance by passing only validated keyword arguments
--> 119 kwargs = cls.__signature__.validate(cls, args, kwargs)
120 return super().__create__(**kwargs)
File ~/Code/ibis/ibis/common/annotations.py:501, in Signature.validate(self, func, args, kwargs)
498 this[name] = result
500 if errors:
--> 501 raise SignatureValidationError(
502 "{call} has failed due to the following errors:{errors}\n\nExpected signature: {sig}",
503 sig=self,
504 func=func,
505 args=args,
506 kwargs=kwargs,
507 errors=errors,
508 )
510 return this
SignatureValidationError: StringConcat((r0 := DatabaseTable: diamonds
carat float64
cut string
color string
clarity string
depth float64
table float64
price int64
x float64
y float64
z float64
color: r0.color, r0 := DatabaseTable: diamonds
carat float64
cut string
color string
clarity string
depth float64
table float64
price int64
x float64
y float64
z float64
price: r0.price)) has failed due to the following errors:
`arg`: (r0 := DatabaseTable: diamonds
carat float64
cut string
color string
clarity string
depth float64
table float64
price int64
x float64
y float64
z float64
color: r0.color, r0 := DatabaseTable: diamonds
carat float64
cut string
color string
clarity string
depth float64
table float64
price int64
x float64
y float64
z float64
price: r0.price) is not a tuple of coercibles to a Value[String, DataShape]
Expected signature: StringConcat(arg: tuple[Value[String, DataShape], ...])
```
Truncating a bit, the lines to focus on are:
... is not a tuple of coercibles to a Value[String, DataShape]
Expected signature: StringConcat(arg: tuple[Value[String, DataShape], ...])
Value[String, DataShape] doesn't necessarily mean much to an ibis user unfamiliar with the parametrized version of our internal ops.Value. This may be better read as "string values".
StringConcat is a class the user doesn't need to know about, and doesn't match the name of the method or operator the user called. For some methods the op name and signature are close to the method name and signature (Table.sample and ops.Sample are one example), but for others the names and signatures can differ significantly.
I'm not sure I have a good proposed solution here, mostly opening this for discussion. I think ideally:
The error message uses the method name and signature that the user called
The error message doesn't leak op names or op types to the user, instead using terminology used at the ibis.expr.types level.
We try to minimize the type system notation usage in cases where something more human readable may be friendlier. In the example above the issue is that all args aren't coercible to strings.
One thing that might be helpful for us to do here is to do a survey of which API calls don't point to a single Op, but that are comprised of many operations
Ibis expression system has two layers:
ibis.expr.types
layer. This is types likeTable
,IntegerColumn
,StringScalar
, ...ibis.expr.operations
layer. This is types likeApproxMultiQuantile
,ExtractEpochSeconds
,DayOfWeekIndex
, ...Most user-facing methods in the former are implemented by creating an instance of the latter (followed by a
.to_expr()
call). In cases where an invalid type or value is passed, ibis's validation layer raises aSignatureValidationError
.These errors can be a bit verbose (see #10092), but they also leak details about the operation layer to the user who won't know anything about those types.
For example:
Hidden for readability
```python In [1]: import ibis In [2]: t = ibis.examples.diamonds.fetch() In [3]: t.color + t.price --------------------------------------------------------------------------- SignatureValidationError Traceback (most recent call last) Cell In[3], line 1 ----> 1 t.color + t.price File ~/Code/ibis/ibis/expr/types/strings.py:1619, in StringValue.__add__(self, other) 1570 def __add__(self, other: str | StringValue) -> StringValue: 1571 """Concatenate strings. 1572 1573 Parameters (...) 1617 └──────────────────────┘ 1618 """ -> 1619 return self.concat(other) File ~/Code/ibis/ibis/expr/types/strings.py:1568, in StringValue.concat(self, other, *args) 1527 def concat(self, other: str | StringValue, *args: str | StringValue) -> StringValue: 1528 """Concatenate strings. 1529 1530 NULLs are propagated. This methods is equivalent to using the `+` operator. (...) 1566 └──────────────────────────┘ 1567 """ -> 1568 return ops.StringConcat((self, other, *args)).to_expr() File ~/Code/ibis/ibis/common/bases.py:72, in AbstractMeta.__call__(cls, *args, **kwargs) 52 def __call__(cls, *args, **kwargs): 53 """Create a new instance of the class. 54 55 The subclass may override the `__create__` classmethod to change the (...) 70 71 """ ---> 72 return cls.__create__(*args, **kwargs) File ~/Code/ibis/ibis/common/grounds.py:119, in Annotable.__create__(cls, *args, **kwargs) 116 @classmethod 117 def __create__(cls, *args: Any, **kwargs: Any) -> Self: 118 # construct the instance by passing only validated keyword arguments --> 119 kwargs = cls.__signature__.validate(cls, args, kwargs) 120 return super().__create__(**kwargs) File ~/Code/ibis/ibis/common/annotations.py:501, in Signature.validate(self, func, args, kwargs) 498 this[name] = result 500 if errors: --> 501 raise SignatureValidationError( 502 "{call} has failed due to the following errors:{errors}\n\nExpected signature: {sig}", 503 sig=self, 504 func=func, 505 args=args, 506 kwargs=kwargs, 507 errors=errors, 508 ) 510 return this SignatureValidationError: StringConcat((r0 := DatabaseTable: diamonds carat float64 cut string color string clarity string depth float64 table float64 price int64 x float64 y float64 z float64 color: r0.color, r0 := DatabaseTable: diamonds carat float64 cut string color string clarity string depth float64 table float64 price int64 x float64 y float64 z float64 price: r0.price)) has failed due to the following errors: `arg`: (r0 := DatabaseTable: diamonds carat float64 cut string color string clarity string depth float64 table float64 price int64 x float64 y float64 z float64 color: r0.color, r0 := DatabaseTable: diamonds carat float64 cut string color string clarity string depth float64 table float64 price int64 x float64 y float64 z float64 price: r0.price) is not a tuple of coercibles to a Value[String, DataShape] Expected signature: StringConcat(arg: tuple[Value[String, DataShape], ...]) ```Truncating a bit, the lines to focus on are:
Value[String, DataShape]
doesn't necessarily mean much to an ibis user unfamiliar with the parametrized version of our internalops.Value
. This may be better read as "string values".StringConcat
is a class the user doesn't need to know about, and doesn't match the name of the method or operator the user called. For some methods the op name and signature are close to the method name and signature (Table.sample
andops.Sample
are one example), but for others the names and signatures can differ significantly.I'm not sure I have a good proposed solution here, mostly opening this for discussion. I think ideally:
ibis.expr.types
level.