Since Pandas 2.0, it is easier to get arrow-based dtypes in pandas (see here) and they certainly will arrive more frequently from the user side in the future.
Currently, this seems to result in errors like
TypeError: boolean value of NA is ambiguous
TypeError: Cannot interpret 'Int64Dtype()' as a data type
This is clearly suboptimal and will only grow worse. GETTSIM should work with these dtypes in the sense that users should be able to pass them and get the same stuff back. Main thing will be what to do with Nullability, which at least Jax will not support anytime soon, it seems.
I did not think much about a potential solution yet and it is not the most pressing issue, but we should keep it in the backs of our minds.
Is your feature request related to a problem?
Since Pandas 2.0, it is easier to get arrow-based dtypes in pandas (see here) and they certainly will arrive more frequently from the user side in the future.
Currently, this seems to result in errors like
TypeError: boolean value of NA is ambiguousTypeError: Cannot interpret 'Int64Dtype()' as a data typeThis is clearly suboptimal and will only grow worse. GETTSIM should work with these dtypes in the sense that users should be able to pass them and get the same stuff back. Main thing will be what to do with Nullability, which at least Jax will not support anytime soon, it seems.
I did not think much about a potential solution yet and it is not the most pressing issue, but we should keep it in the backs of our minds.