function scaling_results (https://github.com/Zeta-and-Company/pydistinto/blob/970fd4cdb262b6bb8a1c27bd3643f2500de220e0/scripts/pipeline/calculate_simple.py#L101) sends an incoming pandas Series to sklearn-s .fit_transform which returns an Error if a Series contains inf values. Why these inf values occur in the first place is unclear to me, but I have at least one corpus on which it reproduces (see demo.zip attached, metadata csv included; in the parameters.txt set contrast=generated). Strangely to me, it's actually a subset of a larger corpus, on which the error does not reproduce. But on this smaller subset it does.
For my own purposes I temporarily 'fixed' the issue by adding Series = Series[~np.isinf(Series)] to the function, but that's just my stopgap.
function scaling_results (https://github.com/Zeta-and-Company/pydistinto/blob/970fd4cdb262b6bb8a1c27bd3643f2500de220e0/scripts/pipeline/calculate_simple.py#L101) sends an incoming pandas Series to sklearn-s
.fit_transformwhich returns an Error if a Series containsinfvalues. Why these inf values occur in the first place is unclear to me, but I have at least one corpus on which it reproduces (see demo.zip attached, metadata csv included; in the parameters.txt setcontrast=generated). Strangely to me, it's actually a subset of a larger corpus, on which the error does not reproduce. But on this smaller subset it does. For my own purposes I temporarily 'fixed' the issue by addingSeries = Series[~np.isinf(Series)]to the function, but that's just my stopgap.