Open nemausus opened 1 month ago
Hey @nemausus, I just looked into your issue and the problem that i found is that the join
method is trying to align the MultiIndex
levels of the two DataFrames, but the names of the index levels are not aligned correctly.
>>> df1
A
X
3 3 1
>>> df2
B
X
3 3 1
So it gets confused because the index levels names do not match, even though the structure of the indices is the same.
So if the Data frames have the same names for their multiindex levels this issue can be fixed is what I think, as I tried to test it this is what I found:
>>> df2 = pd.DataFrame({"B": [1]}, index=pd.MultiIndex.from_tuples([(3, 3)], names=[None, "X"]))
>>> df1 = pd.DataFrame({"A": [1]}, index=pd.MultiIndex.from_tuples([(3, 3)], names=[None, "X"]))
>>> df1.join(df2)
A B
X
3 3 1 1
Let me know if this solves your issue or I can look more further into it, thanks.
@SiddheshBangar - this raises an AssertionError - those shouldn't be exposed to the user (only things like ValueError, TypeError, etc). So this definitely looks like a bug. Also, if you replace None
with "Y"
, then the operation is successful.
Hey @rhshadrach, so you mean to say that the error message should not be prompt or either error message can be fixed somehow and replace it with assertion error message.
At a glance, it seems to me that this should not raise an error. If implementation difficulties make that difficult, then perhaps we can look into other possibilities. Further investigations are welcome!
Okay, cool thanks @rhshadrach this looks a bit confusing to me at this point but I will try to figure out what the best can I do, if you have any extra information to share feel free to drop down that can help me to resolve this issue, Thanks
It looks like the issue is at https://github.com/pandas-dev/pandas/blob/b162331554d7c7f6fd46ddde1ff3908f2dc8bcce/pandas/core/indexes/base.py#L4585
# maintain the order of the index levels
if how == "right":
level_order = other_names_list + ldrop_names
else:
level_order = self_names_list + rdrop_names
multi_join_idx = multi_join_idx.reorder_levels(level_order)
Null index names are removed from other_names_list
and self_names_list
for upstream operations and then also as MultiIndex.reorder_levels()
accepts a list of str or int (not None). Therefore, length of level_order
becomes one less than index length.
Note that if the above lines are removed (as per Pandas==2.2.2) then this issue no longer exists.
Perhaps MultiIndex.reorder_levels()
could be extended to accept None type also, and level_order can be updated to include None if existing?
Pandas version checks
[X] I have checked that this issue has not already been reported.
[X] I have confirmed this bug exists on the latest version of pandas.
[ ] I have confirmed this bug exists on the main branch of pandas.
Reproducible Example
Issue Description
Fails with error: AssertionError: Length of order must be same as number of levels (2), got 1
Expected Behavior
Should not result in AssertionError
Installed Versions