added sort_index parameter to DataFrame.to_pandas function

Arnav-Gr0ver commented 5 months ago

Description

Resolves DataFrame.to_pandas duplication bug when an os_index_field is set, following expected behavior

Issues Resolved

https://github.com/opensearch-project/opensearch-py-ml/issues/382

Check List

[ ] New functionality includes testing.
- [ ] All tests pass
[ ] New functionality has been documented.
- [ ] New functionality has javadoc added
[ ] Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license. For more information on following Developer Certificate of Origin and signing off your commits, please check here.

cbeaujoin-stellar commented 5 months ago

I think it should be query_compiler.index.os_index_field

Arnav-Gr0ver commented 5 months ago

You're right @cbeaujoin-stellar , didn't notice that it was an object method

dhrubo-os commented 5 months ago

@Arnav-Gr0ver could you please take a look at the failures? Thanks

dhrubo-os commented 5 months ago

Integ tests are failing:


FAILED tests/dataframe/test_drop_pytest.py::TestDataFrameDrop::test_drop_rows
FAILED tests/dataframe/test_drop_pytest.py::TestDataFrameDrop::test_drop_columns
FAILED tests/dataframe/test_dtypes_pytest.py::TestDataFrameDtypes::test_select_dtypes
FAILED tests/dataframe/test_filter_pytest.py::TestDataFrameFilter::test_filter_columns_items[items0]
FAILED tests/dataframe/test_filter_pytest.py::TestDataFrameFilter::test_filter_columns_items[items2]
FAILED tests/dataframe/test_filter_pytest.py::TestDataFrameFilter::test_filter_columns_like[Flight]
FAILED tests/dataframe/test_filter_pytest.py::TestDataFrameFilter::test_filter_columns_regex[^Flig]
FAILED tests/dataframe/test_filter_pytest.py::TestDataFrameFilter::test_filter_columns_regex[^Flight.*r$]
FAILED tests/dataframe/test_filter_pytest.py::TestDataFrameFilter::test_filter_columns_regex[.*]
FAILED tests/dataframe/test_filter_pytest.py::TestDataFrameFilter::test_filter_columns_regex[^[^C]]
FAILED tests/dataframe/test_filter_pytest.py::TestDataFrameFilter::test_filter_index_items
FAILED tests/dataframe/test_get_pytest.py::TestDataFrameGet::test_get_one_attribute
FAILED tests/dataframe/test_getitem_pytest.py::TestDataFrameGetItem::test_getitem_one_argument
FAILED tests/dataframe/test_to_csv_pytest.py::TestDataFrameToCSV::test_to_csv_full
FAILED tests/dataframe/test_to_csv_pytest.py::TestDataFrameToCSV::test_pd_to_csv_without_filepath
FAILED tests/series/test_arithmetics_pytest.py::TestSeriesArithmetics::test_ecommerce_datetime_comparisons
FAILED tests/series/test_name_pytest.py::TestSeriesName::test_name - Assertio...
FAILED tests/series/test_rename_pytest.py::TestSeriesRename::test_rename - As...
FAILED tests/series/test_str_arithmetics_pytest.py::TestSeriesArithmetics::test_ser_add_ser
FAILED tests/series/test_str_arithmetics_pytest.py::TestSeriesArithmetics::test_ser_add_str
FAILED tests/series/test_str_arithmetics_pytest.py::TestSeriesArithmetics::test_str_add_ser
FAILED tests/series/test_str_arithmetics_pytest.py::TestSeriesArithmetics::test_bad_str_add_ser
FAILED tests/series/test_str_arithmetics_pytest.py::TestSeriesArithmetics::test_ser_add_str_add_ser

Could you please check? Thanks.

Arnav-Gr0ver commented 5 months ago

I'll take a look!

dhrubo-os commented 5 months ago

@Arnav-Gr0ver integ tests are still failing. Do you know how to run integ test in your end before pushing a change?

Arnav-Gr0ver commented 5 months ago

From the developer guide "nox -s test", the installed version of python on my machine is 3.11 so I'll setup 3.8 - 3.10 and rerun the command so I can run integ tests without pushing.

opensearch-project / opensearch-py-ml