[Feature Request]: New rpc call export_swaps_csv

[Milerius]

Is your feature request related to a problem? Please describe.

Related to #784

In the same idea since the pagination is from mm2 side now, the swaps export is no longer correct for atomicdex-desktop.

Describe the solution you'd like

I would like to have a rpc call export_swaps_csv that can take same filtering parameters as #784 + a path for saving the file.

///! 1st example export whole history between specific range
{
  "userpass": "rpc_password",
  "method": "export_swaps_csv",
  "max": true,
  "filtering": {"date_range": [1609223333, 1609225294]},
 "path": "/tmp/history.csv"
}

///! 2nd example export n first elements in history that respect specific range
{
  "userpass": "rpc_password",
  "method": "export_swaps_csv",
  "limit": 50,
  "filtering": {"date_range": [1609223333, 1609225294]},
 "path": "/tmp/history.csv"
}

///! 3rd example export n first elements in history that respect specific range and that have for base coin `RICK`
{
  "userpass": "rpc_password",
  "method": "export_swaps_csv",
  "limit": 50,
  "filtering": {"date_range": [1609223333, 1609225294], "pattern": "RICK/*"},
 "path": "/tmp/history.csv"
}

///! 4th example export a page with 50 element
{
  "userpass": "rpc_password",
  "method": "export_swaps_csv",
  "limit": 50,
  "page": [1],
 "path": "/tmp/history.csv"
}

///! 4th example export multiple pages at once with 50 elements per page (150 total)
{
  "userpass": "rpc_password",
  "method": "export_swaps_csv",
  "limit": 50,
  "page": [1, 2, 3],
 "path": "/tmp/history.csv"
}

I don't know how answer would look like but you can return {"result": "success"} in case of success or the whole csv as plain text.

I will share how i implemented it in C++:

void
    orders_proxy_model::export_csv_visible_history(const QString& path)
    {
        const fs::path csv_path = path.toStdString();
        SPDLOG_INFO("exporting csv with path: {}", csv_path.string());
        std::ofstream ofs(csv_path.string(), std::ios::out | std::ios::trunc);
        int           nb_items = this->rowCount();
        ofs << "Date, BaseCoin, BaseAmount, Status, RelCoin, RelAmount, UUID, ErrorState" << std::endl;
        for (int cur_idx = 0; cur_idx < nb_items; ++cur_idx)
        {
            QModelIndex idx = this->index(cur_idx, 0);
            ofs << this->data(idx, orders_model::OrdersRoles::HumanDateRole).toString().toStdString() << ",";
            ofs << this->data(idx, orders_model::OrdersRoles::BaseCoinRole).toString().toStdString() << ",";
            ofs << this->data(idx, orders_model::OrdersRoles::BaseCoinAmountRole).toString().toStdString() << ",";
            auto status = this->data(idx, orders_model::OrdersRoles::OrderStatusRole).toString().toStdString();
            ofs << status << ",";
            ofs << this->data(idx, orders_model::OrdersRoles::RelCoinRole).toString().toStdString() << ",";
            ofs << this->data(idx, orders_model::OrdersRoles::RelCoinAmountRole).toString().toStdString() << ",";
            ofs << this->data(idx, orders_model::OrdersRoles::OrderIdRole).toString().toStdString();
            if (status == "failed")
            {
                ofs << "," << this->data(idx, orders_model::OrdersRoles::OrderErrorStateRole).toString().toStdString() << std::endl;
            }
            else
            {
                ofs << ",Success" << std::endl;
            }
        }
        ofs.close();
    }

Example of a csv file:

Date, BaseCoin, BaseAmount, Status, RelCoin, RelAmount, UUID, ErrorState
2020-09-14    19:21:36.594,RICK,2,failed,MORTY,1,6addb09b-65ea-43a1-b64a-ea29ffd44e9b,TakerPaymentWaitConfirmFailed
2020-09-13    08:17:51.366,KMD,103.83944625,successful,DEX,6.11102357,da797ff0-c9c9-4785-bca0-de080ecda052,Success
2020-09-13    07:49:40.271,KMD,37.58094324,failed,DEX,2.51971619,6c5e34a5-a648-482d-945a-10299d957237,NegotiateFailed

Describe alternatives you've considered

Fetch all the swaps once, and then apply filtering myself then exporting the visible history, but with huge swaps history this cost too much memory and this is slow.

Additional context

Maybe related to the fact that MM2 plan to move DB to smth like: embedded DB like SQLite, will be easier to use filtering in this case

Also related to https://github.com/KomodoPlatform/atomicDEX-Desktop/pull/621

[artemii235]

Thanks for opening the issue! I think different GUIs can have different export formats, and it might be hard for MM2 to support all of them. So I prefer to implement the following solution:

1. Add all_swaps_uuids_by_filter RPC, which returns all UUIDs of swaps that match selected filters (request format will be the same as for my_recent_swaps).
2. GUIs can retrieve all the UUIDs and then call my_swap_status for each mapping the JSON to the GUI specific format.

This will allow avoiding huge RAM usage on requesting big data set: 1 UUID in string format takes 36 bytes so maximum memory usage will be number_of_returned_swaps * 36 which seems quite acceptable even for hundreds of thousands of swaps.

I will do it today.

[artemii235]

I've added the new RPC, docs diff: https://github.com/KomodoPlatform/developer-docs/pull/235/commits/542c76876226b0248a73a4d0adef7727628ec4d9. Release will be published by CI soon.