Open MarkoPaasila opened 5 years ago
@MarkoPaasila statistics sounds good. Shall we re-open?
I think statistics are important. Do you like the idea of reporting with a single click? If it's as easy as possible, then people are likely to use it, but it won't violate privacy.
If you ask me, I would like to see something like checkbox "enable periodic statistic reporting". The following metrics are interesting:
All metrics except account may be optional. These metrics are needed to assess strategy performance and find weak points.
yes ok. Lets reopen this so that community developers can elect to contribute
@bitfag that's a good idea. It will be of interest to see:
Should this be made into an epic, as the feature is of no use without the necessary infrastructure
The jabber mysteriously stopped posting to t.me/DEXBotStats last month.
AFAIK it was setup by ihaywood. We've not heard from him so it's time that we develop our own.
We can develop a tickbox allowing data-reporting that sends the account name to a server running a BitShares node(?) to quickly read the data and publish it in real time(?) on dexbot.info and t.me/DEXBotStats.
Data such as:
If this same infrastucture could do the same analysis but for the wider blockchain as a whole (maybe a snapshot once per day at the same time every day?) then the percentage of liquidity provided by DEXBot over time can be estimated.
If we can graph liquidity over time for the entire BitShares Network we'll have some really rich data for statistical analysis and the ability to judge the success of any concentrated PR efforts. For example if a BitShares exchange made a big post about DEXBot in their very popular discord channel on the 1st of the month... how did that affect the liquidity available on their markets on the 14th of the month?
This would be quite a lot of work but I think worth it. Then all this could be paired with hosting a new BitShares node.
If the actual hosting of the node is an issue we could fund another BitShares Team to host the node for us.
To gather data, we need something like a simple API server. We cannot expect all bitshares nodes will also run dexbot statistics service.
To prevent statistics abuse, messages from dexbots can be signed with account active key.
If you ask me, I would like to see something like checkbox "enable periodic statistic reporting".
I think it should be initially checked and people are able to disable sending statistics in the preferences or we could create a dialog which opens just one time where the user initially can decide to active or deactivate it after the decision he is still able to change it. Otherwise I don't think a lot of users will activate the "tracking" by theirself, since they don't really profit from it.
I've asked @dominic22 if he is able to handle this after the windows installer.
@dominic22 has written up a spec of work for this feature, and is now subject to review/feedback from @MarkoPaasila @thehapax @joelvai If all goes well work can start v soon
DEXBot- Statistical Collection
Req. 1000: DEXBot can allow data collection by using a tick box.
Implementation of a checkbox at the settings page, next the nodes selection page of DEXBot. If checked, DEXBot shall send statistics to an API service containing the details specified in Req. 1001. If unchecked, no data shall be sent to the server.
Req. 1001: Send statistical data to the server including: worker usernames, SO strategies, RO strategies, total amount of each asset.
DEXBot shall send the following data for each active worker to the backend by using HTTP-PUSH: Username, workername, SO strategies or RO strategies, asset, asset pair and the order size To verify the transmitted data, the frontend shall sign the message by using the active key of the account. The backend server shall check the signature and if successful add it to the collected data as described in Req. 1002.
The dexbot statistic fields are specified as following:
Accountname: account_name :string
Workername: worker_name : string
Strategy: strategy : so_strategy || ro_strategy
Base Asset: base_asset : string
Quote Asset: quote_asset : string
Order Size: order_size : number
Req. 1002: Sum each user data to provide the total amount of active strategies and the total size of the assets.
The data sent by an account shall be summed by the Backend API to provide the total amount of strategies and used liquidity for each asset pair. Whenever a user turns off a worker, the backend shall remove the amount of the asset pair for the specific worker from the total aggregation. To be able to determine obsolete workers at the backend during frontend failures, the frontend shall send in addition every 30 minutes a heartbeat message for each worker. When the heart beat expires for an account, the backend will remove the amount of the asset pair for the specific worker from the total aggregation, if the worker is still marked as active in the backend. This provides a robust solution to determine obsolete workers whenever something goes wrong on the machine running DEXBot, for instance a network error. On a network error, the worker won’t be able to deregister at the backend.
Req 1003: API for aggregated statistical data
The backend shall provide a Rest API, which can be accessed to request the data by using HTTP- GET.
The Rest API provides the following JSON structure: Array with the following information:
Base Asset: base_asset : string Quote Asset: quote_asset : string SO-Strategy Size: so_stragegy_size: number RO-Strategie Size: ro_strategy_size: number
Req. 1004: Display statistical data at a telegram channel.
The collected data shall be displayed in an open to join telegram channel. Therefore a new telegram channel shall be created. The statistical data shall be displayed on a daily interval in the group. To receive the data in the telegram channel, the channel will request the provided API of Req. 1003.
Req. 1005: Display data on the dexbot.info website.
The aggregated statistical data, which will be requested as described in Req. 1003, shall be displayed on the website dexbot.info. Therefor the website shall have an additional menu entry “Statistics” which can be selected to display the provided DEXBot statistics.
After further thoughts, I could also imagine to open a socket connection to the backend service, to detect obsolete workers, this would eliminate the heart beat and the backend will receive an event if the socket connection get closed, which is less implementation effort.
@bitfag what do you think?
When worker has been stopped the statistics service should not totally remove liquidity of this bitshares account from calculation. If worker isn't running, but orders kept on books it should not be removed. This is normal situation for SO strategy. Optionally, a separate metric could be provided: 1) liquidity provided by online workers, 2) liquidity provided by offline workers. If worker will not return online after 1-2 months, remove it from accounting (assume user has stopped using dexbot).
Heartbeat or persistent connection: I need to think more on this. Do we really need so precise stats gathering? Isn't it enough to just send data once per interval when worker is running, and don't send data when it's stopped?
API should reflect 1001 API (provide strategy specific metrics). Also, plan to extend number of metrics.
chain_id
will do the trick. Otherwise, we need to include chain_id
as a metric to see on what chain dexbot is being used.1001 I agree on what @bitfag said. Worker's name is in my opinion irrelevant information.
1002 I agree here as well, it is good to remember that it is possible to just close DEXBot and leave the RO orders there on purpose. As well as one might run DEXBot once and setup SO and relaunch DEXBot week after so these orders should still be count.
Since collecting data outside the local DEXBot environment GDPR might be something to keep in mind just in case.
Thanks for the feedback.
1001 I just wanted to have the worker name to differentiate between orders of an account. If we can detect if an order is closed or filled while DEXBot is offline this might no be needed though. Since you only want to count orders on the bitshares network we may need to send the chain_id as well or shall we handle it on the DEXBot frontend ?
So new data model could look like:
interface IMetrics {
chain_id:string;
account_name :string
strategy : IStaggeredOrder || IRelativeOrder
}
interface IStaggeredOrder {
strategy: string (dexbot.strategies.staggered_orders)
base_asset : string;
quote_asset : string;
mode: Mode;
order_size? : number;
}
enum Mode {
1 = Mountain,
2 = Neutral,
3 = Valley,
4 = Buy Slope,
5 = Sell Slope
}
interface IRelativeOrder {
strategy: string (dexbot.strategies.relative_orders)
base_asset : string;
quote_asset : string;
order_size?: number;
}
1002 So if we say the orders should be counted as liquidity also when DEXBot is closed, how can we check wether the order is still there and not filled. I think bitshares provides some events to listen for the order or shall the service check the status of the orders in a given intervall ?
If there are some events or something to check, if an order is still active and we do not check wether DEXBot is actively running, I think heartbeat will be irrelevant and so also the socket connection. We could add an timestamp when s.o. sets an order using DEXBot and remove it after 1-2 months if the order is still in the order book, as you recommended.
1003
API should reflect 1001 API (provide strategy specific metrics). Also, plan to extend number of metrics.
But it have to return the accumulated metrics or do you want to get an array of each order? I think this could be a lot of data when there are a lot of users, if we do not accumulate the metrics.
Techonology / Infrastructure
As this is a stand alone tool I think NodeJs would be acceptable
Pybts and pygraphene may be of use https://github.com/bitshares/python-bitshares
@gabev where is the source code for dexbot.info?
Account open orders could be obtained by Account.openorders
(pybitshares). Periodic check should be fine I think.
No, I just mean that backend API should be aware of strategy-specific data, for example a metric "number of workers on valley mode" etc
A second vote here for a python backend. Definitely would lean toward ease of maintainability for the project as it grows.
@dominic22 one other thing -- if you are still looking at flask, there is tornado, which is non-blocking and has async. you can see a bit about the comparison here: https://stackshare.io/stackups/flask-vs-tornado
Having the bot automatically, and without explicit permission, report all active accounts for statistics violates people's right to privacy and freedom of choice in my opinion. But we should make it as easy as possible.
To make reporting accounts to the statistics as easy as possible, a button on the dashboard that does it all automatically would be the best solution. It should say: "Thank you!", and report each account used in a worker.
The link for reporting is: http://jabber.twilightparadox.com/dexbot.shtml