Stat Tracking, DB Schema changes, Checkpoint changed

[jaonewguy]

For #243

Still need:

• Testing and fix tests
• Decision on FCFCheckpoint versions < 1.1.2 (or version of next release)
• Comments
• Some code "structure" decisions, mostly in TODO: comments

Would really like:

• Impl polling rate success per exchange. I think I will implement something easily trackable in RetryableError

Some discussion in looking ahead:

Questions:

• central_currency for central fiat currency like "USD" for the bot. Unless you think this useless?
• future columns e1_poll_success_rate and e2_poll_success_rate for storing the percentage-fraction of times a bot poll is successful against an exchange, in terms of the decimal (so 0.1055 for 10.55% success)
  • this is just a basic rate, so probably want perhaps a separate log file of some sort to show even more details of this? E.g. types of errors so ReadTimeout vs KeyError vs ValueError. Not sure on this one.
  • Something like num_fatal_err for number of fatal errors, would that be a useful column?
• How about other stats like "uptime" that are calculated in reporting now? Should that just be in the fcf_measures table?

[ronaldlam]

Regarding the questions:

• Q: central_currency for central fiat currency like "USD" for the bot. Unless you think this useless?
• A: Useful, let's keep. May want to have extra changes related to forex.
• Q: future columns e1_poll_success_rate and e2_poll_success_rate for storing the percentage-fraction of times a bot poll is successful against an exchange, in terms of the decimal (so 0.1055 for 10.55% success)
  • this is just a basic rate, so probably want perhaps a separate log file of some sort to show even more details of this? E.g. types of errors so ReadTimeout vs KeyError vs ValueError. Not sure on this one.
  • Something like num_fatal_err for number of fatal errors, would that be a useful column?
• A: I think the raw numbers will be more useful instead of rate, since it gives us info about uptime too. Something like total_polls and failed_polls. Are you thinking of num_fatal_errors of being incremented every time the bot dies? This is a tentative yes from me, it'll tell us how often we expect to go down overall.
• Q: How about other stats like "uptime" that are calculated in reporting now? Should that just be in the fcf_measures table?
• A: That would be nice. It would be significantly more accurate too.

[jaonewguy]

A: I think the raw numbers will be more useful instead of rate, since it gives us info about uptime too. Something like total_polls and failed_polls. Are you thinking of num_fatal_errors of being incremented every time the bot dies? This is a tentative yes from me, it'll tell us how often we expect to go down overall.

Re: total_polls and failed_polls relating to uptime. Perhaps this should be held back until the exchange-specific "successes" and "fails" due to Exceptions are decided. I am not sure how useful the current poll statistic is right now, since we are missing all the NetworkError retries. Re: num_fatal_errors This would be easy to add, and if you think it is useful, I think I will just add this column, and track it at autotrageur level.

Q: How about other stats like "uptime" that are calculated in reporting now? Should that just be in the fcf_measures table? A: That would be nice. It would be significantly more accurate too.

Also trade_count can be tracked through stat_tracker, which is fetched manually through reporting. Would that be useful or is it just easily fetched from DB anyway.

[ronaldlam]

trade_count is a tossup for me, you can use your discretion

[jaonewguy]

Unsure why, but it seems like I can't access the link from "Details" link of the CI now. I do see that it was run on worker1 10 hours ago though.

[ronaldlam]

Remember to update the logging_utils path with the fp_libs udpate