search

TeamMentor / TM_4_0_Design

Repo Holds TM 4.x issues
4 stars 3 forks source link

Identify which types of data need to be removed from logs #553

Open DinisCruz opened 9 years ago

DinisCruz commented 9 years ago

Important when handling the case where we can (optionally) use 3rd party logging services

Note that the Winston logging api has as helper method to handle such cases (for example scrubbing CC details or session information)

romichg commented 9 years ago

Related to https://github.com/TeamMentor/TM_4_0_Design/issues/550 I am putting this for Sprint 3 Backlog for now as we have no immediate requirement for this.

DinisCruz commented 9 years ago

I actually think that we do have an immediate requirement for beta.teammentor.net

if we want to understand how TM is being used by real users, then this is the way to do it. Take a look at the LogEntries and Loggly videos (from their site). We really will be able to understand what is going on using the log data + these SaaS services.

Yes we need to think about what is being sent into these services, but that is not that hard to do

This is also very important from a development point of view, since it really helps to understand what is going on in a live server (in addition to giving us early warning signs of any bugs or probs (like the 404 on articles with dots on titles)

romichg commented 9 years ago

Understanding user behavior is a subset of centralized logging. We will address understanding of user behavior with https://github.com/TeamMentor/TM_4_0_Design/issues/534 Centralized logging is a big beast (I've done lots of it) that we do not need to tackle right now.

From the Dev perspective, you have RDP access to the server :)

Again, I am not saying this is not important, I am saying that this is a bit bigger issue than a 4h and we should focus on other things right now.

DinisCruz commented 9 years ago

yeah, but that workflow (having to RDP) is not really a good one, specially since there isn't a live console view (due to the fact that node is running as a service)

A practical example of this inefficiencies is the workflow to debug (and fix) the http://beta.teammentor.net/poc/ bug

image

DinisCruz commented 9 years ago

you are also massive underestimating how important it will be for the dev team (and operations) to have quick feedback on what is happening on the live server

Check out the posts at https://codeascraft.com/ for good examples of it (for example https://codeascraft.com/2011/02/15/measure-anything-measure-everything/ )

romichg commented 9 years ago

We need to be able to get into the habit of diagnosing the issues post mortem based on the logs. The work flow should be.

  1. Get the logs.
  2. Look at the logs and understand the problem
  3. Replicate the problem on the local instance. This is the only way we will be able to solve customer issues.

Regarding RDP here is how to get a console to view the logs:

  1. RDP into the box
  2. Open Git Bash shell
  3. cd to the directory that contains the logs
  4. tail -f /logname/
DinisCruz commented 9 years ago

I just added those instructions to https://github.com/TeamMentor/TM_4_0_Design/wiki/Debugging-issues-on-beta.teammentor.net