MoriTanosuke / glacieruploader

A simple java command line application for Amazon Glacier
https://www.kopis.de/blog/2012/08/22/simple-uploader-for-amazon-glacier-archiving/
GNU General Public License v3.0
237 stars 54 forks source link

Logging - org.apache.http.wire DEBUG log - Usefulness and Verbosity #75

Closed patrickscottbest closed 5 years ago

patrickscottbest commented 5 years ago

Hi, I see that during a large multipart upload (gigs/hours to complete) even at a lowly 20Mb upstream, generates lots of log noise. It's not a lot, but at faster throughput (and presumably lower part sizes than my chunky 128MB), this could eat away at system performance, depending on where the java.jar is launched from.

I question the need for the logging, and how to turn it off. Was this something I selected at Maven time, or is there a flag to kill the verbosity using Java at runtime? Does it require code changes?

2019-05-01 14:44:10,122 [main] DEBUG org.apache.http.wire - http-outgoing-38 >> "Q,[0xb5][0xde]=su=[0xa5][0x1a][0xcb]yj[0xf1][0x9f][0x17]Mp[0xe7]>[0xe5][0xf8][0x8b][0xe6][0x9b][0xab]2[0xab]?[0x15][0xc9][0xe0][0xc6][0x90][0xc8][0xf4]8
[0xcc][0xa3][0xc8]T[0xcc]8`[\n]"
2019-05-01 14:44:10,123 [main] DEBUG org.apache.http.wire - http-outgoing-38 >> "u[0x9c][0x9][0xfd]V5[0xee]Ba[0x11]9i[0x98][0xe9]j[0xdc][0xb4][0x15][0x81][0xef][0xa1]1[0xb5][0xde][0x16][0xe4]'d[0xbb]~[0x6][0x88].l[0xda]TD[0xb7][0xc4]
O[0x1f][0xf5]x[0xc4]L[0xd3][0xc6][0xde][0xfe]_[0x95]/U[0x15][0xae]g[0xc0][0xa3][0xe0]([0xf8][0xb7]C[0xbd][0x1f]oL[0x7][0xc]z@[0x18][0xb]Lf[0xfa][0xdf][0x6][0x9d]{[0x9d][0x8a]u[\r][0x1d]m[0x97][0xca][0xef][0xfe][0xb7]`}[0x1][0x9][0xfa
][0xdc][0x4][0x98][0x6][0xa2][0xd0];[0x8][0x19][0x92]|[0xe2][0xea]2[0x8d][0xe7][0xf4]i[0xbe]u[0xc][0xfe]?[0xdb]B>a[0x19][0xed][0xca][0xb1][0x8a]%[0xa9][0x93][0x1][0x96]D[0x89][0xc8][0xe3]F[0x1][0x13][0xb4][0xed][0xf6][0xa5]hGa[0xf4][
0xfb][0xcf]%b.[0x8f]g[0xa4][0xc9][0xac]f9[0xf0]F [0xc4];[0x1d][0xeb]I(L[0xaf][0xe6][\r]Ej[0xb][0xdd][0xc8][0x8e][0xe6][0xb6][0x14]'[0xc6],G[0x98]f[0x1e][0x96][0x97][0x11][0x91][0x3]Y[0xbb][0x1]qk[0xc8][\r]j[0xb4][0xd2];[0x15]j[0xcb][
0x9d]k[0xf3][0xc6]<[0xa3]H~([0x99][0xfd][0x19][0x93][0x17]!|[0xf7] [0xe][0xa9][0x86][0xb0][0x9f][0x88]^'[0x17]"W|[0xd4][0xff][0x7]A[0xa5][0xc7]0[0xa5]=[0xdf][0xfe][0xdb][0x9f]y[0xdc]R[0xaf][0xdd][0x95]j[0xe2][0x0][0xcb]+[0xb8][0x9a]0
[0xea]l[0x13][0x1f][0x1c]E[0xc2][0xcf][0x91]E[0x3]#[0xb2][0xea] [0x87]][0x17][0xdb][0xfc][0xf8]Gs.[0x9d]Xd[0x92][0x0][0xf6] [0xcc][0x6][0x8f]d*|c[0xd8][0xd0][0x82][0x14]$[0xed]T[0x84][0x84]e[0xb8][0xf9][0x13][0xf6][0xa1][0x99]#[0xdc]
[0xee][0x8][0xc8][0x14][0x94][0xe4][0xb4]a@[0x11][0xeb][0x9c][0xff]`[0xac]c[0xb2][0xa6]k-?[0x2][0xba].B=[0x99][0xd7]"X[0xf9]3[0x8]{[0xa8]7m[0xc1]}[0x90]?[0xc4][0x89][0x19]v[0x9c]V[0x10][0xac]r[0x12]>V1[0xb3][0x7][0x3][0x2][0x87][0xef
]|[0xe9]-]S[0x14][0x83][\n]"
2019-05-01 14:44:10,123 [main] DEBUG org.apache.http.wire - http-outgoing-38 >> "0[0xd8][0xeb]

Here's the 10 logs that roll over every minute:


-rw-r--r--.  1 user user  415128 May  1 14:41 glacieruploader.log
-rw-r--r--.  1 user user 1052188 May  1 14:41 glacieruploader.log.1
-rw-r--r--.  1 user user 1050036 May  1 14:41 glacieruploader.log.10
-rw-r--r--.  1 user user 1049419 May  1 14:41 glacieruploader.log.2
-rw-r--r--.  1 user user 1050103 May  1 14:41 glacieruploader.log.3
-rw-r--r--.  1 user user 1048992 May  1 14:41 glacieruploader.log.4
-rw-r--r--.  1 user user 1050496 May  1 14:41 glacieruploader.log.5
-rw-r--r--.  1 user user 1050510 May  1 14:41 glacieruploader.log.6
-rw-r--r--.  1 user user 1049827 May  1 14:41 glacieruploader.log.7
-rw-r--r--.  1 user user 1049969 May  1 14:41 glacieruploader.log.8
-rw-r--r--.  1 user user 1049367 May  1 14:41 glacieruploader.log.9

[user@localhost glacieruploader]# wc -l glacieruploader.log*
     618 glacieruploader.log
     979 glacieruploader.log.1
     948 glacieruploader.log.10
     919 glacieruploader.log.2
    1003 glacieruploader.log.3
     975 glacieruploader.log.4
     909 glacieruploader.log.5
     932 glacieruploader.log.6
     918 glacieruploader.log.7
     968 glacieruploader.log.8
    1002 glacieruploader.log.9
   10171 total

I'm not a proficient java coder, but i did find another github project addressing this.

https://github.com/jfrog/artifactory-client-java/issues/77

MoriTanosuke commented 5 years ago

The project is using log4j, a very common log system in Java applications. You can configure it to your needs following the docs at https://logging.apache.org/log4j/2.x/manual/configuration.html