search

erikng / Cacher

A python script for extracting macOS Caching Server data
Apache License 2.0
134 stars 40 forks source link

Small data amounts cause parsing failure #43

Closed webdeck closed 7 years ago

webdeck commented 7 years ago

I see this error:

./cacher.py --targetdate=2017-06-20
Traceback (most recent call last):
  File "./cacher.py", line 950, in <module>
    main()
  File "./cacher.py", line 933, in main
    cacherdata = cacher(rawLog.readlines(), targetDate, friendlyNames)
  File "./cacher.py", line 177, in cacher
    frompeers_size = linesplit[17]
IndexError: list index out of range

It seems to be because the "Served all" line isn't using KB / MB / or GB:

bzgrep "Served all" /Library/Server/Caching/Logs/Debug-2017.06.20-07:16.log.bz2
2017-06-20 09:27:13.810 #ASDOW8CvyaXH Served all 530 of 530 bytes; 0 from cache, 530 stored from Internet, 0 from peers
2017-06-20 09:27:14.228 #Gv2SiVDKnjfx Served all 3061 of 3061 bytes; 0 from cache, 3061 stored from Internet, 0 from peers
2017-06-20 09:27:14.868 #0Bjo9F70zTHp Served all 530 of 530 bytes; 530 from cache, 0 stored from Internet, 0 from peers
2017-06-20 09:27:14.893 #Rv5z5LZVfqs/ Served all 3061 of 3061 bytes; 3061 from cache, 0 stored from Internet, 0 from peers
2017-06-20 09:27:17.353 #BYopVZ61tik+ Served all 3061 of 3061 bytes; 3061 from cache, 0 stored from Internet, 0 from peers
2017-06-20 19:08:24.110 #HRFS8tdWl/oo Served all 2671077 of 2671077 bytes; 0 from cache, 2671077 stored from Internet, 0 from peers
2017-06-20 22:09:51.306 #OYiGr6J5oTU2 Served all 3530118 of 3530118 bytes; 0 from cache, 3530118 stored from Internet, 0 from peers
2017-06-20 22:09:51.535 #Hi/jXmnizsOC Served all 2518340 of 2518340 bytes; 0 from cache, 2518340 stored from Internet, 0 from peers
2017-06-20 22:34:11.190 #3t7x/+Ft9289 Served all 5022 of 5022 bytes; 5022 from cache, 0 stored from Internet, 0 from peers
2017-06-20 22:34:11.190 #qB0Ld3yWIDf0 Served all 5023 of 5023 bytes; 5023 from cache, 0 stored from Internet, 0 from peers
2017-06-20 22:34:11.320 #HdH6mXxXQpq/ Served all 5639 of 5639 bytes; 5639 from cache, 0 stored from Internet, 0 from peers
2017-06-20 22:34:11.695 #MFU/uJqbMIOr Served all 5024 of 5024 bytes; 5024 from cache, 0 stored from Internet, 0 from peers
2017-06-20 22:34:13.309 #okBzTwaHA8D9 Served all 5979 of 5979 bytes; 5979 from cache, 0 stored from Internet, 0 from peers
2017-06-21 00:02:20.900 #FDN+/XnFoNtc Served all 2671077 of 2671077 bytes; 2671077 from cache, 0 stored from Internet, 0 from peers
2017-06-21 02:03:57.147 #dqwnWQVRnLIB Served all 35055540 of 35055540 bytes; 13572 from cache, 35041968 stored from Internet, 0 from peers

I'm using the latest version of El Capitan's Server app.

erikng commented 7 years ago

Can you send me the entire debug log?

Thanks, Erik Gomez

From: Mike Riccio notifications@github.com Sent: Wednesday, June 21, 2017 4:51:52 PM To: erikng/Cacher Cc: Subscribed Subject: [erikng/Cacher] Small data amounts cause parsing failure (#43)

I see this error:

./cacher.py --targetdate=2017-06-20 Traceback (most recent call last): File "./cacher.py", line 950, in main() File "./cacher.py", line 933, in main cacherdata = cacher(rawLog.readlines(), targetDate, friendlyNames) File "./cacher.py", line 177, in cacher frompeers_size = linesplit[17] IndexError: list index out of range

It seems to be because the "Served all" line isn't using KB / MB / or GB:

bzgrep "Served all" /Library/Server/Caching/Logs/Debug-2017.06.20-07:16.log.bz2 2017-06-20 09:27:13.810 #ASDOW8CvyaXH Served all 530 of 530 bytes; 0 from cache, 530 stored from Internet, 0 from peers 2017-06-20 09:27:14.228 #Gv2SiVDKnjfx Served all 3061 of 3061 bytes; 0 from cache, 3061 stored from Internet, 0 from peers 2017-06-20 09:27:14.868 #0Bjo9F70zTHp Served all 530 of 530 bytes; 530 from cache, 0 stored from Internet, 0 from peers 2017-06-20 09:27:14.893 #Rv5z5LZVfqs/ Served all 3061 of 3061 bytes; 3061 from cache, 0 stored from Internet, 0 from peers 2017-06-20 09:27:17.353 #BYopVZ61tik+ Served all 3061 of 3061 bytes; 3061 from cache, 0 stored from Internet, 0 from peers 2017-06-20 19:08:24.110 #HRFS8tdWl/oo Served all 2671077 of 2671077 bytes; 0 from cache, 2671077 stored from Internet, 0 from peers 2017-06-20 22:09:51.306 #OYiGr6J5oTU2 Served all 3530118 of 3530118 bytes; 0 from cache, 3530118 stored from Internet, 0 from peers 2017-06-20 22:09:51.535 #Hi/jXmnizsOC Served all 2518340 of 2518340 bytes; 0 from cache, 2518340 stored from Internet, 0 from peers 2017-06-20 22:34:11.190 #3t7x/+Ft9289 Served all 5022 of 5022 bytes; 5022 from cache, 0 stored from Internet, 0 from peers 2017-06-20 22:34:11.190 #qB0Ld3yWIDf0 Served all 5023 of 5023 bytes; 5023 from cache, 0 stored from Internet, 0 from peers 2017-06-20 22:34:11.320 #HdH6mXxXQpq/ Served all 5639 of 5639 bytes; 5639 from cache, 0 stored from Internet, 0 from peers 2017-06-20 22:34:11.695 #MFU/uJqbMIOr Served all 5024 of 5024 bytes; 5024 from cache, 0 stored from Internet, 0 from peers 2017-06-20 22:34:13.309 #okBzTwaHA8D9 Served all 5979 of 5979 bytes; 5979 from cache, 0 stored from Internet, 0 from peers 2017-06-21 00:02:20.900 #FDN+/XnFoNtc Served all 2671077 of 2671077 bytes; 2671077 from cache, 0 stored from Internet, 0 from peers 2017-06-21 02:03:57.147 #dqwnWQVRnLIB Served all 35055540 of 35055540 bytes; 13572 from cache, 35041968 stored from Internet, 0 from peers

I'm using the latest version of El Capitan's Server app.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHubhttps://github.com/erikng/Cacher/issues/43, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AFa_GMM4wB7KQ4ntih6ItLSFUBXNYuLWks5sGZB4gaJpZM4OBkHI.

webdeck commented 7 years ago

Well, I stopped the caching server, blew away its preferences file, restarted it, and reconfigured it, and now this is working. So something about how my caching server has been upgraded over the years was causing it to log differently. Now I get the expected output:

2017-06-27 11:39:24.222 #uKmIZ0Lhg7LG Served all 18.3 MB of 18.3 MB; 13 KB from cache, 18.3 MB stored from Internet, 0 bytes from peers
erikng commented 7 years ago

Interesting. Mind if I close this then?

Thanks, Erik Gomez

From: Mike Riccio notifications@github.com Sent: Tuesday, June 27, 2017 2:43:30 PM To: erikng/Cacher Cc: Erik Gomez; Comment Subject: Re: [erikng/Cacher] Small data amounts cause parsing failure (#43)

Well, I stopped the caching server, blew away its preferences file, restarted it, and reconfigured it, and now this is working. So something about how my caching server has been upgraded over the years was causing it to log differently. Now I get the expected output:

2017-06-27 11:39:24.222 #uKmIZ0Lhg7LG Served all 18.3 MB of 18.3 MB; 13 KB from cache, 18.3 MB stored from Internet, 0 bytes from peers

— You are receiving this because you commented. Reply to this email directly, view it on GitHubhttps://github.com/erikng/Cacher/issues/43#issuecomment-311464081, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AFa_GFsUc5FjpRYfRWVRtDK-TtmxQn_Nks5sIVtigaJpZM4OBkHI.

webdeck commented 7 years ago

Sure. Closing it.