search

mbraceproject / MBrace.Core

MBrace Core Libraries & Runtime Foundations
http://mbrace.io/
Apache License 2.0
209 stars 46 forks source link

ThespianCluster creates tons of empty json log files #162

Closed cotyar closed 8 years ago

cotyar commented 8 years ago

I'm running a local ThespianCluster from F# interactive and having the temp folder blowing up (Windows 10). Each worker creates tons of empty json log files on name like:

"logs-20160310T213935259Z.json"

at more than 1000/min rate in folders like:

"%PATH%\mbrace-uuid3c64f8b9-3f8c-4b51-a1e4-1ecb76b1535d\systemLogs--29760".

I tried to change the loglevel for the cluster and using different logger but it doen't have any effect on the problem at all.

Is it my incorrect setup or some kind of an expected behaviour please? Many thanks in advance. Yuriy

eiriktsarpalis commented 8 years ago

Hi,

This is expected behaviour. This is testing a cloud logging implementation in which new log entries are appended as files in a cloud storage service, which we assume to be immutable. I just pushed commit 6b60ffcb6e99ea8bb92f977f64ff96a95ffb756f which tweaks granularity of generated json files, as well as exposing persist interval configurations for thespian workers. Note that setting this too high may affect latency in which logs appear in the client side.

eiriktsarpalis commented 8 years ago

By the way, are the files you're seeing literally empty or do they simply contain too few entries?

cotyar commented 8 years ago

Many thanks Eirik for a very quick reply and the fix of course.

All of them I've seen were empty. But I'll give it another go find the answer as soon as get back to my machine.

Thanks again

Yuriy On 11 Mar 2016 13:28, "Eirik Tsarpalis" notifications@github.com wrote:

By the way, are the files you're seeing literally empty or do they simply contain too few entries?

— Reply to this email directly or view it on GitHub https://github.com/mbraceproject/MBrace.Core/issues/162#issuecomment-195362027 .

cotyar commented 8 years ago

All of them are empty

On Fri, Mar 11, 2016 at 1:34 PM, Yuriy Habarov khabarov@gmail.com wrote:

Many thanks Eirik for a very quick reply and the fix of course.

All of them I've seen were empty. But I'll give it another go find the answer as soon as get back to my machine.

Thanks again

Yuriy On 11 Mar 2016 13:28, "Eirik Tsarpalis" notifications@github.com wrote:

By the way, are the files you're seeing literally empty or do they simply contain too few entries?

— Reply to this email directly or view it on GitHub https://github.com/mbraceproject/MBrace.Core/issues/162#issuecomment-195362027 .

cotyar commented 8 years ago

Looks solved.

Great thanks Eirik!

On Fri, Mar 11, 2016 at 3:16 PM, Yuriy Habarov khabarov@gmail.com wrote:

All of them are empty

On Fri, Mar 11, 2016 at 1:34 PM, Yuriy Habarov khabarov@gmail.com wrote:

Many thanks Eirik for a very quick reply and the fix of course.

All of them I've seen were empty. But I'll give it another go find the answer as soon as get back to my machine.

Thanks again

Yuriy On 11 Mar 2016 13:28, "Eirik Tsarpalis" notifications@github.com wrote:

By the way, are the files you're seeing literally empty or do they simply contain too few entries?

— Reply to this email directly or view it on GitHub https://github.com/mbraceproject/MBrace.Core/issues/162#issuecomment-195362027 .

isaacabraham commented 8 years ago

Check me out, giving a comment a thumbs up!