Log records get lost if they wind up in the same chunk as a bad log record

[onnheimm]

When some log records contain data that cannot be serialized, log records in the same chunk get lost. The following code provokes the issue.

import logging
from cmreslogging.handlers import CMRESHandler

h = CMRESHandler(hosts = [dict(host='localhost',port=9200)],es_index_name = 'debug', raise_on_indexing_exceptions=False)
log = logging.getLogger(__name__)
log.setLevel(logging.INFO)
log.addHandler(h)

import numpy as np

bad_data = np.int64(1)   #unserializable by json module
good_data = 1

for i in range(100):
    log.info('Test', extra = dict(a_number = bad_data)) 
    log.info('Test', extra = dict(a_number = good_data))  #this also gets lost

[cmanaha]

Makes sense; I've seen there is a pull request for this. I'll take a look and merge as soon as I have a spare hour.

Thanks for the contribution! Carlos

On 26 Apr 2017, at 12:04, onnheimm notifications@github.com wrote:

When some log records contain data that cannot be serialized, log records in the same chunk get lost. The following code provokes the issue.

import logging from cmreslogging.handlers import CMRESHandler

h = CMRESHandler(hosts = [dict(host='localhost',port=9200)],es_index_name = 'debug', raise_on_indexing_exceptions=False) log = logging.getLogger(name) log.setLevel(logging.INFO) log.addHandler(h)

import numpy as np

bad_data = np.int64(1) #unserializable by json module good_data = 1

for i in range(100): log.info('Test', extra = dict(a_number = bad_data)) log.info('Test', extra = dict(a_number = good_data)) #this also gets lost — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

[onnheimm]

Great, thanks for a very useful project!

[aldbr]

I tried to run this example, the problem is fixed now thanks to the new serializer. I think the pull request associated is not necessary anymore. What do you think about this ?

[onnheimm]

I agree!