search

jaegertracing / jaeger

CNCF Jaeger, a Distributed Tracing Platform
https://www.jaegertracing.io/
Apache License 2.0
20.47k stars 2.44k forks source link

hotrod example prints all kinds of errors and stackdumps #312

Closed Dieterbe closed 7 years ago

Dieterbe commented 7 years ago

Hi, jaeger looks interesting. currently evaluating it to implement tracing. the hotrod example is a nice demo, but i notice this output:

2017-08-04T16:48:24.945+0200    INFO    log/spanlogger.go:40    Search successful   {"service": "driver", "num_drivers": 10}
2017-08-04T16:48:24.945+0200    INFO    log/spanlogger.go:40    Found drivers   {"service": "frontend", "drivers": [{"DriverID":"T717426C","Location":"19,193"},{"DriverID":"T750238C","Location":"915,137"},{"DriverID":"T717064C","Location":"686,40"},{"DriverID":"T788184C","Location":"914,691"},{"DriverID":"T720428C","Location":"143,168"},{"DriverID":"T747337C","Location":"34,207"},{"DriverID":"T788603C","Location":"593,284"},{"DriverID":"T741644C","Location":"275,748"},{"DriverID":"T791434C","Location":"948,983"},{"DriverID":"T798733C","Location":"983,369"}]}
2017-08-04T16:48:24.945+0200    INFO    log/spanlogger.go:40    Finding route   {"service": "frontend", "component": "route_client", "pickup": "686,40", "dropoff": "577,322"}
2017-08-04T16:48:24.945+0200    INFO    log/spanlogger.go:40    Finding route   {"service": "frontend", "component": "route_client", "pickup": "19,193", "dropoff": "577,322"}
2017-08-04T16:48:24.945+0200    INFO    log/spanlogger.go:40    Finding route   {"service": "frontend", "component": "route_client", "pickup": "915,137", "dropoff": "577,322"}
2017-08-04T16:48:24.945+0200    INFO    log/spanlogger.go:40    HTTP request received   {"service": "route", "method": "GET", "url": "/route?dropoff=577%2C322&pickup=19%2C193"}
2017-08-04T16:48:24.945+0200    INFO    log/spanlogger.go:40    HTTP request received   {"service": "route", "method": "GET", "url": "/route?dropoff=577%2C322&pickup=686%2C40"}
2017-08-04T16:48:24.945+0200    INFO    log/spanlogger.go:40    HTTP request received   {"service": "route", "method": "GET", "url": "/route?dropoff=577%2C322&pickup=915%2C137"}
2017-08-04T16:48:24.985+0200    INFO    log/spanlogger.go:40    Finding route   {"service": "frontend", "component": "route_client", "pickup": "914,691", "dropoff": "577,322"}
2017-08-04T16:48:24.985+0200    INFO    log/spanlogger.go:40    HTTP request received   {"service": "route", "method": "GET", "url": "/route?dropoff=577%2C322&pickup=914%2C691"}
2017-08-04T16:48:24.987+0200    INFO    log/spanlogger.go:40    Finding route   {"service": "frontend", "component": "route_client", "pickup": "143,168", "dropoff": "577,322"}
2017-08-04T16:48:24.987+0200    INFO    log/spanlogger.go:40    HTTP request received   {"service": "route", "method": "GET", "url": "/route?dropoff=577%2C322&pickup=143%2C168"}
2017-08-04T16:48:24.993+0200    INFO    log/spanlogger.go:40    Finding route   {"service": "frontend", "component": "route_client", "pickup": "34,207", "dropoff": "577,322"}
2017-08-04T16:48:24.993+0200    INFO    log/spanlogger.go:40    HTTP request received   {"service": "route", "method": "GET", "url": "/route?dropoff=577%2C322&pickup=34%2C207"}
2017-08-04T16:48:25.017+0200    INFO    log/spanlogger.go:40    Finding route   {"service": "frontend", "component": "route_client", "pickup": "593,284", "dropoff": "577,322"}
2017-08-04T16:48:25.017+0200    INFO    log/spanlogger.go:40    HTTP request received   {"service": "route", "method": "GET", "url": "/route?dropoff=577%2C322&pickup=593%2C284"}
2017-08-04T16:48:25.033+0200    INFO    log/spanlogger.go:40    Finding route   {"service": "frontend", "component": "route_client", "pickup": "275,748", "dropoff": "577,322"}
2017-08-04T16:48:25.034+0200    INFO    log/spanlogger.go:40    HTTP request received   {"service": "route", "method": "GET", "url": "/route?dropoff=577%2C322&pickup=275%2C748"}
2017-08-04T16:48:25.034+0200    INFO    log/spanlogger.go:40    Finding route   {"service": "frontend", "component": "route_client", "pickup": "948,983", "dropoff": "577,322"}
2017-08-04T16:48:25.035+0200    INFO    log/spanlogger.go:40    HTTP request received   {"service": "route", "method": "GET", "url": "/route?dropoff=577%2C322&pickup=948%2C983"}
2017-08-04T16:48:25.048+0200    INFO    log/spanlogger.go:40    Found customer  {"service": "frontend", "customer": {"ID":"392","Name":"Trom Chocolatier","Location":"577,322"}}
2017-08-04T16:48:25.048+0200    INFO    log/spanlogger.go:40    Finding nearest drivers {"service": "frontend", "component": "driver_client", "location": "577,322"}
2017-08-04T16:48:25.049+0200    INFO    log/spanlogger.go:40    Searching for nearby drivers    {"service": "driver", "location": "577,322"}
2017-08-04T16:48:25.049+0200    INFO    log/spanlogger.go:40    Finding route   {"service": "frontend", "component": "route_client", "pickup": "983,369", "dropoff": "577,322"}
2017-08-04T16:48:25.049+0200    INFO    log/spanlogger.go:40    HTTP request received   {"service": "route", "method": "GET", "url": "/route?dropoff=577%2C322&pickup=983%2C369"}
2017-08-04T16:48:25.104+0200    INFO    log/spanlogger.go:40    Found routes    {"service": "frontend", "routes": [{},{},{},{},{},{},{},{},{},{}]}
2017-08-04T16:48:25.105+0200    INFO    log/spanlogger.go:40    Dispatch successful {"service": "frontend", "driver": "T717064C", "eta": "2m0s"}
2017-08-04T16:48:25.110+0200    ERROR   log/spanlogger.go:45    redis timeout   {"service": "driver", "driver_id": "T715719C", "error": "redis timeout"}
github.com/uber/jaeger/examples/hotrod/vendor/go.uber.org/zap.Stack
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/vendor/go.uber.org/zap/field.go:209
github.com/uber/jaeger/examples/hotrod/vendor/go.uber.org/zap.(*Logger).check
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/vendor/go.uber.org/zap/logger.go:273
github.com/uber/jaeger/examples/hotrod/vendor/go.uber.org/zap.(*Logger).Error
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/vendor/go.uber.org/zap/logger.go:176
github.com/uber/jaeger/examples/hotrod/pkg/log.spanLogger.Error
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/pkg/log/spanlogger.go:45
github.com/uber/jaeger/examples/hotrod/pkg/log.(*spanLogger).Error
    <autogenerated>:1
github.com/uber/jaeger/examples/hotrod/services/driver.(*Redis).GetDriver
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/services/driver/redis.go:89
github.com/uber/jaeger/examples/hotrod/services/driver.(*Server).FindNearest
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/services/driver/server.go:91
github.com/uber/jaeger/examples/hotrod/services/driver/thrift-gen/driver.(*tchanDriverServer).handleFindNearest
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/services/driver/thrift-gen/driver/tchan-driver.go:92
github.com/uber/jaeger/examples/hotrod/services/driver/thrift-gen/driver.(*tchanDriverServer).Handle
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/services/driver/thrift-gen/driver/tchan-driver.go:76
github.com/uber/jaeger/examples/hotrod/vendor/github.com/uber/tchannel-go/thrift.(*Server).handle
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/vendor/github.com/uber/tchannel-go/thrift/server.go:133
github.com/uber/jaeger/examples/hotrod/vendor/github.com/uber/tchannel-go/thrift.(*Server).Handle
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/vendor/github.com/uber/tchannel-go/thrift/server.go:203
github.com/uber/jaeger/examples/hotrod/vendor/github.com/uber/tchannel-go.(*handlerMap).Handle
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/vendor/github.com/uber/tchannel-go/handlers.go:118
github.com/uber/jaeger/examples/hotrod/vendor/github.com/uber/tchannel-go.channelHandler.Handle
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/vendor/github.com/uber/tchannel-go/handlers.go:126
2017-08-04T16:48:25.111+0200    ERROR   log/spanlogger.go:45    Retrying GetDriver after error  {"service": "driver", "retry_no": 1, "error": "redis timeout"}
github.com/uber/jaeger/examples/hotrod/vendor/go.uber.org/zap.Stack
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/vendor/go.uber.org/zap/field.go:209
github.com/uber/jaeger/examples/hotrod/vendor/go.uber.org/zap.(*Logger).check
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/vendor/go.uber.org/zap/logger.go:273
github.com/uber/jaeger/examples/hotrod/vendor/go.uber.org/zap.(*Logger).Error
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/vendor/go.uber.org/zap/logger.go:176
github.com/uber/jaeger/examples/hotrod/pkg/log.spanLogger.Error
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/pkg/log/spanlogger.go:45
github.com/uber/jaeger/examples/hotrod/pkg/log.(*spanLogger).Error
    <autogenerated>:1
github.com/uber/jaeger/examples/hotrod/services/driver.(*Server).FindNearest
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/services/driver/server.go:95
github.com/uber/jaeger/examples/hotrod/services/driver/thrift-gen/driver.(*tchanDriverServer).handleFindNearest
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/services/driver/thrift-gen/driver/tchan-driver.go:92
github.com/uber/jaeger/examples/hotrod/services/driver/thrift-gen/driver.(*tchanDriverServer).Handle
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/services/driver/thrift-gen/driver/tchan-driver.go:76
github.com/uber/jaeger/examples/hotrod/vendor/github.com/uber/tchannel-go/thrift.(*Server).handle
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/vendor/github.com/uber/tchannel-go/thrift/server.go:133
github.com/uber/jaeger/examples/hotrod/vendor/github.com/uber/tchannel-go/thrift.(*Server).Handle
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/vendor/github.com/uber/tchannel-go/thrift/server.go:203
github.com/uber/jaeger/examples/hotrod/vendor/github.com/uber/tchannel-go.(*handlerMap).Handle
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/vendor/github.com/uber/tchannel-go/handlers.go:118
github.com/uber/jaeger/examples/hotrod/vendor/github.com/uber/tchannel-go.channelHandler.Handle
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/vendor/github.com/uber/tchannel-go/handlers.go:126
github.com/uber/jaeger/examples/hotrod/vendor/github.com/uber/tchannel-go.(*Connection).dispatchInbound
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/vendor/github.com/uber/tchannel-go/inbound.go:195
2017-08-04T16:48:25.178+0200    ERROR   log/spanlogger.go:45    redis timeout   {"service": "driver", "driver_id": "T778928C", "error": "redis timeout"}
github.com/uber/jaeger/examples/hotrod/vendor/go.uber.org/zap.Stack
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/vendor/go.uber.org/zap/field.go:209
github.com/uber/jaeger/examples/hotrod/vendor/go.uber.org/zap.(*Logger).check
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/vendor/go.uber.org/zap/logger.go:273
github.com/uber/jaeger/examples/hotrod/vendor/go.uber.org/zap.(*Logger).Error
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/vendor/go.uber.org/zap/logger.go:176
github.com/uber/jaeger/examples/hotrod/pkg/log.spanLogger.Error
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/pkg/log/spanlogger.go:45
github.com/uber/jaeger/examples/hotrod/pkg/log.(*spanLogger).Error
    <autogenerated>:1
github.com/uber/jaeger/examples/hotrod/services/driver.(*Redis).GetDriver
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/services/driver/redis.go:89
github.com/uber/jaeger/examples/hotrod/services/driver.(*Server).FindNearest
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/services/driver/server.go:91
github.com/uber/jaeger/examples/hotrod/services/driver/thrift-gen/driver.(*tchanDriverServer).handleFindNearest
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/services/driver/thrift-gen/driver/tchan-driver.go:92
github.com/uber/jaeger/examples/hotrod/services/driver/thrift-gen/driver.(*tchanDriverServer).Handle
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/services/driver/thrift-gen/driver/tchan-driver.go:76
github.com/uber/jaeger/examples/hotrod/vendor/github.com/uber/tchannel-go/thrift.(*Server).handle
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/vendor/github.com/uber/tchannel-go/thrift/server.go:133
github.com/uber/jaeger/examples/hotrod/vendor/github.com/uber/tchannel-go/thrift.(*Server).Handle
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/vendor/github.com/uber/tchannel-go/thrift/server.go:203
github.com/uber/jaeger/examples/hotrod/vendor/github.com/uber/tchannel-go.(*handlerMap).Handle
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/vendor/github.com/uber/tchannel-go/handlers.go:118
github.com/uber/jaeger/examples/hotrod/vendor/github.com/uber/tchannel-go.channelHandler.Handle
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/vendor/github.com/uber/tchannel-go/handlers.go:126
2017-08-04T16:48:25.178+0200    ERROR   log/spanlogger.go:45    Retrying GetDriver after error  {"service": "driver", "retry_no": 1, "error": "redis timeout"}
github.com/uber/jaeger/examples/hotrod/vendor/go.uber.org/zap.Stack
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/vendor/go.uber.org/zap/field.go:209
github.com/uber/jaeger/examples/hotrod/vendor/go.uber.org/zap.(*Logger).check
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/vendor/go.uber.org/zap/logger.go:273
github.com/uber/jaeger/examples/hotrod/vendor/go.uber.org/zap.(*Logger).Error
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/vendor/go.uber.org/zap/logger.go:176
github.com/uber/jaeger/examples/hotrod/pkg/log.spanLogger.Error
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/pkg/log/spanlogger.go:45
github.com/uber/jaeger/examples/hotrod/pkg/log.(*spanLogger).Error
    <autogenerated>:1
github.com/uber/jaeger/examples/hotrod/services/driver.(*Server).FindNearest
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/services/driver/server.go:95
github.com/uber/jaeger/examples/hotrod/services/driver/thrift-gen/driver.(*tchanDriverServer).handleFindNearest
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/services/driver/thrift-gen/driver/tchan-driver.go:92
github.com/uber/jaeger/examples/hotrod/services/driver/thrift-gen/driver.(*tchanDriverServer).Handle
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/services/driver/thrift-gen/driver/tchan-driver.go:76
github.com/uber/jaeger/examples/hotrod/vendor/github.com/uber/tchannel-go/thrift.(*Server).handle
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/vendor/github.com/uber/tchannel-go/thrift/server.go:133
github.com/uber/jaeger/examples/hotrod/vendor/github.com/uber/tchannel-go/thrift.(*Server).Handle
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/vendor/github.com/uber/tchannel-go/thrift/server.go:203
github.com/uber/jaeger/examples/hotrod/vendor/github.com/uber/tchannel-go.(*handlerMap).Handle
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/vendor/github.com/uber/tchannel-go/handlers.go:118
github.com/uber/jaeger/examples/hotrod/vendor/github.com/uber/tchannel-go.channelHandler.Handle
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/vendor/github.com/uber/tchannel-go/handlers.go:126
github.com/uber/jaeger/examples/hotrod/vendor/github.com/uber/tchannel-go.(*Connection).dispatchInbound
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/vendor/github.com/uber/tchannel-go/inbound.go:195
2017-08-04T16:48:25.212+0200    INFO    log/spanlogger.go:40    Search successful   {"service": "driver", "num_drivers": 10}
2017-08-04T16:48:25.213+0200    INFO    log/spanlogger.go:40    Found drivers   {"service": "frontend", "drivers": [{"DriverID":"T703625C","Location":"422,896"},{"DriverID":"T700478C","Location":"674,181"},{"DriverID":"T715719C","Location":"755,794"},{"DriverID":"T739411C","Location":"38,908"},{"DriverID":"T751568C","Location":"919,323"},{"DriverID":"T779787C","Location":"47,33"},{"DriverID":"T778928C","Location":"891,104"},{"DriverID":"T757184C","Location":"57,451"},{"DriverID":"T701033C","Location":"569,89"},{"DriverID":"T750236C","Location":"430,602"}]}
2017-08-04T16:48:25.213+0200    INFO    log/spanlogger.go:40    Finding route   {"service": "frontend", "component": "route_client", "pickup": "755,794", "dropoff": "577,322"}
2017-08-04T16:48:25.213+0200    INFO    log/spanlogger.go:40    Finding route   {"service": "frontend", "component": "route_client", "pickup": "674,181", "dropoff": "577,322"}
2017-08-04T16:48:25.213+0200    INFO    log/spanlogger.go:40    Finding route   {"service": "frontend", "component": "route_client", "pickup": "422,896", "dropoff": "577,322"}
2017-08-04T16:48:25.214+0200    INFO    log/spanlogger.go:40    HTTP request received   {"service": "route", "method": "GET", "url": "/route?dropoff=577%2C322&pickup=755%2C794"}
2017-08-04T16:48:25.214+0200    INFO    log/spanlogger.go:40    HTTP request received   {"service": "route", "method": "GET", "url": "/route?dropoff=577%2C322&pickup=674%2C181"}
2017-08-04T16:48:25.215+0200    INFO    log/spanlogger.go:40    HTTP request received   {"service": "route", "method": "GET", "url": "/route?dropoff=577%2C322&pickup=422%2C896"}
2017-08-04T16:48:25.247+0200    INFO    log/spanlogger.go:40    Finding route   {"service": "frontend", "component": "route_client", "pickup": "38,908", "dropoff": "577,322"}
2017-08-04T16:48:25.248+0200    INFO    log/spanlogger.go:40    HTTP request received   {"service": "route", "method": "GET", "url": "/route?dropoff=577%2C322&pickup=38%2C908"}
2017-08-04T16:48:25.255+0200    INFO    log/spanlogger.go:40    Finding route   {"service": "frontend", "component": "route_client", "pickup": "919,323", "dropoff": "577,322"}
2017-08-04T16:48:25.255+0200    INFO    log/spanlogger.go:40    HTTP request received   {"service": "route", "method": "GET", "url": "/route?dropoff=577%2C322&pickup=919%2C323"}
2017-08-04T16:48:25.265+0200    INFO    log/spanlogger.go:40    Finding route   {"service": "frontend", "component": "route_client", "pickup": "47,33", "dropoff": "577,322"}
2017-08-04T16:48:25.265+0200    INFO    log/spanlogger.go:40    HTTP request received   {"service": "route", "method": "GET", "url": "/route?dropoff=577%2C322&pickup=47%2C33"}
2017-08-04T16:48:25.298+0200    INFO    log/spanlogger.go:40    Finding route   {"service": "frontend", "component": "route_client", "pickup": "891,104", "dropoff": "577,322"}
2017-08-04T16:48:25.299+0200    INFO    log/spanlogger.go:40    HTTP request received   {"service": "route", "method": "GET", "url": "/route?dropoff=577%2C322&pickup=891%2C104"}
2017-08-04T16:48:25.306+0200    INFO    log/spanlogger.go:40    Finding route   {"service": "frontend", "component": "route_client", "pickup": "57,451", "dropoff": "577,322"}
2017-08-04T16:48:25.306+0200    INFO    log/spanlogger.go:40    HTTP request received   {"service": "route", "method": "GET", "url": "/route?dropoff=577%2C322&pickup=57%2C451"}
2017-08-04T16:48:25.310+0200    INFO    log/spanlogger.go:40    Finding route   {"service": "frontend", "component": "route_client", "pickup": "569,89", "dropoff": "577,322"}
2017-08-04T16:48:25.310+0200    INFO    log/spanlogger.go:40    HTTP request received   {"service": "route", "method": "GET", "url": "/route?dropoff=577%2C322&pickup=569%2C89"}
2017-08-04T16:48:25.336+0200    INFO    log/spanlogger.go:40    Finding route   {"service": "frontend", "component": "route_client", "pickup": "430,602", "dropoff": "577,322"}
2017-08-04T16:48:25.337+0200    INFO    log/spanlogger.go:40    HTTP request received   {"service": "route", "method": "GET", "url": "/route?dropoff=577%2C322&pickup=430%2C602"}
2017-08-04T16:48:25.347+0200    INFO    log/spanlogger.go:40    Found customer  {"service": "frontend", "customer": {"ID":"392","Name":"Trom Chocolatier","Location":"577,322"}}
2017-08-04T16:48:25.347+0200    INFO    log/spanlogger.go:40    Finding nearest drivers {"service": "frontend", "component": "driver_client", "location": "577,322"}
2017-08-04T16:48:25.347+0200    INFO    log/spanlogger.go:40    Searching for nearby drivers    {"service": "driver", "location": "577,322"}
2017-08-04T16:48:25.395+0200    ERROR   log/spanlogger.go:45    redis timeout   {"service": "driver", "driver_id": "T710314C", "error": "redis timeout"}
github.com/uber/jaeger/examples/hotrod/vendor/go.uber.org/zap.Stack
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/vendor/go.uber.org/zap/field.go:209
github.com/uber/jaeger/examples/hotrod/vendor/go.uber.org/zap.(*Logger).check
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/vendor/go.uber.org/zap/logger.go:273
github.com/uber/jaeger/examples/hotrod/vendor/go.uber.org/zap.(*Logger).Error
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/vendor/go.uber.org/zap/logger.go:176
github.com/uber/jaeger/examples/hotrod/pkg/log.spanLogger.Error
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/pkg/log/spanlogger.go:45
github.com/uber/jaeger/examples/hotrod/pkg/log.(*spanLogger).Error
    <autogenerated>:1
github.com/uber/jaeger/examples/hotrod/services/driver.(*Redis).GetDriver
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/services/driver/redis.go:89
github.com/uber/jaeger/examples/hotrod/services/driver.(*Server).FindNearest
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/services/driver/server.go:91
github.com/uber/jaeger/examples/hotrod/services/driver/thrift-gen/driver.(*tchanDriverServer).handleFindNearest
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/services/driver/thrift-gen/driver/tchan-driver.go:92
github.com/uber/jaeger/examples/hotrod/services/driver/thrift-gen/driver.(*tchanDriverServer).Handle
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/services/driver/thrift-gen/driver/tchan-driver.go:76
github.com/uber/jaeger/examples/hotrod/vendor/github.com/uber/tchannel-go/thrift.(*Server).handle
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/vendor/github.com/uber/tchannel-go/thrift/server.go:133
github.com/uber/jaeger/examples/hotrod/vendor/github.com/uber/tchannel-go/thrift.(*Server).Handle
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/vendor/github.com/uber/tchannel-go/thrift/server.go:203
github.com/uber/jaeger/examples/hotrod/vendor/github.com/uber/tchannel-go.(*handlerMap).Handle
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/vendor/github.com/uber/tchannel-go/handlers.go:118
github.com/uber/jaeger/examples/hotrod/vendor/github.com/uber/tchannel-go.channelHandler.Handle
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/vendor/github.com/uber/tchannel-go/handlers.go:126
github.com/uber/jaeger/examples/hotrod/vendor/github.com/uber/tchannel-go.(*Connection).dispatchInbound
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/vendor/github.com/uber/tchannel-go/inbound.go:195
2017-08-04T16:48:25.395+0200    ERROR   log/spanlogger.go:45    Retrying GetDriver after error  {"service": "driver", "retry_no": 1, "error": "redis timeout"}
github.com/uber/jaeger/examples/hotrod/vendor/go.uber.org/zap.Stack
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/vendor/go.uber.org/zap/field.go:209
github.com/uber/jaeger/examples/hotrod/vendor/go.uber.org/zap.(*Logger).check
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/vendor/go.uber.org/zap/logger.go:273
github.com/uber/jaeger/examples/hotrod/vendor/go.uber.org/zap.(*Logger).Error
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/vendor/go.uber.org/zap/logger.go:176
github.com/uber/jaeger/examples/hotrod/pkg/log.spanLogger.Error
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/pkg/log/spanlogger.go:45
github.com/uber/jaeger/examples/hotrod/pkg/log.(*spanLogger).Error
    <autogenerated>:1
github.com/uber/jaeger/examples/hotrod/services/driver.(*Server).FindNearest
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/services/driver/server.go:95
github.com/uber/jaeger/examples/hotrod/services/driver/thrift-gen/driver.(*tchanDriverServer).handleFindNearest
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/services/driver/thrift-gen/driver/tchan-driver.go:92
github.com/uber/jaeger/examples/hotrod/services/driver/thrift-gen/driver.(*tchanDriverServer).Handle
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/services/driver/thrift-gen/driver/tchan-driver.go:76
github.com/uber/jaeger/examples/hotrod/vendor/github.com/uber/tchannel-go/thrift.(*Server).handle
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/vendor/github.com/uber/tchannel-go/thrift/server.go:133
github.com/uber/jaeger/examples/hotrod/vendor/github.com/uber/tchannel-go/thrift.(*Server).Handle
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/vendor/github.com/uber/tchannel-go/thrift/server.go:203
github.com/uber/jaeger/examples/hotrod/vendor/github.com/uber/tchannel-go.(*handlerMap).Handle
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/vendor/github.com/uber/tchannel-go/handlers.go:118
github.com/uber/jaeger/examples/hotrod/vendor/github.com/uber/tchannel-go.channelHandler.Handle
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/vendor/github.com/uber/tchannel-go/handlers.go:126
github.com/uber/jaeger/examples/hotrod/vendor/github.com/uber/tchannel-go.(*Connection).dispatchInbound
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/vendor/github.com/uber/tchannel-go/inbound.go:195
2017-08-04T16:48:25.397+0200    INFO    log/spanlogger.go:40    Found routes    {"service": "frontend", "routes": [{},{},{},{},{},{},{},{},{},{}]}
2017-08-04T16:48:25.397+0200    INFO    log/spanlogger.go:40    Dispatch successful {"service": "frontend", "driver": "T703625C", "eta": "2m0s"}
2017-08-04T16:48:25.463+0200    ERROR   log/spanlogger.go:45    redis timeout   {"service": "driver", "driver_id": "T778491C", "error": "redis timeout"}
github.com/uber/jaeger/examples/hotrod/vendor/go.uber.org/zap.Stack
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/vendor/go.uber.org/zap/field.go:209
github.com/uber/jaeger/examples/hotrod/vendor/go.uber.org/zap.(*Logger).check
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/vendor/go.uber.org/zap/logger.go:273
github.com/uber/jaeger/examples/hotrod/vendor/go.uber.org/zap.(*Logger).Error
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/vendor/go.uber.org/zap/logger.go:176
github.com/uber/jaeger/examples/hotrod/pkg/log.spanLogger.Error
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/pkg/log/spanlogger.go:45
github.com/uber/jaeger/examples/hotrod/pkg/log.(*spanLogger).Error
    <autogenerated>:1
github.com/uber/jaeger/examples/hotrod/services/driver.(*Redis).GetDriver
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/services/driver/redis.go:89
github.com/uber/jaeger/examples/hotrod/services/driver.(*Server).FindNearest
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/services/driver/server.go:91
github.com/uber/jaeger/examples/hotrod/services/driver/thrift-gen/driver.(*tchanDriverServer).handleFindNearest
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/services/driver/thrift-gen/driver/tchan-driver.go:92
github.com/uber/jaeger/examples/hotrod/services/driver/thrift-gen/driver.(*tchanDriverServer).Handle
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/services/driver/thrift-gen/driver/tchan-driver.go:76
github.com/uber/jaeger/examples/hotrod/vendor/github.com/uber/tchannel-go/thrift.(*Server).handle
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/vendor/github.com/uber/tchannel-go/thrift/server.go:133
github.com/uber/jaeger/examples/hotrod/vendor/github.com/uber/tchannel-go/thrift.(*Server).Handle
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/vendor/github.com/uber/tchannel-go/thrift/server.go:203
github.com/uber/jaeger/examples/hotrod/vendor/github.com/uber/tchannel-go.(*handlerMap).Handle
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/vendor/github.com/uber/tchannel-go/handlers.go:118
github.com/uber/jaeger/examples/hotrod/vendor/github.com/uber/tchannel-go.channelHandler.Handle
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/vendor/github.com/uber/tchannel-go/handlers.go:126
2017-08-04T16:48:25.463+0200    ERROR   log/spanlogger.go:45    Retrying GetDriver after error  {"service": "driver", "retry_no": 1, "error": "redis timeout"}
github.com/uber/jaeger/examples/hotrod/vendor/go.uber.org/zap.Stack
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/vendor/go.uber.org/zap/field.go:209
github.com/uber/jaeger/examples/hotrod/vendor/go.uber.org/zap.(*Logger).check
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/vendor/go.uber.org/zap/logger.go:273
github.com/uber/jaeger/examples/hotrod/vendor/go.uber.org/zap.(*Logger).Error
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/vendor/go.uber.org/zap/logger.go:176
github.com/uber/jaeger/examples/hotrod/pkg/log.spanLogger.Error
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/pkg/log/spanlogger.go:45
github.com/uber/jaeger/examples/hotrod/pkg/log.(*spanLogger).Error
    <autogenerated>:1
github.com/uber/jaeger/examples/hotrod/services/driver.(*Server).FindNearest
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/services/driver/server.go:95
github.com/uber/jaeger/examples/hotrod/services/driver/thrift-gen/driver.(*tchanDriverServer).handleFindNearest
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/services/driver/thrift-gen/driver/tchan-driver.go:92
github.com/uber/jaeger/examples/hotrod/services/driver/thrift-gen/driver.(*tchanDriverServer).Handle
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/services/driver/thrift-gen/driver/tchan-driver.go:76
github.com/uber/jaeger/examples/hotrod/vendor/github.com/uber/tchannel-go/thrift.(*Server).handle
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/vendor/github.com/uber/tchannel-go/thrift/server.go:133
github.com/uber/jaeger/examples/hotrod/vendor/github.com/uber/tchannel-go/thrift.(*Server).Handle
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/vendor/github.com/uber/tchannel-go/thrift/server.go:203
github.com/uber/jaeger/examples/hotrod/vendor/github.com/uber/tchannel-go.(*handlerMap).Handle
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/vendor/github.com/uber/tchannel-go/handlers.go:118
github.com/uber/jaeger/examples/hotrod/vendor/github.com/uber/tchannel-go.channelHandler.Handle
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/vendor/github.com/uber/tchannel-go/handlers.go:126
github.com/uber/jaeger/examples/hotrod/vendor/github.com/uber/tchannel-go.(*Connection).dispatchInbound
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/vendor/github.com/uber/tchannel-go/inbound.go:195
2017-08-04T16:48:25.534+0200    ERROR   log/spanlogger.go:45    redis timeout   {"service": "driver", "driver_id": "T784813C", "error": "redis timeout"}
github.com/uber/jaeger/examples/hotrod/vendor/go.uber.org/zap.Stack
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/vendor/go.uber.org/zap/field.go:209
github.com/uber/jaeger/examples/hotrod/vendor/go.uber.org/zap.(*Logger).check
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/vendor/go.uber.org/zap/logger.go:273
github.com/uber/jaeger/examples/hotrod/vendor/go.uber.org/zap.(*Logger).Error
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/vendor/go.uber.org/zap/logger.go:176
github.com/uber/jaeger/examples/hotrod/pkg/log.spanLogger.Error
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/pkg/log/spanlogger.go:45
github.com/uber/jaeger/examples/hotrod/pkg/log.(*spanLogger).Error
    <autogenerated>:1
github.com/uber/jaeger/examples/hotrod/services/driver.(*Redis).GetDriver
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/services/driver/redis.go:89
github.com/uber/jaeger/examples/hotrod/services/driver.(*Server).FindNearest
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/services/driver/server.go:91
github.com/uber/jaeger/examples/hotrod/services/driver/thrift-gen/driver.(*tchanDriverServer).handleFindNearest
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/services/driver/thrift-gen/driver/tchan-driver.go:92
github.com/uber/jaeger/examples/hotrod/services/driver/thrift-gen/driver.(*tchanDriverServer).Handle
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/services/driver/thrift-gen/driver/tchan-driver.go:76
github.com/uber/jaeger/examples/hotrod/vendor/github.com/uber/tchannel-go/thrift.(*Server).handle
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/vendor/github.com/uber/tchannel-go/thrift/server.go:133
github.com/uber/jaeger/examples/hotrod/vendor/github.com/uber/tchannel-go/thrift.(*Server).Handle
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/vendor/github.com/uber/tchannel-go/thrift/server.go:203
github.com/uber/jaeger/examples/hotrod/vendor/github.com/uber/tchannel-go.(*handlerMap).Handle
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/vendor/github.com/uber/tchannel-go/handlers.go:118
github.com/uber/jaeger/examples/hotrod/vendor/github.com/uber/tchannel-go.channelHandler.Handle
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/vendor/github.com/uber/tchannel-go/handlers.go:126
github.com/uber/jaeger/examples/hotrod/vendor/github.com/uber/tchannel-go.(*Connection).dispatchInbound
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/vendor/github.com/uber/tchannel-go/inbound.go:195
2017-08-04T16:48:25.534+0200    ERROR   log/spanlogger.go:45    Retrying GetDriver after error  {"service": "driver", "retry_no": 1, "error": "redis timeout"}
github.com/uber/jaeger/examples/hotrod/vendor/go.uber.org/zap.Stack
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/vendor/go.uber.org/zap/field.go:209
github.com/uber/jaeger/examples/hotrod/vendor/go.uber.org/zap.(*Logger).check
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/vendor/go.uber.org/zap/logger.go:273
github.com/uber/jaeger/examples/hotrod/vendor/go.uber.org/zap.(*Logger).Error
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/vendor/go.uber.org/zap/logger.go:176
github.com/uber/jaeger/examples/hotrod/pkg/log.spanLogger.Error
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/pkg/log/spanlogger.go:45
github.com/uber/jaeger/examples/hotrod/pkg/log.(*spanLogger).Error
    <autogenerated>:1
github.com/uber/jaeger/examples/hotrod/services/driver.(*Server).FindNearest
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/services/driver/server.go:95
github.com/uber/jaeger/examples/hotrod/services/driver/thrift-gen/driver.(*tchanDriverServer).handleFindNearest
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/services/driver/thrift-gen/driver/tchan-driver.go:92
github.com/uber/jaeger/examples/hotrod/services/driver/thrift-gen/driver.(*tchanDriverServer).Handle
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/services/driver/thrift-gen/driver/tchan-driver.go:76
github.com/uber/jaeger/examples/hotrod/vendor/github.com/uber/tchannel-go/thrift.(*Server).handle
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/vendor/github.com/uber/tchannel-go/thrift/server.go:133
github.com/uber/jaeger/examples/hotrod/vendor/github.com/uber/tchannel-go/thrift.(*Server).Handle
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/vendor/github.com/uber/tchannel-go/thrift/server.go:203
github.com/uber/jaeger/examples/hotrod/vendor/github.com/uber/tchannel-go.(*handlerMap).Handle
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/vendor/github.com/uber/tchannel-go/handlers.go:118
github.com/uber/jaeger/examples/hotrod/vendor/github.com/uber/tchannel-go.channelHandler.Handle
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/vendor/github.com/uber/tchannel-go/handlers.go:126
github.com/uber/jaeger/examples/hotrod/vendor/github.com/uber/tchannel-go.(*Connection).dispatchInbound
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/vendor/github.com/uber/tchannel-go/inbound.go:195
2017-08-04T16:48:25.555+0200    INFO    log/spanlogger.go:40    Search successful   {"service": "driver", "num_drivers": 10}
2017-08-04T16:48:25.555+0200    INFO    log/spanlogger.go:40    Found drivers   {"service": "frontend", "drivers": [{"DriverID":"T710314C","Location":"949,213"},{"DriverID":"T795190C","Location":"636,353"},{"DriverID":"T707941C","Location":"157,254"},{"DriverID":"T764823C","Location":"108,443"},{"DriverID":"T778491C","Location":"253,658"},{"DriverID":"T783036C","Location":"540,650"},{"DriverID":"T715947C","Location":"736,692"},{"DriverID":"T724896C","Location":"598,500"},{"DriverID":"T784813C","Location":"129,205"},{"DriverID":"T703028C","Location":"482,210"}]}
2017-08-04T16:48:25.555+0200    INFO    log/spanlogger.go:40    Finding route   {"service": "frontend", "component": "route_client", "pickup": "157,254", "dropoff": "577,322"}
2017-08-04T16:48:25.555+0200    INFO    log/spanlogger.go:40    Finding route   {"service": "frontend", "component": "route_client", "pickup": "949,213", "dropoff": "577,322"}
2017-08-04T16:48:25.555+0200    INFO    log/spanlogger.go:40    Finding route   {"service": "frontend", "component": "route_client", "pickup": "636,353", "dropoff": "577,322"}
2017-08-04T16:48:25.555+0200    INFO    log/spanlogger.go:40    HTTP request received   {"service": "route", "method": "GET", "url": "/route?dropoff=577%2C322&pickup=949%2C213"}
2017-08-04T16:48:25.555+0200    INFO    log/spanlogger.go:40    HTTP request received   {"service": "route", "method": "GET", "url": "/route?dropoff=577%2C322&pickup=157%2C254"}
2017-08-04T16:48:25.556+0200    INFO    log/spanlogger.go:40    HTTP request received   {"service": "route", "method": "GET", "url": "/route?dropoff=577%2C322&pickup=636%2C353"}
2017-08-04T16:48:25.578+0200    INFO    log/spanlogger.go:40    Finding route   {"service": "frontend", "component": "route_client", "pickup": "108,443", "dropoff": "577,322"}
2017-08-04T16:48:25.578+0200    INFO    log/spanlogger.go:40    HTTP request received   {"service": "route", "method": "GET", "url": "/route?dropoff=577%2C322&pickup=108%2C443"}
2017-08-04T16:48:25.604+0200    INFO    log/spanlogger.go:40    Finding route   {"service": "frontend", "component": "route_client", "pickup": "253,658", "dropoff": "577,322"}
2017-08-04T16:48:25.604+0200    INFO    log/spanlogger.go:40    HTTP request received   {"service": "route", "method": "GET", "url": "/route?dropoff=577%2C322&pickup=253%2C658"}
2017-08-04T16:48:25.607+0200    INFO    log/spanlogger.go:40    Finding route   {"service": "frontend", "component": "route_client", "pickup": "540,650", "dropoff": "577,322"}
2017-08-04T16:48:25.607+0200    INFO    log/spanlogger.go:40    HTTP request received   {"service": "route", "method": "GET", "url": "/route?dropoff=577%2C322&pickup=540%2C650"}
2017-08-04T16:48:25.641+0200    INFO    log/spanlogger.go:40    Finding route   {"service": "frontend", "component": "route_client", "pickup": "736,692", "dropoff": "577,322"}
2017-08-04T16:48:25.641+0200    INFO    log/spanlogger.go:40    HTTP request received   {"service": "route", "method": "GET", "url": "/route?dropoff=577%2C322&pickup=736%2C692"}
2017-08-04T16:48:25.642+0200    INFO    log/spanlogger.go:40    Found customer  {"service": "frontend", "customer": {"ID":"392","Name":"Trom Chocolatier","Location":"577,322"}}
2017-08-04T16:48:25.642+0200    INFO    log/spanlogger.go:40    Finding nearest drivers {"service": "frontend", "component": "driver_client", "location": "577,322"}
2017-08-04T16:48:25.643+0200    INFO    log/spanlogger.go:40    Searching for nearby drivers    {"service": "driver", "location": "577,322"}
2017-08-04T16:48:25.646+0200    INFO    log/spanlogger.go:40    Finding route   {"service": "frontend", "component": "route_client", "pickup": "598,500", "dropoff": "577,322"}
2017-08-04T16:48:25.646+0200    INFO    log/spanlogger.go:40    HTTP request received   {"service": "route", "method": "GET", "url": "/route?dropoff=577%2C322&pickup=598%2C500"}
2017-08-04T16:48:25.658+0200    INFO    log/spanlogger.go:40    Finding route   {"service": "frontend", "component": "route_client", "pickup": "129,205", "dropoff": "577,322"}
2017-08-04T16:48:25.659+0200    INFO    log/spanlogger.go:40    HTTP request received   {"service": "route", "method": "GET", "url": "/route?dropoff=577%2C322&pickup=129%2C205"}
2017-08-04T16:48:25.679+0200    INFO    log/spanlogger.go:40    Finding route   {"service": "frontend", "component": "route_client", "pickup": "482,210", "dropoff": "577,322"}
2017-08-04T16:48:25.680+0200    INFO    log/spanlogger.go:40    HTTP request received   {"service": "route", "method": "GET", "url": "/route?dropoff=577%2C322&pickup=482%2C210"}
2017-08-04T16:48:25.720+0200    ERROR   log/spanlogger.go:45    redis timeout   {"service": "driver", "driver_id": "T708765C", "error": "redis timeout"}
github.com/uber/jaeger/examples/hotrod/vendor/go.uber.org/zap.Stack
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/vendor/go.uber.org/zap/field.go:209
github.com/uber/jaeger/examples/hotrod/vendor/go.uber.org/zap.(*Logger).check
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/vendor/go.uber.org/zap/logger.go:273
github.com/uber/jaeger/examples/hotrod/vendor/go.uber.org/zap.(*Logger).Error
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/vendor/go.uber.org/zap/logger.go:176
github.com/uber/jaeger/examples/hotrod/pkg/log.spanLogger.Error
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/pkg/log/spanlogger.go:45
github.com/uber/jaeger/examples/hotrod/pkg/log.(*spanLogger).Error
    <autogenerated>:1
github.com/uber/jaeger/examples/hotrod/services/driver.(*Redis).GetDriver
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/services/driver/redis.go:89
github.com/uber/jaeger/examples/hotrod/services/driver.(*Server).FindNearest
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/services/driver/server.go:91
github.com/uber/jaeger/examples/hotrod/services/driver/thrift-gen/driver.(*tchanDriverServer).handleFindNearest
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/services/driver/thrift-gen/driver/tchan-driver.go:92
github.com/uber/jaeger/examples/hotrod/services/driver/thrift-gen/driver.(*tchanDriverServer).Handle
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/services/driver/thrift-gen/driver/tchan-driver.go:76
github.com/uber/jaeger/examples/hotrod/vendor/github.com/uber/tchannel-go/thrift.(*Server).handle
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/vendor/github.com/uber/tchannel-go/thrift/server.go:133
github.com/uber/jaeger/examples/hotrod/vendor/github.com/uber/tchannel-go/thrift.(*Server).Handle
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/vendor/github.com/uber/tchannel-go/thrift/server.go:203
github.com/uber/jaeger/examples/hotrod/vendor/github.com/uber/tchannel-go.(*handlerMap).Handle
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/vendor/github.com/uber/tchannel-go/handlers.go:118
github.com/uber/jaeger/examples/hotrod/vendor/github.com/uber/tchannel-go.channelHandler.Handle
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/vendor/github.com/uber/tchannel-go/handlers.go:126
github.com/uber/jaeger/examples/hotrod/vendor/github.com/uber/tchannel-go.(*Connection).dispatchInbound
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/vendor/github.com/uber/tchannel-go/inbound.go:195
2017-08-04T16:48:25.720+0200    ERROR   log/spanlogger.go:45    Retrying GetDriver after error  {"service": "driver", "retry_no": 1, "error": "redis timeout"}
github.com/uber/jaeger/examples/hotrod/vendor/go.uber.org/zap.Stack
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/vendor/go.uber.org/zap/field.go:209
github.com/uber/jaeger/examples/hotrod/vendor/go.uber.org/zap.(*Logger).check
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/vendor/go.uber.org/zap/logger.go:273
github.com/uber/jaeger/examples/hotrod/vendor/go.uber.org/zap.(*Logger).Error
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/vendor/go.uber.org/zap/logger.go:176
github.com/uber/jaeger/examples/hotrod/pkg/log.spanLogger.Error
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/pkg/log/spanlogger.go:45
github.com/uber/jaeger/examples/hotrod/pkg/log.(*spanLogger).Error
    <autogenerated>:1
github.com/uber/jaeger/examples/hotrod/services/driver.(*Server).FindNearest
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/services/driver/server.go:95
github.com/uber/jaeger/examples/hotrod/services/driver/thrift-gen/driver.(*tchanDriverServer).handleFindNearest
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/services/driver/thrift-gen/driver/tchan-driver.go:92
github.com/uber/jaeger/examples/hotrod/services/driver/thrift-gen/driver.(*tchanDriverServer).Handle
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/services/driver/thrift-gen/driver/tchan-driver.go:76
github.com/uber/jaeger/examples/hotrod/vendor/github.com/uber/tchannel-go/thrift.(*Server).handle
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/vendor/github.com/uber/tchannel-go/thrift/server.go:133
github.com/uber/jaeger/examples/hotrod/vendor/github.com/uber/tchannel-go/thrift.(*Server).Handle
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/vendor/github.com/uber/tchannel-go/thrift/server.go:203
github.com/uber/jaeger/examples/hotrod/vendor/github.com/uber/tchannel-go.(*handlerMap).Handle
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/vendor/github.com/uber/tchannel-go/handlers.go:118
github.com/uber/jaeger/examples/hotrod/vendor/github.com/uber/tchannel-go.channelHandler.Handle
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/vendor/github.com/uber/tchannel-go/handlers.go:126
github.com/uber/jaeger/examples/hotrod/vendor/github.com/uber/tchannel-go.(*Connection).dispatchInbound
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/vendor/github.com/uber/tchannel-go/inbound.go:195
2017-08-04T16:48:25.751+0200    INFO    log/spanlogger.go:40    Found routes    {"service": "frontend", "routes": [{},{},{},{},{},{},{},{},{},{}]}
2017-08-04T16:48:25.751+0200    INFO    log/spanlogger.go:40    Dispatch successful {"service": "frontend", "driver": "T710314C", "eta": "2m0s"}
2017-08-04T16:48:25.791+0200    ERROR   log/spanlogger.go:45    redis timeout   {"service": "driver", "driver_id": "T790852C", "error": "redis timeout"}
github.com/uber/jaeger/examples/hotrod/vendor/go.uber.org/zap.Stack
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/vendor/go.uber.org/zap/field.go:209
github.com/uber/jaeger/examples/hotrod/vendor/go.uber.org/zap.(*Logger).check
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/vendor/go.uber.org/zap/logger.go:273
github.com/uber/jaeger/examples/hotrod/vendor/go.uber.org/zap.(*Logger).Error
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/vendor/go.uber.org/zap/logger.go:176
github.com/uber/jaeger/examples/hotrod/pkg/log.spanLogger.Error
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/pkg/log/spanlogger.go:45
github.com/uber/jaeger/examples/hotrod/pkg/log.(*spanLogger).Error
    <autogenerated>:1
github.com/uber/jaeger/examples/hotrod/services/driver.(*Redis).GetDriver
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/services/driver/redis.go:89
github.com/uber/jaeger/examples/hotrod/services/driver.(*Server).FindNearest
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/services/driver/server.go:91
github.com/uber/jaeger/examples/hotrod/services/driver/thrift-gen/driver.(*tchanDriverServer).handleFindNearest
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/services/driver/thrift-gen/driver/tchan-driver.go:92
github.com/uber/jaeger/examples/hotrod/services/driver/thrift-gen/driver.(*tchanDriverServer).Handle
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/services/driver/thrift-gen/driver/tchan-driver.go:76
github.com/uber/jaeger/examples/hotrod/vendor/github.com/uber/tchannel-go/thrift.(*Server).handle
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/vendor/github.com/uber/tchannel-go/thrift/server.go:133
github.com/uber/jaeger/examples/hotrod/vendor/github.com/uber/tchannel-go/thrift.(*Server).Handle
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/vendor/github.com/uber/tchannel-go/thrift/server.go:203
github.com/uber/jaeger/examples/hotrod/vendor/github.com/uber/tchannel-go.(*handlerMap).Handle
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/vendor/github.com/uber/tchannel-go/handlers.go:118
github.com/uber/jaeger/examples/hotrod/vendor/github.com/uber/tchannel-go.channelHandler.Handle
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/vendor/github.com/uber/tchannel-go/handlers.go:126
2017-08-04T16:48:25.791+0200    ERROR   log/spanlogger.go:45    Retrying GetDriver after error  {"service": "driver", "retry_no": 1, "error": "redis timeout"}
github.com/uber/jaeger/examples/hotrod/vendor/go.uber.org/zap.Stack
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/vendor/go.uber.org/zap/field.go:209
github.com/uber/jaeger/examples/hotrod/vendor/go.uber.org/zap.(*Logger).check
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/vendor/go.uber.org/zap/logger.go:273
github.com/uber/jaeger/examples/hotrod/vendor/go.uber.org/zap.(*Logger).Error
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/vendor/go.uber.org/zap/logger.go:176
github.com/uber/jaeger/examples/hotrod/pkg/log.spanLogger.Error
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/pkg/log/spanlogger.go:45
github.com/uber/jaeger/examples/hotrod/pkg/log.(*spanLogger).Error
    <autogenerated>:1
github.com/uber/jaeger/examples/hotrod/services/driver.(*Server).FindNearest
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/services/driver/server.go:95
github.com/uber/jaeger/examples/hotrod/services/driver/thrift-gen/driver.(*tchanDriverServer).handleFindNearest
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/services/driver/thrift-gen/driver/tchan-driver.go:92
github.com/uber/jaeger/examples/hotrod/services/driver/thrift-gen/driver.(*tchanDriverServer).Handle
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/services/driver/thrift-gen/driver/tchan-driver.go:76
github.com/uber/jaeger/examples/hotrod/vendor/github.com/uber/tchannel-go/thrift.(*Server).handle
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/vendor/github.com/uber/tchannel-go/thrift/server.go:133
github.com/uber/jaeger/examples/hotrod/vendor/github.com/uber/tchannel-go/thrift.(*Server).Handle
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/vendor/github.com/uber/tchannel-go/thrift/server.go:203
github.com/uber/jaeger/examples/hotrod/vendor/github.com/uber/tchannel-go.(*handlerMap).Handle
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/vendor/github.com/uber/tchannel-go/handlers.go:118
github.com/uber/jaeger/examples/hotrod/vendor/github.com/uber/tchannel-go.channelHandler.Handle
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/vendor/github.com/uber/tchannel-go/handlers.go:126
github.com/uber/jaeger/examples/hotrod/vendor/github.com/uber/tchannel-go.(*Connection).dispatchInbound
    /home/dieter/go/src/github.com/uber/jaeger/examples/hotrod/vendor/github.com/uber/tchannel-go/inbound.go:195
jpkrohling commented 7 years ago

Looks like the "expected" errors to me. The example injects some errors, to simulate backend failures. See this: https://medium.com/opentracing/take-opentracing-for-a-hotrod-ride-f6e3141f7941

yurishkuro commented 7 years ago

Yes, the errors are expected, and the whole logging output has a point of showing how logging is not the right tool for capturing transaction-scoped events.

yurishkuro commented 7 years ago

I'm going to close this as it's not an issue. Feel free to comment / reopen if needed.

Dieterbe commented 7 years ago

aha i didn't gather that from the readme, this is great, thanks!

yurishkuro commented 7 years ago

@Dieterbe fyi https://blog.openshift.com/openshift-commons-briefing-82-distributed-tracing-with-jaeger-prometheus-on-kubernetes/