search

goadapp / goad

Goad is an AWS Lambda powered, highly distributed, load testing tool
https://goad.io
MIT License
1.88k stars 190 forks source link

segfault on the queue #155

Open gserra-olx opened 7 years ago

gserra-olx commented 7 years ago 
Region: us-west-2
   TotReqs   TotBytes    AvgTime    AvgReq/s  (post)unzip
     44271     1.8 GB     0.352s       31.50panic: runtime error: invalid memory address or nil pointer dereferenceTotErrors
    3.583s     0.103s          0  [signal SIGSEGV: segmentation violation code=0x1 addr=0x20 pc=0x147e1e1]

                         goroutine 379 [running]:
                                                 github.com/goadapp/goad/queue.SQSAdaptor.Receive(0xc4203d4068, 0xc4203beea0, 0x5a, 0xc4201fc188)
                                                                    /private/tmp/goad-20170731-18597-cbyzjk/src/github.com/goadapp/goad/queue/sqsadaptor.go:80 +0x4b1
Press cgithub.com/goadapp/goad/queue.aggregateFromSqs(0xc4207d17a0, 0xc4200fc630, 0xc4203beea0, 0x5a, 0xf4240, 0x64)
                                    /private/tmp/goad-20170731-18597-cbyzjk/src/github.com/goadapp/goad/queue/aggregation.go:124 +0x14d
                                                          created by github.com/goadapp/goad/queue.Aggregate
                            /private/tmp/goad-20170731-18597-cbyzjk/src/github.com/goadapp/goad/queue/aggregation.go:106 +0x12f
                                                  $
zeph commented 7 years ago

see if we can include AirBrake or NewRelic for tracking this

cwaltken-edrans commented 7 years ago

I agree, there is some really bad error handling in parts of the cli and even the lambda function. I'm not sure about the idea to add some third-party metrics tracking. Wouldn't that require a paid plan on those platforms and some kind of terms of service agreement sending user-data to those services?

How about for now we add a global recover function in goad that prints out something like this:

signal SIGSEGV: segmentation violation code=0x1 addr=0x20 pc=0x147e1e1]
goroutine 379 [running]:
github.com/goadapp/goad/queue.SQSAdaptor.Receive(0xc4203d4068, 0xc4203beea0, 0x5a, 0xc4201fc188)
/private/tmp/goad-20170731-18597-cbyzjk/src/github.com/goadapp/goad/queue/sqsadaptor.go:80 +0x4b1
github.com/goadapp/goad/queue.aggregateFromSqs(0xc4207d17a0, 0xc4200fc630, 0xc4203beea0, 0x5a, 0xf4240, 0x64)
/private/tmp/goad-20170731-18597-cbyzjk/src/github.com/goadapp/goad/queue/aggregation.go:124 +0x14d
created by github.com/goadapp/goad/queue.Aggregate
/private/tmp/goad-20170731-18597-cbyzjk/src/github.com/goadapp/goad/queue/aggregation.go:106 +0x12f

Sorry, looks like goad has crashed. We are currently working hard to make goad more reliable.
Please open an issue on our github repository. [Link to issues page on github]
gserra-olx commented 7 years ago

@cwaltken-edrans let's plan for a parameter in the INI file with a default key from/paid by OLX (commented out)

if someone wants to let us know about the problem, has to uncomment it, and the feature will activate and send us the data