search

s8sg / goflow

A Golang based high performance, scalable and distributed workflow framework
MIT License
1.06k stars 129 forks source link

Http requests get dropped when trying to send multiple requests #47

Closed dan-ash closed 11 months ago

dan-ash commented 1 year ago

How to reproduce

  1. Running the basic example from the README.
  2. Send a single request using curl -d hello http://localhost:8080/myflow - Everything works
  3. Send multiple requests using for i in {1..50}; do curl -d hello http://localhost:8080/myflow >> output.log & ; done

the following is happening:

2023/04/24 13:39:29 http: panic serving 127.0.0.1:51880: runtime error: invalid memory address or nil pointer dereference                                                                                                                     
goroutine 622 [running]:                                                                                                                                                                                                                      
net/http.(*conn).serve.func1()                                                                                         
        /nix/store/i6a9j315dl95az9a9vjpgya90y6j9z66-go-1.20.2/share/go/src/net/http/server.go:1854 +0xbf
panic({0x765840, 0xaa66e0})                                
        /nix/store/i6a9j315dl95az9a9vjpgya90y6j9z66-go-1.20.2/share/go/src/runtime/panic.go:890 +0x263
github.com/s8sg/goflow/eventhandler.(*TraceHandler).StopNodeSpan(...)                  
        /home/danash/go/pkg/mod/github.com/s8sg/goflow@v0.1.1/eventhandler/trace_handler.go:110
github.com/s8sg/goflow/eventhandler.(*GoFlowEventHandler).ReportNodeEnd(0x79c520?, {0xc00088c910?, 0x7f6d9e1e69a8?}, {0x7f6dc6d6e3c8?, 0xc000051400?})
        /home/danash/go/pkg/mod/github.com/s8sg/goflow@v0.1.1/eventhandler/goflow_event_handler.go:57 +0x3d
github.com/s8sg/goflow/core/sdk/executor.(*FlowExecutor).findNextNodeToExecute(0xc000177740)
        /home/danash/go/pkg/mod/github.com/s8sg/goflow@v0.1.1/core/sdk/executor/executor.go:627 +0x4f8
github.com/s8sg/goflow/core/sdk/executor.(*FlowExecutor).Execute(0xc000177740, 0xc0010bd720)
        /home/danash/go/pkg/mod/github.com/s8sg/goflow@v0.1.1/core/sdk/executor/executor.go:1265 +0x9d8
github.com/s8sg/goflow/core/runtime/controller/handler.ExecuteFlowHandler(0xc0011484b0, 0xc0007fc600, {0x885bb8, 0xc000902680})
        /home/danash/go/pkg/mod/github.com/s8sg/goflow@v0.1.1/core/runtime/controller/handler/execute_flow_handler.go:33 +0x265
github.com/s8sg/goflow/runtime.newRequestHandlerWrapper.func1({0x882080, 0xc000d382a0}, 0xc00114c000, {0xc000882d80, 0x1, 0xc000c30b70?})
        /home/danash/go/pkg/mod/github.com/s8sg/goflow@v0.1.1/runtime/new_request_handler_wrapper.go:52 +0x6ec
github.com/julienschmidt/httprouter.(*Router).ServeHTTP(0xc00010e1e0, {0x882080, 0xc000d382a0}, 0xc00114c000)
        /home/danash/go/pkg/mod/github.com/julienschmidt/httprouter@v1.3.0/router.go:387 +0x81c
net/http.serverHandler.ServeHTTP({0x880de8?}, {0x882080, 0xc000d382a0}, 0xc00114c000)  
        /nix/store/i6a9j315dl95az9a9vjpgya90y6j9z66-go-1.20.2/share/go/src/net/http/server.go:2936 +0x316
net/http.(*conn).serve(0xc0002a8e10, {0x8822d8, 0xc0001181e0})              
        /nix/store/i6a9j315dl95az9a9vjpgya90y6j9z66-go-1.20.2/share/go/src/net/http/server.go:1995 +0x612
created by net/http.(*Server).Serve                                                                                    
        /nix/store/i6a9j315dl95az9a9vjpgya90y6j9z66-go-1.20.2/share/go/src/net/http/server.go:3089 +0x5ed
2023/04/24 13:39:29 Reporting span 338343586c9e519f:62cdaa3f404a5f84:6cad4d79616ee4e9:1
2023/04/24 13:39:29 Reporting span 338343586c9e519f:6cad4d79616ee4e9:338343586c9e519f:1
2023/04/24 13:39:29 Reporting span 338343586c9e519f:7818de4473a991b2:338343586c9e519f:1
2023/04/24 13:39:29 Reporting span 338343586c9e519f:354240b2e50c0ada:6cad4d79616ee4e9:1
2023/04/24 13:39:29 Reporting span 338343586c9e519f:6cad4d79616ee4e9:338343586c9e519f:1
2023/04/24 13:39:29 Reporting span 338343586c9e519f:6cad4d79616ee4e9:338343586c9e519f:1
2023/04/24 13:39:29 Executing flow myflow                                                                              
2023/04/24 13:39:29 Reporting span 16ba1e6c69d0d230:38cef6c7b9bc1643:16ba1e6c69d0d230:1
2023/04/24 13:39:29 Reporting span 16ba1e6c69d0d230:06ac2ac89bb22560:2cec2b7378dfba32:1
2023/04/24 13:39:29 http: panic serving 127.0.0.1:51896: [request `ch35ooasig6serk31o8g`] Pipeline is not active
goroutine 639 [running]:                                                                                               
net/http.(*conn).serve.func1()                                                                                         
        /nix/store/i6a9j315dl95az9a9vjpgya90y6j9z66-go-1.20.2/share/go/src/net/http/server.go:1854 +0xbf
panic({0x747500, 0xc000292900})                                                                                        
        /nix/store/i6a9j315dl95az9a9vjpgya90y6j9z66-go-1.20.2/share/go/src/runtime/panic.go:890 +0x263
github.com/s8sg/goflow/core/sdk/executor.(*FlowExecutor).executeNode(0xc0008f5740, {0xc0011c0200, 0x5, 0x200})
        /home/danash/go/pkg/mod/github.com/s8sg/goflow@v0.1.1/core/sdk/executor/executor.go:346 +0x645
github.com/s8sg/goflow/core/sdk/executor.(*FlowExecutor).Execute(0xc0008f5740, 0xc000a49720)
        /home/danash/go/pkg/mod/github.com/s8sg/goflow@v0.1.1/core/sdk/executor/executor.go:1257 +0x8cb
github.com/s8sg/goflow/core/runtime/controller/handler.ExecuteFlowHandler(0xc00047c2d0, 0xc000181c80, {0x885bb8, 0xc0009b8000})
        /home/danash/go/pkg/mod/github.com/s8sg/goflow@v0.1.1/core/runtime/controller/handler/execute_flow_handler.go:33 +0x265
github.com/s8sg/goflow/runtime.newRequestHandlerWrapper.func1({0x882080, 0xc0011bc0e0}, 0xc000454000, {0xc000cfe080, 0x1, 0xc000a49b70?})
        /home/danash/go/pkg/mod/github.com/s8sg/goflow@v0.1.1/runtime/new_request_handler_wrapper.go:52 +0x6ec
github.com/julienschmidt/httprouter.(*Router).ServeHTTP(0xc00010e1e0, {0x882080, 0xc0011bc0e0}, 0xc000454000)
        /home/danash/go/pkg/mod/github.com/julienschmidt/httprouter@v1.3.0/router.go:387 +0x81c
net/http.serverHandler.ServeHTTP({0x880de8?}, {0x882080, 0xc0011bc0e0}, 0xc000454000)
        /nix/store/i6a9j315dl95az9a9vjpgya90y6j9z66-go-1.20.2/share/go/src/net/http/server.go:2936 +0x316
net/http.(*conn).serve(0xc0003223f0, {0x8822d8, 0xc0001181e0})
        /nix/store/i6a9j315dl95az9a9vjpgya90y6j9z66-go-1.20.2/share/go/src/net/http/server.go:1995 +0x612
created by net/http.(*Server).Serve
        /nix/store/i6a9j315dl95az9a9vjpgya90y6j9z66-go-1.20.2/share/go/src/net/http/server.go:3089 +0x5ed
2023/04/24 13:39:29 debug logging disabled
2023/04/24 13:39:29 Initializing logging reporter
2023/04/24 13:39:29 Reporting span 16ba1e6c69d0d230:2cec2b7378dfba32:16ba1e6c69d0d230:1
2023/04/24 13:39:29 Reporting span 16ba1e6c69d0d230:37ed544c17db1cb0:2cec2b7378dfba32:1
2023/04/24 13:39:29 Reporting span 16ba1e6c69d0d230:2cec2b7378dfba32:16ba1e6c69d0d230:1
2023/04/24 13:39:29 Reporting span 1b7dd13c2c4c0da7:1b7dd13c2c4c0da7:0000000000000000:1
2023/04/24 13:39:29 Reporting span 1b7dd13c2c4c0da7:027da292581982b2:0d43717ff3b76e9f:1
2023/04/24 13:39:29 Reporting span 1b7dd13c2c4c0da7:1b7dd13c2c4c0da7:0000000000000000:1
2023/04/24 13:39:29 ERROR: Repeated attempt to close the reporter is ignored
2023/04/24 13:39:29 Reporting span 1b7dd13c2c4c0da7:0d43717ff3b76e9f:1b7dd13c2c4c0da7:1
2023/04/24 13:39:29 debug logging disabled
2023/04/24 13:39:29 Reporting span 1b7dd13c2c4c0da7:1b7dd13c2c4c0da7:0000000000000000:1
2023/04/24 13:39:29 ERROR: Repeated attempt to close the reporter is ignored
2023/04/24 13:39:29 Reporting span 1b7dd13c2c4c0da7:1b7dd13c2c4c0da7:0000000000000000:1
2023/04/24 13:39:29 ERROR: Repeated attempt to close the reporter is ignored
2023/04/24 13:39:29 Reporting span 5983b20c54d0ee71:13e86d6139141d0a:79fe1508dadda560:1
2023/04/24 13:39:29 Reporting span 5983b20c54d0ee71:79fe1508dadda560:5983b20c54d0ee71:1
2023/04/24 13:39:29 Reporting span 5983b20c54d0ee71:5983b20c54d0ee71:0000000000000000:1
2023/04/24 13:39:29 Reporting span 5983b20c54d0ee71:5983b20c54d0ee71:0000000000000000:1
2023/04/24 13:39:29 ERROR: Repeated attempt to close the reporter is ignored
2023/04/24 13:39:29 Reporting span 5983b20c54d0ee71:5983b20c54d0ee71:0000000000000000:1
2023/04/24 13:39:29 ERROR: Repeated attempt to close the reporter is ignored
2023/04/24 13:39:29 Reporting span 5983b20c54d0ee71:5983b20c54d0ee71:0000000000000000:1
2023/04/24 13:39:29 ERROR: Repeated attempt to close the reporter is ignored

By implementing https://github.com/s8sg/goflow/issues/46 we might achieve a more robustness handling for requests

dan-ash commented 1 year ago

Can be closed when #50 merged