search

gogo / protobuf

[Deprecated] Protocol Buffers for Go with Gadgets
Other
5.66k stars 809 forks source link

Panic: runtime error: index out of range [-1] #668

Open lukashes opened 4 years ago

lukashes commented 4 years ago

Hi! I have panic in production really frequent now.

I am using prototool from Uber to build my pb-files with this configuration:

protoc:
  version: 3.9.1
generate:
  plugins:
    - name: gogofaster
      type: gogo
      flags: plugins=grpc
      output: ..

Stack:

And several times per one day I can see panic:

panic: runtime error: index out of range [-1]                                                                                                                                            

goroutine 11143676 [running]:                                                                                                                                                            
app/domain.encodeVarintEntity(0xc0013a6000, 0x903, 0x14339, 0x3, 0xfc6690b, 0x4)                                                                         
        /app/domain/entity.pb.go:1442 +0xb4                                                                                                       
app/domain.(*Entity).MarshalToSizedBuffer(0xc00a2fbc00, 0xc0013a6000, 0x903, 0x14339, 0x8e9, 0x904, 0x0)                                                 
        /app/domain/entity.pb.go:992 +0x1098                                                                                                      
app/grpc/open.(*FindEntitysResponse).MarshalToSizedBuffer(0xc0187e3400, 0xc0013a6000, 0x14339, 0x14339, 0x40bd53, 0xdcee40, 0xe92260)                    
        /app/grpc/open/entity.pb.go:1283 +0xbc                                                                                                   
app/grpc/open.(*FindEntitysResponse).Marshal(0xc0187e3400, 0xe92260, 0xc0187e3400, 0x7ff040ece2f8, 0xc0187e3400, 0xeaf901)                               
        /app/grpc/open/entitys.pb.go:1263 +0x7a                                                                                                   
app/vendor/google.golang.org/grpc/encoding/proto.codec.Marshal(0xe92260, 0xc0187e3400, 0xc01814b170, 0xe08c80, 0x40a8fb, 0xc000020000, 0xdd6d20)
        /app/vendor/google.golang.org/grpc/encoding/proto/proto.go:70 +0x194
app/vendor/google.golang.org/grpc.encode(0x7ff040db9368, 0x17810c0, 0xe92260, 0xc0187e3400, 0x17810c0, 0xeaf940, 0xc01814b170, 0xc00107bac0, 0xc48128)
        /app/vendor/google.golang.org/grpc/rpc_util.go:543 +0x52
app/vendor/google.golang.org/grpc.(*Server).sendResponse(0xc00049e600, 0x1054a20, 0xc001547c80, 0xc014fe8b00, 0xe92260, 0xc0187e3400, 0x0, 0x0, 0xc01901228
8, 0x0, ...)
        /app/vendor/google.golang.org/grpc/server.go:845 +0x89

Entity is really big structure I can't post it here. Do you have any ideas whats wrong?

larytet commented 4 years ago

What was the problem? Here is a theory: while MarshalToSizedBuffer() is performing it's magic the structure is being modified in another thread .

oguzyildizz commented 4 years ago

@larytet yes, I think that's what's happening too, because we're having this problem as well. Is there a way to work around?

003random commented 3 years ago

Got the same issue :/

003random commented 3 years ago

Am on the latest version and am getting the exact same panic as OP. index out of range [-1]. The data I was trying to send was also pretty large and I'm getting this error like once every x hours. (95% of the time I don't get the error and everything works as it should work)

larytet commented 3 years ago

@003random Make sure that the data is not being modified while the MarshalToSizedBuffer() performs serialization. The most likely reason is that one thread called MarshalToSizedBuffer(data) and another thread in the same time modifies one of the buffers in the "data".

jansoneo commented 3 years ago

gogofaster is non-thread-safe

larytet commented 3 years ago

I suggest to close the issue. This is not a problem with the protobuf serialization.

lukashes commented 3 years ago

When I wrote the issue, I used serialization in one thread. And I did not expect this kind of problem.

But issue is not actual for me, because I do not use protobuf now and can't confirm or refute problem is not reproduced. I think we can close issue already.

larytet commented 3 years ago

When I wrote the issue, I used serialization in one thread. And I did not expect this kind of problem.

But issue is not actual for me, because I do not use protobuf now and can't confirm or refute problem is not reproduced. I think we can close issue already.

The likely scenario: