voltbras / go-ocpp

v1.5 and v1.6 OCPP implementation in Golang
GNU General Public License v3.0
46 stars 7 forks source link

Usage of ReadMessage() eats high amount of memory #7

Closed oscarjairIT closed 3 years ago

oscarjairIT commented 4 years ago

Hello, first of all, I would like to say thanks for the great job of this tool. I get an issue about optimization of memory, I'm getting like 30 cp connections currently, and each one send messages to the cs like 10 messages per minute, so in just 1 hour I get like 10 GB of ram usage ...The problem seems to be the allocation of memory with the ReadMessage method of gorilla/websocket. I've been reading about the high usage of memory with gorilla websocket, and the problem it's solved when change the usage of ReadMessage helper to NextReader function, calling NextReader directly and passing the preallocated buffer to the Read method, but I don't know how implement this one to the ReadMessage() function on ws/conn.go file. Any idea? or future or actual missed implementation?.

eduhenke commented 3 years ago

Hi sorry for the late response.

To change the ReadMessage(https://github.com/voltbras/go-ocpp/blob/master/ws/conn.go#L152), you would have to change the UnmarshalMessage function as well, probably by using the json.NewDecoder, that given a Reader, unmarshals the message(https://github.com/voltbras/go-ocpp/blob/master/ws/conn.go#L102). We would lose the log.Debug that logs the current message.

But something is off to me, you said that in an hour you notice a huge bump in the memory usage. You have about 10 messages per minute of 30 CPs, a message averaging around 1KB would not be even near the 10GB mark. So that would suggest a memory leak somewhere. I don't see that changing the ReadMessage to the NextReader would help in this memory leak, it would only make the unmarshalling of the message to happen in a stream-like fashion instead of in a "drop single large byte array", and this single byte array would be only about 1KB.

You could try changing that and see if it works, but it seems to me that something else is responsible for this possible memory leak.