Mem-Leak: new mqtt messagio use more memery and course GC frequently, while new string()

[wsky]

https://github.com/wsky/top-push/blob/master/src/test/java/com/tmall/top/push/websocket/WebSocketPushServerTest.java#L243

send 10000messages cost 60s, use 400M+ mem, and slow.

reduce maxMessageBufferCount from 10000 to 1000 can work

initHolder.setInitParameter("maxMessageBufferCount", "1000");

MqttMessageIO maybe have mem leak bug

[wsky]

MqttMessageIO

public static void readVariableHeader(MqttVariableHeader header, ByteBuffer buffer){}
public static void writeVariableHeader(MqttVariableHeader header, ByteBuffer buffer){}

have memery leak bug!

public void read_variable_header_perf() throws Exception {
        byte[] bytes = new byte[1024];
        ByteBuffer buffer = ByteBuffer.wrap(bytes);
        MqttHeader header = new MqttHeader();
        header.Qos = MqttQos.AtLeastOnce;
        MqttPublishVariableHeader vHeader1 = new MqttPublishVariableHeader(
                header);
            vHeader1.MessageIdentifier = 10;
        vHeader1.TopicName = "abc中文";

        MqttMessageIO.writeVariableHeader(vHeader1, buffer);

        int total = 10000000;
        StopWatch watch = new StopWatch();
        watch.start();
        for (int i = 0; i < total; i++) {
            buffer.position(0);
            MqttMessageIO.readVariableHeader(vHeader1, buffer);
        }
        watch.stop();
        System.out
                .println(String.format("---- readVariableHeader %s cost %sms",
                        total, watch.getTime()));

    }

the

        vHeader1.TopicName = "abc中文";

[wsky]

https://github.com/wsky/top-push/commit/f65e4fad86bd20b79e80fd924080392fc2135e49#L4R281

public static String readMqttString(ByteBuffer buffer) {
        int l = (buffer.get() << 8) + buffer.get();
        if (l == 0)
            return "";
        // int msb = buffer.get() & 0x00FF;
        // int lsb = buffer.get() & 0x00FF;
        // msb = (msb << 8) | lsb;
        String str = new String(buffer.array(), buffer.position(), l,
                Charset.forName("UTF-8"));
        buffer.position(buffer.position() + l);
        return str;
    }

memory leak here

even terrible than:

public static String readMqttString(ByteBuffer buffer) {
        byte[] lengthBytes = new byte[2];
        buffer.get(lengthBytes, 0, 2);
        short stringLength = (short) ((lengthBytes[0] << 8) + lengthBytes[1]);
        byte[] stringBytes = new byte[stringLength];
        buffer.get(stringBytes, 0, stringLength);
        return new String(stringBytes, Charset.forName("UTF-8"));
    }

?

String str = "";
         for (int i = 0; i < l; i++)
         str += buffer.getChar();

also slow

[wsky]

avoid Chinese?

ascii is fast

[wsky]

https://github.com/wsky/top-push/commit/55d18d8e6458ba4bc5e583fa4f05ba2ef10cf8e6

[wsky]

MessageIO

public static String padClientId(String id) {
        // HACK:8 is faster!
        if(id.length()==8)
            return id;
        return String.format("%8s", id);//bad perf
    }

[wsky]

MqttPublishMessage pub = (MqttPublishMessage) message;
        buffer.position(0);

        pub.Header.RemainingLength = MqttMessageIO
                .getVariableHeaderWriteLength(pub.VariableHeader)
                + MessageIO.getFullMessageSize(pub.remainingLength);

        writeHeader(message.Header, buffer);
        writeVariableHeader(pub.VariableHeader, buffer);

        MessageIO.writeMessageType(buffer, message.messageType);
        MessageIO.writeClientId(buffer, message.to);
        MessageIO.writeRemainingLength(buffer, message.remainingLength);

variable header is complex, but remainglength contain it, we need calculate it twice before sending

MqttMessageIO.getVariableHeaderWriteLength, it was bad perf while dealing with string and UTF-8.

[wsky]

https://blogs.oracle.com/xuemingshen/entry/faster_new_string_bytes_cs

Faster new String(bytes, cs/csn) and String.getBytes(cs/csn) in JDK7