starwing / lua-protobuf

A Lua module to work with Google protobuf
MIT License
1.75k stars 387 forks source link

反序列化来自C#的二进制流的时候,出现错误 #16

Closed Jayatubi closed 6 years ago

Jayatubi commented 6 years ago

通过C#序列化一个带有repeated int32字段的proto结构体到二进制流,在通过lua-protobuf反序列化将会出现错误:

type mismatch at offset 2, varint expected for type int32, got bytes

proto文件定义:

syntax="proto3";

message MyMessage
{
    repeated int32 intList = 1;
}

C#代码:

using System.IO;

public class Application
{
    public static int Main(string[] args)
    {
        var instance = Google.Protobuf.JsonParser.Default.Parse<MyMessage>(File.ReadAllText("data.json"));
        using (var ms = new MemoryStream())
        using (var cs = new Google.Protobuf.CodedOutputStream(ms))
        {
            instance.WriteTo(cs);
            cs.Flush();
            File.WriteAllBytes("pb.bin", ms.ToArray());
        }

        return 0;
    }
}

data.json

{
    "intList": [
        1,
        2,
        3
    ]    
}

C#生成的二进制流:

  Offset: 00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F   
00000000: 0A 03 01 02 03                                     .....

Lua代码:

local pb = require("pb")
local json = require("rapidjson")

if pb.loadfile("../proto/pb.description") then
    local file = io.open("../csharp/pb.bin", "rb")
    local bytes = file:read("*a")
    file:close()
    local obj, message = pb.decode("MyMessage", bytes)
    if obj ~= nil then
        print(json.encode(obj))
    else
        print(message)
    end
else
    print("pb description load failed")
end
Jayatubi commented 6 years ago

大概知道原因了。Proto3里面的repeated默认是[packed=true],但是lua-protobuf没有做这个处理。手动在proto里面加上[packed=true]可以破。

官方描述摘录:https://developers.google.com/protocol-buffers/docs/encoding

Version 2.1.0 introduced packed repeated fields, which in proto2 are declared like repeated fields but with the special [packed=true] option. In proto3, repeated fields are packed by default. These function like repeated fields, but are encoded differently. A packed repeated field containing zero elements does not appear in the encoded message. Otherwise, all of the elements of the field are packed into a single key-value pair with wire type 2 (length-delimited). Each element is encoded the same way it would be normally, except without a tag preceding it.

starwing commented 6 years ago

fix it by HEAD

JackZheng1989 commented 3 years ago

大概知道原因了。Proto3里面的repeated默认是[packed=true],但是lua-protobuf没有做这个处理。手动在proto里面加上[packed=true]可以破。

官方描述摘录:https://developers.google.com/protocol-buffers/docs/encoding

Version 2.1.0 introduced packed repeated fields, which in proto2 are declared like repeated fields but with the special [packed=true] option. In proto3, repeated fields are packed by default. These function like repeated fields, but are encoded differently. A packed repeated field containing zero elements does not appear in the encoded message. Otherwise, all of the elements of the field are packed into a single key-value pair with wire type 2 (length-delimited). Each element is encoded the same way it would be normally, except without a tag preceding it.

尝试在proto中repeated字段后添加[packed=true],还是没有解决,使用的proto3: image

image

starwing commented 3 years ago

@JackZheng1989 检查你的lua-protobuf版本,尽量用最新的,最新的已经不需要packed了

TongZiGang commented 3 years ago

@starwing 作者你好,上面提到的错误,在ARM平台上依然会报这个错误,目前我是用Unity打出来的apk,如果编译选项里面 image 勾选了ARM64,那么打出来的apk在真机上就会报这个错误,如果在电脑的模拟器上运行则不会,如果不勾选ARM64,则都不会报这个错,可是由于上架Google Play支持ARM64的,所以麻烦作者大大能看下么?

starwing commented 3 years ago

你ARM里包的lua_protobuf版本有更新吗?

TongZiGang commented 3 years ago

有更新,我今天拉的,重新打的包放进去的,使用的是这边推荐的https://github.com/chexiongsheng/build_xlua_with_libs这个地址上的工具重新构建的

TongZiGang commented 2 years ago

@starwing 作者大大有头绪么?这个我应该怎么做?

starwing commented 2 years ago

@TongZiGang 如果Windows的正常,那Android的肯定也正常,除非版本不一样。因为代码都是跨平台的。建议还是检察一下pb.c/pb.h和protoc.lua这三个文件和HEAD的diff。

TongZiGang commented 2 years ago

@starwing 好的,谢谢

TongZiGang commented 2 years ago

@starwing 我之前没有在Unity平台上跑就直接打包跑了,今天突然发现我用我重新打的库跑的话在Unity上跑也会报错,我看了一下后面的问答,看看能不能找出问题来。