vm: Override GetString for Buffer/ByteString ?

[ixje]

GetString() is described as https://github.com/neo-project/neo-vm/blob/f596fa8fa4fc3c394019e2d299d6a795befaeec7/src/Neo.VM/Types/StackItem.cs#L175 which I interpret to mean that for a Buffer/ByteString with data of byte[] {0xF1, 0xF2, 0xF3} I should get "F1F2F3"

Instead I get

Unable to translate bytes [F1] at index 0 from specified code page to Unicode.

That is because the default implementation uses StrictUTF8 https://github.com/neo-project/neo-vm/blob/f596fa8fa4fc3c394019e2d299d6a795befaeec7/src/Neo.VM/Types/StackItem.cs#L180

As a result I think that in many cases StdLib.jsonSerialize can fail e.g. as soon as the provided stack item contains a script hash like this example script that calls Ledger.currentHash -> StdLib.jsonSerialize

c21f0c0b63757272656e74486173680c14bef2043140362a77c15099c7e64c12f700b665da41627d5b5211c01f0c0d6a736f6e53657269616c697a650c14c0ef39cee0e4e925c6c2a06a79e1440dd86fceac41627d5b52

I know this is a bit of a silly example, but that's not the point

[shargon]

As a result I think that in many cases StdLib.jsonSerialize can fail e.g. as soon as the provided stack item contains a script hash like this example script that calls Ledger.currentHash -> StdLib.jsonSerialize

This hash must be encoded as hex or base64, and it should work, the unique problem is with non utf8 strings, isn't it?

[ixje]

The problem is I can't jsonSerialize a pretty simple and valid StackItem. I first have to figure out what items I have to base64 encode. That to me breaks the idea behind accepting StackItem as type.

[cschuchardt88]

@ixje Hope this helps

private JObject ToJObject(StackItem item, IList<(StackItem, JObject)> context)
        {
            JObject o = null;
            switch (item)
            {
                case Array array:
                    if (context is null)
                        context = new List<(StackItem, JObject)>();
                    else
                        (_, o) = context.FirstOrDefault(f => ReferenceEquals(f.Item1, item));
                    if (o is null)
                    {
                        context.Add((item, o));
                        var a = array.Select(s => ToJObject(s, context));
                        o = new()
                        {
                            new JProperty("Type", StackItemType.Array.ToString()),
                            //new JProperty("size", array.Count),
                            new JProperty("Value", a),
                        };
                    }
                    break;
                case Map map:
                    if (context is null)
                        context = new List<(StackItem, JObject)>();
                    else
                        (_, o) = context.FirstOrDefault(f => ReferenceEquals(f.Item1, item));
                    if (o is null)
                    {
                        context.Add((item, o));
                        var kvp = map.Select(s => new KeyValuePair<JObject, JObject>(ToJObject(s.Key, context), ToJObject(s.Value, context)));
                        var js = JsonConvert.SerializeObject(kvp);
                        o = new()
                        {
                            new JProperty("Type", StackItemType.Map.ToString()),
                            //new JProperty("size", map.Count),
                            new JProperty("Value", JArray.Parse(js)),
                        };
                    }
                    break;
                case Boolean _:
                    o = new()
                    {
                        new JProperty("Type", StackItemType.Boolean.ToString()),
                        new JProperty("Value", item.GetBoolean()),
                    };
                    break;
                case Buffer buffer:
                    o = new()
                    {
                        new JProperty("Type", StackItemType.Buffer.ToString()),
                        new JProperty("Value", buffer.GetSpan().ToArray()),
                    };
                    break;
                case ByteString array:
                    o = array.Size switch
                    {
                        UInt160.Length => new()
                            {
                                new JProperty("Type", nameof(UInt160)),
                                new JProperty("Value", new UInt160(array.GetSpan()).ToString()),
                            },
                        UInt256.Length => new()
                            {
                                new JProperty("Type", nameof(UInt256)),
                                new JProperty("Value", new UInt256(array.GetSpan()).ToString()),
                            },
                        _ => new()
                            {
                                new JProperty("Type", StackItemType.ByteString.ToString()),
                                new JProperty("Value", array.GetSpan().ToHexString()),
                            },
                    };
                    break;
                case Integer i:
                    o = new()
                    {
                        new JProperty("Type", StackItemType.Integer.ToString()),
                        new JProperty("Value", i.GetInteger()),
                    };
                    break;
                case InteropInterface _:
                    o = new()
                    {
                        new JProperty("Type", StackItemType.InteropInterface.ToString()),
                    };
                    break;
                case Pointer pointer:
                    o = new()
                    {
                        new JProperty("Type", StackItemType.Pointer.ToString()),
                        new JProperty("Vaule", pointer.Position),
                    };
                    break;
                case Null _:
                    o = new()
                    {
                        new JProperty("Type", StackItemType.Any.ToString()),
                    };
                    break;
                default:
                    throw new NotImplementedException($"StackItemType({item.Type}) is not supported to JSON.");
            }
            return o;
        }

[ixje]

Thanks @cschuchardt88 but this issue is something that needs to be resolved for all nodes, not just my use-case. The main problem is that there is data on the chain that you can't retrieve via RPC because of this issue.