Dimension error of multi-dimensional arrays at serialization

[Perpete]

Hello, While developing a Vb project with visual studio 16.10.1 using FrameWork 4.8, I noticed a problem with the realization of multi-dimensional arrays. In my project, I have a List (Of ..) which contains a 2-dimensional array in a property. After the following operations:

• realization of my List (Of ..)
• saving a .json file
• opening the .json file
• desealization in a new List (Of ..)

you can see that the array has changed to 3 dimensions.

During the serialization, the dimension of the array changes with the value of the digit of the 2nd dimension of the array. Which is abnormal.

Dim MyTableau (1,1) Serealization gives : [0,0], [0,0], [0,0], [0,0] Dim MyTableau (1,2) Sealization gives : [0,0,0], [0,0,0], instead of: [0,0], [0,0], [0,0], [0,0], [0,0], [0,0] Dim MyTableau (1,3) Serealization gives : [0,0,0,0], [0,0,0,0] instead of: [0,0], [0,0], [0,0], [0,0], [0,0], [0,0], [0,0], [0,0]

[elgonzo]

Please provide a brief code example that clearly reproduces your issue. (Also make sure you are using the current version 13.0.1 of Json.NET)

As you can see here on dotnetfiddle https://dotnetfiddle.net/QT1oRf (the example code is in C#, but that shouldn't matter), there are no issues when doing straightforward serialization of two-dimensional arrays, nested lists or nested arrays (you said you are using 2-dimensional arrays, but your mention of List Of and the debugger screenshot hint at you rather using nested lists and/or nested arrays; the latter often also called "jagged arrays"). Which means, for understanding the issue you are facing and whether it is an issue with Json.NET or with some custom serialization code you might employ, it would help to know how exactly your serialization-related code looks like...

[Perpete]

Hello, I am using the latest version of Json.NET My program reads an file.xml and retrieves some elements being variable definitions some of which are arrays.

In this example, from the information in the file.xml, I am constructing an array. I place in a List (Of TcpServerResponse.ItemVariableTCP), the necessary elements for my variable. `

Public Class ItemVariableTCP

    'Définition des informations sur les variables de l'automate

    Public Property NameVariable As String
    Public Property Value As Object
    Public Property ReadEnable As Boolean
    Public Property WriteEnable As Boolean
    Public Property TypeVariable As Type
    Public Property Tableau As Boolean

End Class

The Value property contains my array created from : `

'Crée le tableau comme valeur
Dim DimensionalArray As Array = Array.CreateInstance(newItemVariableTCP.TypeVariable, ArrayDimension)
newItemVariableTCP.Value = DimensionalArray`

After reading my whole file.xml and composing my variable list, I serialize this list and save the serialization to a file.json. `

    'Séréalise le fichier.xml
    Dim fileJson As String = JsonConvert.SerializeObject(MyItemVariableTCP)

    'Sauvegarde le fichier
    File.WriteAllText(pathPrefix & fileName & ".json", fileJson)`

It is by editing the file.json that I see that the dimension value of the array has changed.

Then, I read the file.json and deserialize into a new list of variables. The Value property of the list confirms the change in the dimensions of the array. `

    'Lecture du fichier Json
    fileJson = File.ReadAllText(pathPrefix & fileName & ".json")

    Dim MyItemVariableTCPopen As New List(Of TcpServerResponse.ItemVariableTCP)

    MyItemVariableTCPopen = JsonConvert.DeserializeObject(Of List(Of TcpServerResponse.ItemVariableTCP))(fileJson)`

Here is the full code :

`

Private Sub btImportListCodesys_Click(sender As Object, e As RoutedEventArgs) Handles btImportListCodesys.Click

    'Création de la liste des variables à partir d'un fichier exporter pour Codesys dans le programme WAGO 'e!COCKPIT

    Dim Dimension As List(Of Integer)
    Dim ArrayDimension As Integer()
    Dim MyItemVariableTCP As New List(Of TcpServerResponse.ItemVariableTCP)
    Dim fileName As String = "OPC_Symbole"
    Dim docXml As New XmlDocument()
    Dim pathPrefix As String = Path.GetFullPath(AppDomain.CurrentDomain.BaseDirectory & "\..\..\ListVariable\")

    'Ouvre le document xml d'exportation pour codesys
    docXml.Load(pathPrefix & fileName & ".export")

    'Recherche le début des listes des variables dans les différents éléments de l'automate
    Dim ElementList As XmlNodeList = docXml.SelectNodes("//List2[@Name='SignVariables']")

    For Each itemElementList As XmlNode In ElementList

        'Recherche le début des listes des variables dans l'élément de l'automate
        Dim VariableList As XmlNodeList = itemElementList.SelectNodes("Single")

        For Each itemVariableList As XmlNode In VariableList

            'Recherche les données définissant la variable
            Dim dataVariableList As XmlNodeList = itemVariableList.SelectNodes("Single")

            Dim newItemVariableTCP As New TcpServerResponse.ItemVariableTCP
            For Each itemdataVariableList As XmlNode In dataVariableList

                If itemdataVariableList.Attributes("Name").Value = "VarAccess" Then

                    'Attribue le type d'accès de la variable
                    Select Case CInt(itemdataVariableList.InnerText)
                        Case 1
                            newItemVariableTCP.WriteEnable = False
                            newItemVariableTCP.ReadEnable = True
                        Case 2
                            newItemVariableTCP.WriteEnable = True
                            newItemVariableTCP.ReadEnable = False
                        Case 3
                            newItemVariableTCP.WriteEnable = True
                            newItemVariableTCP.ReadEnable = True
                    End Select

                ElseIf itemdataVariableList.Attributes("Name").Value = "VarType" Then

                    Dim ExtractTypeAutomate As String = itemdataVariableList.InnerText

                    'Vérifie certain type de variable

                    If itemdataVariableList.InnerText.Contains("ARRAY [") Then
                        'La variable est un tableau
                        newItemVariableTCP.Tableau = True

                        'Extrait la dimension

                        Dimension = New List(Of Integer)
                        Dim ExtractDimension As String = itemdataVariableList.InnerText.Substring(itemdataVariableList.InnerText.IndexOf("[") + 1)
                        ExtractDimension = ExtractDimension.Substring(0, ExtractDimension.IndexOf("]"))
                        Dim splitDimension As String() = ExtractDimension.Split(","c)
                        For Each indice As String In splitDimension
                            'Récupère l'indice de départ et de fin
                            Dim ValueIndiceStart As Integer = CInt(indice.Substring(0, indice.IndexOf(".")))
                            Dim ValueIndiceEnd As Integer = CInt(indice.Substring(indice.LastIndexOf(".") + 1))

                            'Crée l'indice en base zéro et l'ajoute à une liste
                            Dim indiceBase0 As Integer = ValueIndiceEnd - ValueIndiceStart + 1
                            Dimension.Add(indiceBase0)
                        Next

                        'Crée un tableau avec les dimensions
                        ArrayDimension = Dimension.ToArray

                        'Extrait le type
                        Dim str() As String = ExtractTypeAutomate.Split(" "c)
                        ExtractTypeAutomate = str(str.GetUpperBound(0))
                    Else
                        'La variable n'est pas un tableau
                        newItemVariableTCP.Tableau = False
                    End If

                    If ExtractTypeAutomate.Contains("STRING (") Then
                        'La variable est une chaine de caractère limitée à une taille
                        ExtractTypeAutomate = "STRING"

                    ElseIf ExtractTypeAutomate.Contains("POINTER TO ") Then
                        'La variable est un pointeur (pt:POINTER TO INT;)

                        'Extrait le type
                        Dim str() As String = ExtractTypeAutomate.Split(" "c)
                        ExtractTypeAutomate = str(str.GetUpperBound(0))

                    ElseIf ExtractTypeAutomate.Contains("REFERENCE TO ") Then
                        'La variable est une référence (Alias pour un objet), elle n'est pas prise en compte (ref_int : REFERENCE TO INT;) 
                        Continue For

                    ElseIf ExtractTypeAutomate.Contains("(") And ExtractTypeAutomate.Contains("..") Then
                        'La variable est type domaine partiel, elle n'est pas prise en compte (i : INT (-4095..4095);) 
                        Continue For
                    End If

                    'Attribue le type de la variable et défini si la variable est un tableau
                    Select Case ExtractTypeAutomate
                        Case "BOOL"

                            newItemVariableTCP.TypeVariable = GetType(Boolean)

                            'Place la valeur en fonction du type de variable
                            If newItemVariableTCP.Tableau = True Then

                                'Crée le tableau comme valeur
                                Dim DimensionalArray As Array = Array.CreateInstance(newItemVariableTCP.TypeVariable, ArrayDimension)
                                newItemVariableTCP.Value = DimensionalArray
                            Else
                                newItemVariableTCP.Value = False
                            End If

                        Case "SINT"

                            newItemVariableTCP.TypeVariable = GetType(SByte)

                            'Place la valeur en fonction du type de variable
                            If newItemVariableTCP.Tableau = True Then

                                'Crée le tableau comme valeur
                                Dim DimensionalArray As Array = Array.CreateInstance(newItemVariableTCP.TypeVariable, ArrayDimension)
                                newItemVariableTCP.Value = DimensionalArray
                            Else
                                newItemVariableTCP.Value = 0
                            End If

                        Case "USINT", "BYTE"

                            newItemVariableTCP.TypeVariable = GetType(Byte)

                            'Place la valeur en fonction du type de variable
                            If newItemVariableTCP.Tableau = True Then

                                'Crée le tableau comme valeur
                                Dim DimensionalArray As Array = Array.CreateInstance(newItemVariableTCP.TypeVariable, ArrayDimension)
                                newItemVariableTCP.Value = DimensionalArray
                            Else
                                newItemVariableTCP.Value = 0
                            End If

                        Case "INT"

                            newItemVariableTCP.TypeVariable = GetType(Int16)

                            'Place la valeur en fonction du type de variable
                            If newItemVariableTCP.Tableau = True Then

                                'Crée le tableau comme valeur
                                Dim DimensionalArray As Array = Array.CreateInstance(newItemVariableTCP.TypeVariable, ArrayDimension)
                                newItemVariableTCP.Value = DimensionalArray
                            Else
                                newItemVariableTCP.Value = 0
                            End If

                        Case "UINT", "WORD"

                            newItemVariableTCP.TypeVariable = GetType(UInt16)

                            'Place la valeur en fonction du type de variable
                            If newItemVariableTCP.Tableau = True Then

                                'Crée le tableau comme valeur
                                Dim DimensionalArray As Array = Array.CreateInstance(newItemVariableTCP.TypeVariable, ArrayDimension)
                                newItemVariableTCP.Value = DimensionalArray
                            Else
                                newItemVariableTCP.Value = 0
                            End If

                        Case "DINT"

                            newItemVariableTCP.TypeVariable = GetType(Int32)

                            'Place la valeur en fonction du type de variable
                            If newItemVariableTCP.Tableau = True Then

                                'Crée le tableau comme valeur
                                Dim DimensionalArray As Array = Array.CreateInstance(newItemVariableTCP.TypeVariable, ArrayDimension)
                                newItemVariableTCP.Value = DimensionalArray
                            Else
                                newItemVariableTCP.Value = 0
                            End If

                        Case "UDINT", "DWORD"

                            newItemVariableTCP.TypeVariable = GetType(UInt32)

                            'Place la valeur en fonction du type de variable
                            If newItemVariableTCP.Tableau = True Then

                                'Crée le tableau comme valeur
                                Dim DimensionalArray As Array = Array.CreateInstance(newItemVariableTCP.TypeVariable, ArrayDimension)
                                newItemVariableTCP.Value = DimensionalArray
                            Else
                                newItemVariableTCP.Value = 0
                            End If

                        Case "LINT", "TIME", "LTIME"

                            newItemVariableTCP.TypeVariable = GetType(Int64)

                            'Place la valeur en fonction du type de variable
                            If newItemVariableTCP.Tableau = True Then

                                'Crée le tableau comme valeur
                                Dim DimensionalArray As Array = Array.CreateInstance(newItemVariableTCP.TypeVariable, ArrayDimension)
                                newItemVariableTCP.Value = DimensionalArray
                            Else
                                newItemVariableTCP.Value = 0
                            End If

                        Case "LUINT", "LWORD"

                            newItemVariableTCP.TypeVariable = GetType(UInt64)

                            'Place la valeur en fonction du type de variable
                            If newItemVariableTCP.Tableau = True Then

                                'Crée le tableau comme valeur
                                Dim DimensionalArray As Array = Array.CreateInstance(newItemVariableTCP.TypeVariable, ArrayDimension)
                                newItemVariableTCP.Value = DimensionalArray
                            Else
                                newItemVariableTCP.Value = 0
                            End If

                        Case "REAL"

                            newItemVariableTCP.TypeVariable = GetType(Single)

                            'Place la valeur en fonction du type de variable
                            If newItemVariableTCP.Tableau = True Then

                                'Crée le tableau comme valeur
                                Dim DimensionalArray As Array = Array.CreateInstance(newItemVariableTCP.TypeVariable, ArrayDimension)
                                newItemVariableTCP.Value = DimensionalArray
                            Else
                                newItemVariableTCP.Value = 0
                            End If

                        Case "LREAL"

                            newItemVariableTCP.TypeVariable = GetType(Double)

                            'Place la valeur en fonction du type de variable
                            If newItemVariableTCP.Tableau = True Then

                                'Crée le tableau comme valeur
                                Dim DimensionalArray As Array = Array.CreateInstance(newItemVariableTCP.TypeVariable, ArrayDimension)
                                newItemVariableTCP.Value = DimensionalArray
                            Else
                                newItemVariableTCP.Value = 0
                            End If

                        Case "STRING", "WSTRING"

                            newItemVariableTCP.TypeVariable = GetType(String)

                            'Place la valeur en fonction du type de variable
                            If newItemVariableTCP.Tableau = True Then

                                'Crée le tableau comme valeur
                                Dim DimensionalArray As Array = Array.CreateInstance(newItemVariableTCP.TypeVariable, ArrayDimension)
                                newItemVariableTCP.Value = DimensionalArray
                            Else
                                newItemVariableTCP.Value = ""
                            End If

                        Case "DATE", "DATE_AND_TIME", "TIME_OF_DAY"

                            newItemVariableTCP.TypeVariable = GetType(DateTime)

                            'Place la valeur en fonction du type de variable
                            If newItemVariableTCP.Tableau = True Then

                                'Crée le tableau comme valeur
                                Dim DimensionalArray As Array = Array.CreateInstance(newItemVariableTCP.TypeVariable, ArrayDimension)
                                newItemVariableTCP.Value = DimensionalArray
                            Else
                                newItemVariableTCP.Value = New DateTime(1, 1, 1)
                            End If

                        Case Else
                            Continue For
                    End Select

                ElseIf itemdataVariableList.Attributes("Name").Value = "VarName" Then

                    'Attribue le nom de la variable
                    newItemVariableTCP.NameVariable = itemdataVariableList.InnerText
                End If
            Next

            If newItemVariableTCP.TypeVariable IsNot Nothing Then

                'Ajoute les données de la variable à la liste
                MyItemVariableTCP.Add(newItemVariableTCP)
            End If
        Next
    Next

    'Sérialise le fichier.xml
    Dim fileJson As String = JsonConvert.SerializeObject(MyItemVariableTCP)

    'Sauvegarde le fichier
    File.WriteAllText(pathPrefix & fileName & ".json", fileJson)

    'Lecture du fichier Json
    fileJson = File.ReadAllText(pathPrefix & fileName & ".json")

    Dim MyItemVariableTCPopen As New List(Of TcpServerResponse.ItemVariableTCP)

    MyItemVariableTCPopen = JsonConvert.DeserializeObject(Of List(Of TcpServerResponse.ItemVariableTCP))(fileJson)

End Sub

`

[elgonzo]

Your code does not demonstrate the issue you describe. It seems to work as intended as far as i can tell.

Look at the array definition in the XML sample you have given. It defines ARRAY [0..1, 0..3] OF INT. This translates to a two-dimensional array with the the respective dimensional lengths of 2 and 4.

The json produced by your code contains the array [[0,0,0,0],[0,0,0,0]], which is basically a table with two rows with 4 elements/numbers each, matching the definition as given by the XML.

But i guess i know what the problem is: It looks like you got confused by how the debugger presents the data in your screenshot (and so did i when i looked at the screenshot and the coloring).

In your debugger screenshot, note the green number tuples in the first upper red rectangle. Those numbers themselves are not the array content of the MyItemVariableTCP.Value property. Rather, those (0, 0), (0, 1), ... , (1, 2) are the row/column coordinates of the respective array cells in your two-dimensional cells (the values of those cells being 0 are shown on the right hand of your screenshot). The value of each individual array cell is presented here by its "coordinate" in the two-dimensional array. If you plot these coordinates out, you'll realize that you have a two-dimensional array in MyItemVariableTCP.Value, with two rows (the row coordinate) and 3 elements per row (the column coordinate).

This corresponds perfectly with the json you get. Also, in the debugger screenshot in the lower red rectangle, note that the Value array is shown as a whole (unlike the display in the upper red rectangle, the individual array element are not displayed in separate lines), also showing two rows with 3 elements each.

I have annotated your screenshot to make this clearer:

That said, i have no idea how the final part of your issue report (the lines about Dim MyTableau (1,1) and so on) relate to your code or your screenshots...

[Perpete]

Thank you for your reply, I now understand the difference between the interpretations of the array value displays for vb and json. The only problem is while deserializing, as my Value property is an object (contains different types of values), i was thinking of directly fetching this object as a Vb array. Here is my confusion. So I will have to rebuild the array by interpreting the value of the property after deserialization. [0,0,0], [0,0,0] will give an array (1,2) with (0,0) = 0, (0,1) = 0 ... etc [1,2,0,0], [10,0,0,0] will give an array (1,3) with (0,0) = 1, (0,1) = 2 .... (1,0) = 10 ...etc

Is there another way to reconstruct the arrays?

[elgonzo]

Since the Value property in your model class is type object, Json.NET has no idea about the intended data type during deserialization and thus deserializes the value of this property as one of the concrete JToken types (JValue, JObject, JArray depending on what the actual content of the respective json property is).

Unless you allow Json.NET to store type information along-side the serialized Value property data, you have to do the multi-dimension array conversion kinda manually, unfortunately. You didn't say how exactly you reconstruct the n-dimensional array. In case you are not aware of this already, you could create a type object for your n-dimensional array (using Type.MakeArrayType(int dimensions)) and then using JToken's/JArray's .ToObject(Type t) method with the type object created to create the array in at once. Something like (sorry, C# again. I am not really a VB programmer):

if (DeserializedItemVariableTCP.Value is JArray ja)
{
    var dimensions = ... Inspect the JArray ja to get the number of dimensions
    var multiDimArrayType = DeserializedItemVariableTCP.TypeVariable.MakeArrayType(dimensions);
    var array = ja.ToObject(multiDimArrayType);
    ...  array is ready to be used ...
}

On the other hand, if it would be permissible for you to let Json.NET store type information in the json -- that is not your custom type information, but type information Json.NET and other serializers understand -- you could probably avoid this manual processing of the Value property after/during deserialization by simply annotating the Value property with the JsonPropertyAttribute and setting its TypeNameHandling parameter accordingly (https://www.newtonsoft.com/json/help/html/P_Newtonsoft_Json_JsonPropertyAttribute_TypeNameHandling.htm), something like this:

Public Class ItemVariableTCP
    ....
    <Newtonsoft.Json.JsonProperty(TypeNameHandling:=Newtonsoft.Json.TypeNameHandling.All)>
    Public Property Value As Object
    ....
End Class

[Perpete]

Hello, I used the type name handling used when serializing the Value property as described in your comment. (<Newtonsoft.Json.JsonProperty(TypeNameHandling:=Newtonsoft.Json.TypeNameHandling.All)>)

It works perfectly without intervention on the reconstruction of the array. It is much simpler.

Many thanks for your follow-up.