Jagged Array to Tensor

[gktval]

Coming from my memory replay, I have a an array of states (float[]). When I put these into a batch, I have a jagged array (float[][]). I am trying to convert these from the c# array to a tensor, but am getting errors.

Here is what I have tried

float[] stateArray = expBatch.Select(f => f.State).ToArray();
Tensor states = from_array(stateArray, ScalarType.Float32);

This gives me an error that System.Single[] is not supported.

Is there another way I can get an array of state values into a Tensor? If not, how else can I batch process the array of state values?

[yueyinqiu]

Hmm... But in your example stateArray is a float[] rather than a jagged array or something else? So I'm not really sure what are you asking... Anyway the codes you have provided could definitely work:

---

Maybe you are asking for this?

using TorchSharp;
using static TorchSharp.torch;

float[][] stateArray = [[1, 2], [3, 4]];
Tensor states = from_array(stateArray, ScalarType.Float32);
Console.WriteLine(states.cstr());

I suppose it's a bit tricky to support those jagged array, since if your array looks like [[1, 2], [3]], what the shape should be the tensor?

[gktval]

Sorry for the confusion. stateArray is batched. In this case, my batch size is 64. So stateArray would look something similar to below:

float[][] stateArray = new float[64][];
stateArray[0] = new float[8] { .22f, -.45f, .76f, .54f, -.02f, .11f, -.23f, -.35f };
stateArray[1] = new float[8] { -.45f, .87f, -.35f, .05f, -.76f, .94f, -.73f, .24f };
...
stateArray[63] = new float[8] {...}

[yueyinqiu]

My suggestion is to convert it to a multidimensional array (i.e. float[64, 8]) on your own. (And if you could change the way to create the stateArray, which just returns an multidimensional array, it would be much better.)

One reason to not support that in TorchSharp could be that we can't create tensors from all the jagged arrays, like [[1, 2], [3]]. And the other point is that the only thing TorchSharp could do is to just convert it in advance, because jagged arrays are not continuous in memory and it cannot be directly passed to LibTorch, I suppose.

[NiklasGustafsson]

Supporting jagged arrays is a reasonable request, but they are not supported at the moment. You have to first convert to a MD-array, or (which I think is better) create an empty tensor with the intended shape, and then write a few lines to copy data into it from your jagged array. That's what a generic implementation would do, and it wouldn't be as efficient as what you can do it when you know the number of dimensions and length of each dimension.

A generic solution will be quite slow in comparison.

[gktval]

Thanks for helping me out. A multidimensional array works great.