What on earth is a Tensor?

Tensors

Tensors are specialized data structures that are very similar to arrays and matrices. PyTorch uses tensors to encode inputs and outputs of a model as well as the model's parameters.

Like NumPy arrays and ndArrays, but tensors run on GPUs or other hardware accelerators. Think robots and drones and the lot. Also, if you are familiar with ndArrays, you'll be right at home, if not, that's ok too.

In fact, tensors and NumPy arrays often share the same underlying memory address with capabilities called bridge-to-np-labels. This eliminates the need to actually copy data. Tensors are optimized for automatic differentiation (I promise this will be fun).

Ok let's set up our environment: Environment

%matplotlib inline import torch import numpy as np

Initializing a tensor Tensors can be initialized in a few ways. We're going to start with the one that I am most comfortable with explaining.

Directly from a data source Tensors can be created directly from a data source. The cool thing is the data type is automatically inferred.

Tensor From Data data = [[1, 2],[3, 4]] x_data = torch.tensor(data)

From a NumPy array Tensors can also be created from NumPy arrays and what's even cooler is Tensors can create NumPy arrays. Since, numpy _'nparray' and _'xnp' share the same memory address, changing the value for one will change the other.

Tensor From NumPy Array

np_array = np.array(data) x_np = torch.from_numpy(np_array)

print(f"Numpy np_array value: \n {np_array} \n") print(f"Tensor x_np value: \n {x_np} \n")

np.multiply(np_array, 2, out=np_array)

print(f"Numpy np_array after * 2 operation: \n {np_array} \n") print(f"Tensor x_np value after modifying numpy array: \n {x_np} \n")

From Another Tensor Ok hear me out, the new tensor retains the properties like shape, data type, etc of the argument tensor, unless you explicitly override it.

Tensor From Tensor

x_ones = torch.ones_like(x_data) # retains the properties of x_data print(f"Ones Tensor: \n {x_ones} \n")

x_rand = torch.rand_like(x_data, dtype=torch.float) # overrides the datatype of x_data print(f"Random Tensor: \n {x_rand} \n")

From Random or Constant Values Shape is defined by a tuple of tensor dimensions, which set the number of rows and columns in a tensor. In the functions below, shape determines the dimensionality of the output tensor.

Tensor From Random Values

shape = (2,3) rand_tensor = torch.rand(shape) ones_tensor = torch.ones(shape) zeros_tensor = torch.zeros(shape)

print(f"Random Tensor: \n {rand_tensor} \n") print(f"Ones Tensor: \n {ones_tensor} \n") print(f"Zeros Tensor: \n {zeros_tensor}")

Tensor Attributes Tensor attributes describ their shape, data type, and the device in which they are stored.

Tensor Attributes

tensor = torch.rand(3,4)

print(f"Shape of tensor: {tensor.shape}") print(f"Datatype of tensor: {tensor.dtype}") print(f"Device tensor is stored on: {tensor.device}")

Tensor Operations Ok this is going to sound intimidating but there are more than 100 tensor operations, like arithmetic, linear algebra, matrices (transposing, indexing and dicing (or is it slicing?). For collecting samples and reviewing, I found this comprehensive description here.

Each of these operations can be run on the GPU because they can typically handle higher speeds than on a CPU.

CPUs have up to 16 cores. Cores are units that do the actual computation. Each core processes tasks in a sequential order - one at a time.

GPUs have 1000s of cores. GPU cores handle computations in parallel processing. Tasks are divided and processed across the different cores. That's what makes GPUs faster than CPUs in most cases. GPUs perform better with large datasets compared to small datasets. GPU are typically used for high-intensive computation of graphics or neural networks (we'll learn more about that later in a Neural Network unit).

PyTorch can use the Nvidia CUDA library to take advantage of their GPU cards.

By default, tensors are created on the CPU (but why though?). Tensors can also be computed to GPUs; to do that, you need to move them using the .to method after checking for GPU availability (that's why). Keep in mind that copying large tensors across devices can be expensive in terms of time and memory!

Tensor Operations

# We move our tensor to the GPU if available if torch.cuda.is_available(): tensor = tensor.to('cuda')

Click here for some sample operations.

hmustafajr / PyTorch-Fundamentals

What on earth is a Tensor? #1