04_pytorch_custom_datasets_video.ipynb - RuntimeError: only batches of spatial targets supported (3D tensors) but got targets of dimension: 1

[nickawesome07]

Create the train_step()

def train_step(model: torch.nn.Module, dataloader: torch.utils.data.DataLoader, loss_fn: torch.nn.Module, optimizer: torch.optim.Optimizer, device=device):

Put the model in train mode

model.train()

Setup train loss and train accuracy values

train_loss, train_acc = 0, 0

Loop through data loader and data batches

for batch, (X, y) in enumerate(dataloader):

Send data to the target device

X, y = X.to(device), y.to(device)

# 1. Forward pass
y_pred = model(X) # output model logits

# 2. Calculate loss
loss = loss_fn(y_pred, y)
train_loss += loss.item()

# 3. Optimizer zero grad
optimizer.zero_grad()

# 4. Loss backward
loss.backward()

# 5. Optimizer step
optimizer.step()

# Caclulate the accuracy metric
y_pred_class = torch.argmax(torch.softmax(y_pred, dim=1), dim=1)
train_acc += (y_pred_class == y).sun().item() / len(y_pred)

Adjust metrics to get the average loss and accuracy per batch

train_loss = train_loss / len(dataloader) train_acc = train_acc / len(dataloader) return train_loss, train_acc

Define the test step function

def test_step(model: torch.nn.Module, data_loader: torch.utils.data.DataLoader, loss_fn: torch.nn.Module, device=device): """Performs testing loop on model for data_loader"""

Put model in training mode

model.eval()

Setup test loss and test accuracy

test_loss, test_acc = 0, 0

Testing

Turn on inference mode context manager

with torch.inference_mode(): for batch, (X, y) in enumerate(data_loader):

Put data on device

  X, y = X.to(device), y.to(device)

  # 1. forward pass
  test_pred_logits = model(X)

  # 2. Cacl the loss (accumulative)
  test_loss += loss_fn(test_pred_logits, y.long())
  # test_loss += loss.item() # Remove this line, it's causing the loss to accumulate incorrectly

  # Calculate the accuracy
  test_pred_labels = test_pred_logits.argmax(dim=1)
  test_acc += ((test_pred_labels == y).sum().item()/len(test_pred_labels))

Adjust metrics to get average loss and accuracy per batch

test_loss = test_loss / len(data_loader) test_acc = test_acc / len(data_loader) return test_loss, test_acc

from tqdm.auto import tqdm

1. Create a train function that takes in various model params + optimizer + etc

def train(model: torch.nn.Module, train_dataloader: torch.utils.data.DataLoader, test_dataloader: torch.utils.data.DataLoader, optimizer: torch.optim.Optimizer, loss_fn: torch.nn.Module = nn.CrossEntropyLoss(), epochs: int = 5, seed: int = 42, device=device):

2. Create an empty results dict

results = {"train_loss": [], "train_acc": [], "test_loss": [], "test_acc": []}

3. Loop through training and testing steps for number of epochs

torch.manual_seed(seed) torch.cuda.manual_seed(seed) epochs = epochs for epoch in tqdm(range(epochs)): train_loss, train_acc = train_step(model=model, dataloader=train_dataloader, loss_fn=loss_fn, optimizer=optimizer, device=device) test_loss, test_acc = test_step(model=model, data_loader=test_dataloader, loss_fn=loss_fn, device=device)

# 4. Print out whats happening
print(f"Epoch: {epoch} | Train loss: {train_loss:.4f} | Train acc: {train_acc:.4f} | Test loss: {test_loss:.4f} | Test acc: {test_acc:.4f}")

# 5. Update results dict
results["train_loss"].append(train_loss)
results["train_acc"].append(train_acc)
results["test_loss"].append(test_loss)
results["test_acc"].append(test_acc)

6. Return the filled results at the end of the epochs

return results

Set random seeds

torch.manual_seed(42) torch.cuda.manual_seed(42)

Set number of epochs

NUM_EPOCHS = 5

Recreate an instance of TinyVGG

model_0 = TinyVGG(input_shape=3, # number of color channels of our target images hidden_units=10, output_shape=len(train_data.classes)).to(device)

Setup loss function and optimizer

loss_fn = nn.CrossEntropyLoss() optimizer = torch.optim.Adam(params=model_0.parameters(), lr=0.001)

Start the timer

from timeit import default_timer as timer start_time = timer()

Train model_0

model_0_results = train(model=model_0, train_dataloader=train_dataloader_simple, test_dataloader=test_dataloader_simple, optimizer=optimizer, epochs=NUM_EPOCHS)

End the timer and print out how long it took

end_time = timer() print(f"Total training time: {end_time - start_time:.3f} seconds")

Error

torch.Size([32, 3, 64, 64]) torch.Size([32, 3, 64, 64]) torch.Size([32, 3, 64, 64])

RuntimeError Traceback (most recent call last) in <cell line: 25>() 23 24 # Train model_0 ---> 25 model_0_results = train(model=model_0, 26 train_dataloader=train_dataloader_simple, 27 test_dataloader=test_dataloader_simple,

5 frames /usr/local/lib/python3.10/dist-packages/torch/nn/functional.py in cross_entropy(input, target, weight, size_average, ignore_index, reduce, reduction, label_smoothing) 3084 if size_average is not None or reduce is not None: 3085 reduction = _Reduction.legacy_get_string(size_average, reduce) -> 3086 return torch._C._nn.cross_entropy_loss(input, target, weight, _Reduction.get_enum(reduction), ignore_index, label_smoothing) 3087 3088 RuntimeError: only batches of spatial targets supported (3D tensors) but got targets of dimension: 1

[pritesh2000]

you can use ``` (a sign before "1" thrice) to open and close code snippet

print("hello world")

[pritesh2000]

mention how you created transforms and dataloaders.

[nickawesome07]

1. Load and transform data

from torchvision import datasets train_data_simple = datasets.ImageFolder(root=train_dir, transform=simple_transform) test_data_simple = datasets.ImageFolder(root=test_dir, transform=simple_transform)

2. Turn the datasets into dataloaders

import os from torch.utils.data import DataLoader

setup batchsize and num of workers

BATCH_SIZE = 32 NUM_WORKERS = os.cpu_count()

Create DataLoaders

train_dataloader_simple = DataLoader(dataset=train_data_simple, batch_size=BATCH_SIZE, shuffle=True, num_workers=NUM_WORKERS) test_dataloader_simple = DataLoader(dataset=test_data_simple, shuffle=False, num_workers=NUM_WORKERS)

[pritesh2000]

I don't think you have made any error in what code you had sent, I tried to copy your code but it is all massy , can you provide link to your code(colab link like) or link .ipynb file here