multi-input-neural-network-pytorch/

[utterances-bot]

Multi-Input Deep Neural Networks with PyTorch-Lightning - Combine Image and Tabular Data - Python Tutorials for Machine Learning, Deep Learning and Data Visualization

A small tutorial on how to combine tabular and image data for regression prediction in PyTorch-Lightning.

https://rosenfelder.ai/multi-input-neural-network-pytorch/

[asheeshmathur]

Very Interesting and informative exercise you did, would like to explore and try out this example. Could not find images and other data on github. Could you please share that as well

[asheeshmathur]

Got it, its mentioned on the top. data.zip. Wonderful article

[k-darshil]

Hello. The link doesn't seem to be working. I could not find the dataset in Kaggle either. Can you someone help me to access the dataset please.

[MarkusRosen]

Hello. The link doesn't seem to be working. I could not find the dataset in Kaggle either. Can you someone help me to access the dataset please.

This link is working for me in Chrome and Firefox. Can you try again please?

https://1drv.ms/u/s!AqUPqx8G81xZiL1l80RtZbjPj43MhA?e=KagzKc

[riskiem]

Hello,

This is very helpful for me for a project I am doing. I made the following changes to main since I don't have cuda installed: if name == "main": logger = TensorBoardLogger("lightning_logs", name="image_only") early_stop_callback = EarlyStopping(monitor="val_loss", min_delta=5000, patience=7, verbose=False, mode="min")

model = LitClassifier()
#trainer = pl.Trainer(gpus=1, logger=logger, early_stop_callback=early_stop_callback)
trainer = pl.Trainer(accelerator='cpu', logger=logger,early_stop_callback=early_stop_callback)

lr_finder = trainer.tuner.lr_find(model)
fig = lr_finder.plot(suggest=True, show=True)
new_lr = lr_finder.suggestion()
print(new_lr)
model.hparams.lr = new_lr

trainer.fit(model)
trainer.test(model)

Firstly, I want to know if this is correct? And then I keep getting following error:

---

TypeError Traceback (most recent call last) Cell In [6], line 7 5 model = LitClassifier() 6 #trainer = pl.Trainer(gpus=1, logger=logger, early_stop_callback=early_stop_callback) ----> 7 trainer = pl.Trainer(accelerator='cpu', logger=logger,early_stop_callback=early_stop_callback) 9 lr_finder = trainer.tuner.lr_find(model) 10 fig = lr_finder.plot(suggest=True, show=True)

File ~/opt/miniconda3/envs/CNNRegression/lib/python3.8/site-packages/pytorch_lightning/utilities/argparse.py:345, in _defaults_from_env_vars..insert_env_defaults(self, *args, kwargs) 342 kwargs = dict(list(env_variables.items()) + list(kwargs.items())) 344 # all args were already moved to kwargs --> 345 return fn(self, kwargs)

TypeError: init() got an unexpected keyword argument 'early_stop_callback'

Any help would be much appreciated, this is the only source I can find that can help me with a CNN Regression and I am very thankful for this.

[MarkusRosen]

@riskiem Most likely the newer versions of PyTorch Lightning have changed their API for callbacks. Could you check if your version of PyTorch and PyTorch Lightning are the same as the ones used by the tutorial? You can check the versions in my requirements.txt file. In your environment, use pip list to see the version you are using.

[Mayor88]

Great tutorial. I have been trying to pass in my own image and numerical data. Used an image from the image dataset and passed in numerical data as tensor like this:

image_data = ImageDataset(pickle_file=f"{data_path}df.pkl", image_dir=f"{data_path}processed_images/")

test_prediction = model(image_data[0][0].unsqueeze(0).to(device)), torch.FloatTensor([40.8127, -73.9191, 5.0000, 3.0000, 2280.0000]).to(device)

print(test_prediction)

But then I get this error:

TypeError: forward() missing 1 required positional argument: 'tab'

Would appreciate some help with this. Thanks.

[Mayor88]

Noticed a problem with parenthesis in the comment above. When I fixed it like this:

test_pred = model(image_data[0][0].unsqueeze(0).to(device), torch.FloatTensor([40.8127, -73.9191, 5.0000, 3.0000, 2280.0000]).to(device))

I got this error:

RuntimeError: Tensors must have same number of dimensions: got 2 and 1

[Mayor88]

I was able to get predictions by adding a batch dimension to the Float Tensor for the numeric data.

test_pred = model(image_data[0][0].unsqueeze(0).to(device), torch.FloatTensor([40.8127, -73.9191, 5.0000, 3.0000, 2280.0000]).unsqueeze(0).to(device))

Thanks for this great tutorial.

[d-krolicki]

Hi, could you explain how do you calculate the input shape for the linear layer here? self.ln1 = nn.Linear(64 * 26 * 26, 16) It's present at line 71 in your notebook. Thanks in advance!

[raffdoc]

Hi, How you'll change your code if instead of regression problem you should have a classification? Thanks.