Data in GDrive + How-To Access

henrylao commented 3 years ago

Overview

The dataset is large and requiring everyone to download and install when using Google's Colab is impractical since the cloud notebooks can go offline.

TODO

[x] Create data directory in a cloud drive for team to access
[x] Add example of how-to access when using Colab (didn't know we can share and access drives shared to us)

henrylao commented 3 years ago

Google Drive Data Directory for Agriculture 2021: https://drive.google.com/drive/folders/1uxoS9R9aFeaQhy0g2ejJ7a7auR02Tb2f?usp=sharing

ManishKakarla commented 3 years ago

we can share the whole directory and used by anyone in colab but for that we need to setup a shared drive which means we have to upload the dataset to that newly created share drive. -how to use the shared drive in colab -For now me and henry downloaded the dataset

henrylao commented 3 years ago

Reopened issue of the shared dataset drive CS663-MSCG-Net/input/Agricultural-Vision-2021/train/labels missing files due to extraction issues

henrylao commented 3 years ago

Overview

Current issue when running the notebook that can be found at: https://colab.research.google.com/drive/1R5bmswERYFjyL7MJZdgCjrxpDfrMTlN7?usp=sharing

Code

Running the notebook with the following code results in the following exception due to a missing filepath for the ground-truth file(s)

!python train_R50.py

Exception

[Errno 2] No such file or directory: 'tools'
/content/P3-SemanticSegmentation/tools
Dataset Root Directory: /content/drive/MyDrive/shortcuts/CS663-MSCG-Net/input/Agriculture-Vision-2021
train set ------- 56944
val set --------- 18334
---curr_iter: 0, num_iter per epoch: 5695---
/content/drive/MyDrive/shortcuts/CS663-MSCG-Net/input/Agriculture-Vision-2021/train/images/nir/CEK6CVXQL_1096-7487-1608-7999.jpg
False
/content/drive/MyDrive/shortcuts/CS663-MSCG-Net/input/Agriculture-Vision-2021/train/gt/CEK6CVXQL_1096-7487-1608-7999.png
True
Traceback (most recent call last):
  File "train_R50.py", line 257, in <module>
    main()
  File "train_R50.py", line 111, in main
    for i, (inputs, labels) in enumerate(train_loader):
  File "/usr/local/lib/python3.7/dist-packages/torch/utils/data/dataloader.py", line 521, in __next__
    data = self._next_data()
  File "/usr/local/lib/python3.7/dist-packages/torch/utils/data/dataloader.py", line 561, in _next_data
    data = self._dataset_fetcher.fetch(index)  # may raise StopIteration
  File "/usr/local/lib/python3.7/dist-packages/torch/utils/data/_utils/fetch.py", line 49, in fetch
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "/usr/local/lib/python3.7/dist-packages/torch/utils/data/_utils/fetch.py", line 49, in <listcomp>
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "/content/P3-SemanticSegmentation/data/AgricultureVision/loader.py", line 53, in __getitem__
    label = imload(self.mask_files[idx], gray=True, scale_rate=self.scale)
  File "/content/P3-SemanticSegmentation/data/augmt.py", line 61, in imload
    image = np.asarray(image, dtype='uint8')
  File "/usr/local/lib/python3.7/dist-packages/numpy/core/_asarray.py", line 83, in asarray
    return array(a, dtype, copy=False, order=order)
TypeError: int() argument must be a string, a bytes-like object or a number, not 'NoneType'

akailany / Pattern-Segmentation-in-Aerial-Agricultural-Images