Question on training with TM dataset

[e96031413]

I used the following command to train the model with TM dataset python ./train.py --model Conv4NP --method comet --train_aug

But, the following errors occur

Traceback (most recent call last):
  File "./train.py", line 13, in <module>
    from data.datamgr import SimpleDataManager, SetDataManager
  File "/root/notebooks/nfs/work/yanwei.liu/comet/TM/data/__init__.py", line 1, in <module>
    from . import datamgr
  File "/root/notebooks/nfs/work/yanwei.liu/comet/TM/data/datamgr.py", line 7, in <module>
    import data.additional_transforms as add_transforms
ModuleNotFoundError: No module named 'data.additional_transforms'

Thus I read comet/TM/data__init__.py. Line 3 from . import additional_transforms so I checkcomet/TM/data/ folder, there's no any .py file named additional_transforms. Did you forget to push the file to GitHub? or the command I used wasn't correct?

Thanks

PS. I have used the process.py to preprocess the dataset according to your guideline in README file.

[e96031413]

I solve the issue by copy comet/CUB/data/additional_transforms.py to comet/TM/data/

[e96031413]

After execute the same command to train, I encounter the following error:

_________Gene count processed_________
mgi2go_set 24581
adata_set 2866
union 0
./filelists/tabula_muris/go-basic.obo: fmt(1.2) rel(2018-10-24) 47,358 GO Terms
Num filtered GOs: 0
Trying to set attribute `.obs` of view, copying.
/opt/conda/lib/python3.6/site-packages/scanpy/preprocessing/_simple.py:343: UserWarning: Revieved a view of an AnnData. Making a copy.
  view_to_actual(adata)
_________Gene count processed_________
mgi2go_set 24581
adata_set 2866
union 0
./filelists/tabula_muris/go-basic.obo: fmt(1.2) rel(2018-10-24) 47,358 GO Terms
Num filtered GOs: 0
Traceback (most recent call last):
  File "./train.py", line 160, in <module>
    model = train(base_loader, val_loader,  model, optimization, start_epoch, stop_epoch, params, tf_writer)
  File "./train.py", line 35, in train
    model.train_loop(epoch, base_loader,  optimizer, tf_writer) #model are called by reference, no need to return
  File "/root/notebooks/nfs/work/yanwei.liu/comet/TM/methods/meta_template.py", line 58, in train_loop
    for i, (x, _) in enumerate(train_loader):
  File "/opt/conda/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 435, in __next__
    data = self._next_data()
  File "/opt/conda/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 1085, in _next_data
    return self._process_data(data)
  File "/opt/conda/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 1111, in _process_data
    data.reraise()
  File "/opt/conda/lib/python3.6/site-packages/torch/_utils.py", line 428, in reraise
    raise self.exc_type(msg)
IndexError: Caught IndexError in DataLoader worker process 0.
Original Traceback (most recent call last):
  File "/opt/conda/lib/python3.6/site-packages/torch/utils/data/_utils/worker.py", line 198, in _worker_loop
    data = fetcher.fetch(index)
  File "/opt/conda/lib/python3.6/site-packages/torch/utils/data/_utils/fetch.py", line 47, in fetch
    return self.collate_fn(data)
  File "/opt/conda/lib/python3.6/site-packages/torch/utils/data/_utils/collate.py", line 45, in default_collate
    elem = batch[0]
IndexError: list index out of range

[JulianRein]

Not one of the authors, but my understanding from reading the paper is: The default model for COMET on the TM / Single-cell-transcriptomics is "EnFCNet", Conv4NP is for images. So the command should probably be: EDIT: and --train-aug enables data augmentation, which is probably also image-specific. Would leave it out. python ./train.py --model EnFCNet --method comet