Problem with DataLoader and pickles

[FraLuca]

Hi,

I'm trying to reproduce your experiment and i had some problems.

First of all i create an environment, as suggested by Trajectron++ with all their requirements, and then produced the pickle files to run your code on ETH-UCY.

Then, as you suggested, i create the path: data\ETH_UCY_trajectron\trajectories and i put here the pickles processed by trajectron++. In this way your configuration file (eg. bitrap_np_ETH.yml) matches exactly the data folder.

I run this: python tools\train.py --config_file configs/bitrap_np_ETH.yml I'm using Anaconda on Windows 10 system without cuda.

Issue 1

Traceback (most recent call last):
  File "tools/train.py", line 14, in <module>
    from datasets.build import make_dataloader
ModuleNotFoundError: No module named 'datasets'

I solved this moving the main script "train.py" to its parent folder, where it can see datasets folder. It looks like the row: sys.path.append(os.path.realpath('..')) doesn't work.

So i'm running this: python train.py --config_file configs/bitrap_np_ETH.yml

Issue 2

Traceback (most recent call last):
  File "train.py", line 14, in <module>
    from datasets.build import make_dataloader
ModuleNotFoundError: No module named 'datasets.build'

I solved changing the import of:

from datasets.build import make_dataloader
from bitrap.modeling.build  import make_model

to:

from datasets.__init__ import make_dataloader
from bitrap.modeling.__init__ import make_model

Issue 3

optimizer built!

Traceback (most recent call last):
  File "train.py", line 146, in <module>
    main()
  File "train.py", line 79, in main
    train_dataloader = make_dataloader(cfg, 'train')
  File "C:\Users\Admin\Desktop\Dottorato\ICCV 2021\BITRAP\bidireaction-trajectory-prediction\datasets\__init__.py", line 50, in make_dataloader
    dataset = make_dataset(cfg, split)
  File "C:\Users\Admin\Desktop\Dottorato\ICCV 2021\BITRAP\bidireaction-trajectory-prediction\datasets\__init__.py", line 36, in make_dataset
    return data_layer(cfg, split)
  File "C:\Users\Admin\Desktop\Dottorato\ICCV 2021\BITRAP\bidireaction-trajectory-prediction\datasets\ETH_UCY.py", line 41, in __init__
    train_env = dill.load(f, encoding='latin1')
  File "C:\Users\Admin\Anaconda3\envs\trajectron++\lib\site-packages\dill\_dill.py", line 270, in load
    return Unpickler(file, ignore=ignore, **kwds).load()
  File "C:\Users\Admin\Anaconda3\envs\trajectron++\lib\site-packages\dill\_dill.py", line 472, in load
    obj = StockUnpickler.load(self)
  File "C:\Users\Admin\Anaconda3\envs\trajectron++\lib\site-packages\dill\_dill.py", line 462, in find_class
    return StockUnpickler.find_class(self, module, name)
ModuleNotFoundError: No module named 'environment'

The problem occurs when i load the pickles with dill module. I solved copy pasting the three folders named: "environment", "model" and "utils" from the Trajectron++ repo (Trajectron-plus-plus\trajectron) to the root folder of your github. Because i think they are missing the modules used to generate the pickles. So it is necessary to reload them.

Issue 4

optimizer built!
train dataloader: 296
val dataloader: 43
test dataloader: 3
Dataloader built!
Training engine built!
Schedulers built!
  0%|                                                              | 0/296 [00:00<?, ?it/s]

Traceback (most recent call last):
  File "train.py", line 146, in <module>
    main()
  File "train.py", line 134, in main
    do_train(cfg, epoch, model, optimizer, train_dataloader, cfg.DEVICE, logger=logger, lr_scheduler=lr_scheduler)
  File "C:\Users\Admin\Desktop\Dottorato\ICCV 2021\BITRAP\bidireaction-trajectory-prediction\bitrap\engine\trainer.py", line 29, in do_train
    for iters, batch in enumerate(tqdm(dataloader), start=1):
  File "C:\Users\Admin\Anaconda3\envs\trajectron++\lib\site-packages\tqdm\std.py", line 1127, in __iter__
    for obj in iterable:
  File "C:\Users\Admin\Anaconda3\envs\trajectron++\lib\site-packages\torch\utils\data\dataloader.py", line 279, in __iter__
    return _MultiProcessingDataLoaderIter(self)
  File "C:\Users\Admin\Anaconda3\envs\trajectron++\lib\site-packages\torch\utils\data\dataloader.py", line 719, in __init__
    w.start()
  File "C:\Users\Admin\Anaconda3\envs\trajectron++\lib\multiprocessing\process.py", line 105, in start
    self._popen = self._Popen(self)
  File "C:\Users\Admin\Anaconda3\envs\trajectron++\lib\multiprocessing\context.py", line 223, in _Popen
    return _default_context.get_context().Process._Popen(process_obj)
  File "C:\Users\Admin\Anaconda3\envs\trajectron++\lib\multiprocessing\context.py", line 322, in _Popen
    return Popen(process_obj)
  File "C:\Users\Admin\Anaconda3\envs\trajectron++\lib\multiprocessing\popen_spawn_win32.py", line 65, in __init__
    reduction.dump(process_obj, to_child)
  File "C:\Users\Admin\Anaconda3\envs\trajectron++\lib\multiprocessing\reduction.py", line 60, in dump
    ForkingPickler(file, protocol).dump(obj)
_pickle.PicklingError: Can't pickle <function augment at 0x000001D914E0F1E0>: attribute lookup augment on __main__ failed

So i stopped here because it looks like something is still missing. Maybe something you changed from Trajectron++. I saw a similar issue in their github repo, where it is suggested to change the NUM_WORKERS to 0. So i tried switching in the config file DATALOADER.NUM_WORKERS to 0. However it produces a new error:

Traceback (most recent call last):
  File "train.py", line 146, in <module>
    main()
  File "train.py", line 134, in main
    do_train(cfg, epoch, model, optimizer, train_dataloader, cfg.DEVICE, logger=logger, lr_scheduler=lr_scheduler)
  File "C:\Users\Admin\Desktop\Dottorato\ICCV 2021\BITRAP\bidireaction-trajectory-prediction\bitrap\engine\trainer.py", line 29, in do_train
    for iters, batch in enumerate(tqdm(dataloader), start=1):
  File "C:\Users\Admin\Anaconda3\envs\trajectron++\lib\site-packages\tqdm\std.py", line 1127, in __iter__
    for obj in iterable:
  File "C:\Users\Admin\Anaconda3\envs\trajectron++\lib\site-packages\torch\utils\data\dataloader.py", line 345, in __next__
    data = self._next_data()
  File "C:\Users\Admin\Anaconda3\envs\trajectron++\lib\site-packages\torch\utils\data\dataloader.py", line 385, in _next_data
    data = self._dataset_fetcher.fetch(index)  # may raise StopIteration
  File "C:\Users\Admin\Anaconda3\envs\trajectron++\lib\site-packages\torch\utils\data\_utils\fetch.py", line 44, in fetch
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "C:\Users\Admin\Anaconda3\envs\trajectron++\lib\site-packages\torch\utils\data\_utils\fetch.py", line 44, in <listcomp>
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "C:\Users\Admin\Desktop\Dottorato\ICCV 2021\BITRAP\bidireaction-trajectory-prediction\datasets\ETH_UCY.py", line 67, in __getitem__
    neighbors_edge_value, robot_traj_st_t, map_tuple, scene_name, timestep = self.dataset.__getitem__(index) #
  File "C:\Users\Admin\Desktop\Dottorato\ICCV 2021\BITRAP\bidireaction-trajectory-prediction\datasets\ETH_UCY.py", line 128, in __getitem__
    self.edge_types, self.max_ht, self.max_ft, self.hyperparams)
  File "C:\Users\Admin\Desktop\Dottorato\ICCV 2021\BITRAP\bidireaction-trajectory-prediction\datasets\preprocessing.py", line 84, in get_node_timestep_data
    x, _ = node.get(timestep_range_x, state[node.type])
ValueError: too many values to unpack (expected 2)

Thank you for your help in advance, Kind regards and good work. Luca

[MoonBlvd]

Hi @FraLuca Thank you for your interest and your patience, and sorry for the delay of response! I have solved your issues by 1) adding PYTHONPATH instruction to readme file 2) modifying sys.path in tools/train.py and 3) adding costomized trajectron++ dependency to the repo. Please merge with the updated repo and feel free to let me know if this deos not solve your problem!

[sanketh1691]

Hi @MoonBlvd , I am trying to extract the pedestrian trajectory as asked in the README of this repo in the form of pickle files from JAAD and PIE original repos but unable to find proper functions that do the required operations to generate the pedestrian trajectory data. Can you help/guide me on how to extract this data?

Thank you, Sanketh

[MoonBlvd]

@sanketh1691 please refer to #13