libffcv / ffcv

FFCV: Fast Forward Computer Vision (and other ML workloads!)
https://ffcv.io
Apache License 2.0
2.81k stars 180 forks source link

Fix translate fill #184

Closed andrewilyas closed 1 year ago

andrewilyas commented 2 years ago

The translate transformation used to ignore its fill parameter and just use whatever happened to be in the dst tensor. This fixes that.

lengstrom commented 2 years ago

Tried running this and was missing the astor pip dependency

lengstrom commented 2 years ago

also get the following error when making a CIFAR data loader: (this is fixed by using the pip version of FFCV)

Traceback (most recent call last):
  File "/mnt/nfs/home/engstrom/conda_envs/ffcvrgb/lib/python3.9/runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/mnt/nfs/home/engstrom/conda_envs/ffcvrgb/lib/python3.9/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/mnt/nfs/home/engstrom/src/datamodeler/datamodels/training/worker.py", line 94, in <module>
    main()
  File "/mnt/nfs/home/engstrom/conda_envs/ffcvrgb/lib/python3.9/site-packages/fastargs/decorators.py", line 63, in result
    return func(*args, **kwargs)
  File "/mnt/nfs/home/engstrom/conda_envs/ffcvrgb/lib/python3.9/site-packages/fastargs/decorators.py", line 35, in __call__
    return self.func(*args, **filled_args)
  File "/mnt/nfs/home/engstrom/src/datamodeler/datamodels/training/worker.py", line 76, in main
    to_log = module.main(index=index, logdir=worker_logs)
  File "/mnt/nfs/home/engstrom/src/membership_inference/cifar10/train_cifar.py", line 174, in main
    loaders = make_dataloaders(mask=np.nonzero(mask)[0])
  File "/mnt/nfs/home/engstrom/conda_envs/ffcvrgb/lib/python3.9/site-packages/fastargs/decorators.py", line 63, in result
    return func(*args, **kwargs)
  File "/mnt/nfs/home/engstrom/conda_envs/ffcvrgb/lib/python3.9/site-packages/fastargs/decorators.py", line 35, in __call__
    return self.func(*args, **filled_args)
  File "/mnt/nfs/home/engstrom/src/membership_inference/cifar10/train_cifar.py", line 86, in make_dataloaders
    loaders[name] = Loader(paths[name], indices=(mask if name == 'train' else None),
  File "/mnt/nfs/home/engstrom/src/ffcvrgb/ffcv/loader/loader.py", line 210, in __init__
    self.generate_code()
  File "/mnt/nfs/home/engstrom/src/ffcvrgb/ffcv/loader/loader.py", line 275, in generate_code
    queries, code = self.graph.collect_requirements()
  File "/mnt/nfs/home/engstrom/src/ffcvrgb/ffcv/pipeline/graph.py", line 347, in collect_requirements
    self.collect_requirements(next_state, node, allocations, code, source_field=source_field)
  File "/mnt/nfs/home/engstrom/src/ffcvrgb/ffcv/pipeline/graph.py", line 347, in collect_requirements
    self.collect_requirements(next_state, node, allocations, code, source_field=source_field)
  File "/mnt/nfs/home/engstrom/src/ffcvrgb/ffcv/pipeline/graph.py", line 347, in collect_requirements
    self.collect_requirements(next_state, node, allocations, code, source_field=source_field)
  [Previous line repeated 3 more times]
  File "/mnt/nfs/home/engstrom/src/ffcvrgb/ffcv/pipeline/graph.py", line 328, in collect_requirements
    if next_state.device.type != 'cuda' and isinstance(operation,
AttributeError: 'str' object has no attribute 'type'

If the problem isn't immediately obvious I can debug / make a test case.

andrewilyas commented 2 years ago

I think these are both properties in the v1.0.0 branch and not of the specific pull request---I can make a few new pull requests to fix these issues (should just be a matter of removing an import astor and changing the string device to ch.device.

andrewilyas commented 1 year ago

Merging this!