search

pytorch / pytorch

Tensors and Dynamic neural networks in Python with strong GPU acceleration
https://pytorch.org
Other
82.67k stars 22.26k forks source link

torch._dynamo.exc.Unsupported: call_function args: UserDefinedObjectVariable(EasyDict) #120219

Closed adonnini closed 1 month ago

adonnini commented 7 months ago

Hi, I am trying run for inference another model on an android device using executorch. Execution fails with a torch._dynamo.exc.Unsupported: call_function args: UserDefinedObjectVariable(EasyDict) error. Below, you will find the relevant portion of the traceback log. The model I am trying to process is largely based on this model https://github.com/mie-lab/location-prediction/tree/main I did not modify the modules involved in the execution error.

I re-ran the code using 

TORCH_LOGS="+dynamo" and TORCHDYNAMO_VERBOSE=1

The log and traceback log produced are ver long. Please let me know if you want me to post them.

Please let me know if you need additional information. Thanks

TRACEBACK LOG

Traceback (most recent call last):
  File "/home/adonnini1/anaconda3/lib/python3.9/site-packages/torch/_dynamo/utils.py", line 569, in proxy_args_kwargs
    proxy_args = tuple(arg.as_proxy() for arg in args)
  File "/home/adonnini1/anaconda3/lib/python3.9/site-packages/torch/_dynamo/utils.py", line 569, in <genexpr>
    proxy_args = tuple(arg.as_proxy() for arg in args)
  File "/home/adonnini1/anaconda3/lib/python3.9/site-packages/torch/_dynamo/variables/base.py", line 273, in as_proxy
    raise NotImplementedError(str(self))
NotImplementedError: UserDefinedObjectVariable(EasyDict)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/adonnini1/Development/ContextQSourceCode/NeuralNetworks/LocationPrediction/main.py", line 67, in <module>
    res_single = single_run(train_loader, val_loader, test_loader, config, device, log_dir)
  File "/home/adonnini1/Development/ContextQSourceCode/NeuralNetworks/LocationPrediction/main.py", line 23, in single_run
    model, perf = get_trainedNets(config, model, train_loader, val_loader, device, log_dir)
  File "/home/adonnini1/Development/ContextQSourceCode/NeuralNetworks/LocationPrediction/utils/utils.py", line 47, in get_trainedNets
    best_model, performance = trainNet(config, model, train_loader, val_loader, device, log_dir=log_dir)
  File "/home/adonnini1/Development/ContextQSourceCode/NeuralNetworks/LocationPrediction/utils/train.py", line 312, in trainNet
    pre_autograd_aten_dialect = capture_pre_autograd_graph(m, (config, config.total_loc_num, device))
  File "/home/adonnini1/anaconda3/lib/python3.9/site-packages/torch/_export/__init__.py", line 133, in capture_pre_autograd_graph
    m = torch._dynamo.export(
  File "/home/adonnini1/anaconda3/lib/python3.9/site-packages/torch/_dynamo/eval_frame.py", line 1268, in inner
    result_traced = opt_f(*args, **kwargs)
  File "/home/adonnini1/anaconda3/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/adonnini1/anaconda3/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/adonnini1/anaconda3/lib/python3.9/site-packages/torch/_dynamo/eval_frame.py", line 417, in _fn
    return fn(*args, **kwargs)
  File "/home/adonnini1/anaconda3/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/adonnini1/anaconda3/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/adonnini1/anaconda3/lib/python3.9/site-packages/torch/_dynamo/eval_frame.py", line 580, in catch_errors
    return callback(frame, cache_entry, hooks, frame_state)
  File "/home/adonnini1/anaconda3/lib/python3.9/site-packages/torch/_dynamo/convert_frame.py", line 384, in _convert_frame_assert
    return _compile(
  File "/home/adonnini1/anaconda3/lib/python3.9/site-packages/torch/_dynamo/convert_frame.py", line 643, in _compile
    guarded_code = compile_inner(code, one_graph, hooks, transform)
  File "/home/adonnini1/anaconda3/lib/python3.9/site-packages/torch/_dynamo/utils.py", line 247, in time_wrapper
    r = func(*args, **kwargs)
  File "/home/adonnini1/anaconda3/lib/python3.9/site-packages/torch/_dynamo/convert_frame.py", line 524, in compile_inner
    out_code = transform_code_object(code, transform)
  File "/home/adonnini1/anaconda3/lib/python3.9/site-packages/torch/_dynamo/bytecode_transformation.py", line 1033, in transform_code_object
    transformations(instructions, code_options)
  File "/home/adonnini1/anaconda3/lib/python3.9/site-packages/torch/_dynamo/convert_frame.py", line 151, in _fn
    return fn(*args, **kwargs)
  File "/home/adonnini1/anaconda3/lib/python3.9/site-packages/torch/_dynamo/convert_frame.py", line 489, in transform
    tracer.run()
  File "/home/adonnini1/anaconda3/lib/python3.9/site-packages/torch/_dynamo/symbolic_convert.py", line 2098, in run
    super().run()
  File "/home/adonnini1/anaconda3/lib/python3.9/site-packages/torch/_dynamo/symbolic_convert.py", line 780, in run
    and self.step()
  File "/home/adonnini1/anaconda3/lib/python3.9/site-packages/torch/_dynamo/symbolic_convert.py", line 743, in step
    getattr(self, inst.opname)(inst)
  File "/home/adonnini1/anaconda3/lib/python3.9/site-packages/torch/_dynamo/symbolic_convert.py", line 462, in wrapper
    return inner_fn(self, inst)
  File "/home/adonnini1/anaconda3/lib/python3.9/site-packages/torch/_dynamo/symbolic_convert.py", line 1190, in CALL_FUNCTION
    self.call_function(fn, args, {})
  File "/home/adonnini1/anaconda3/lib/python3.9/site-packages/torch/_dynamo/symbolic_convert.py", line 644, in call_function
    self.push(fn.call_function(self, args, kwargs))
  File "/home/adonnini1/anaconda3/lib/python3.9/site-packages/torch/_dynamo/variables/nn_module.py", line 329, in call_function
    return tx.inline_user_function_return(
  File "/home/adonnini1/anaconda3/lib/python3.9/site-packages/torch/_dynamo/symbolic_convert.py", line 650, in inline_user_function_return
    return InliningInstructionTranslator.inline_call(self, fn, args, kwargs)
  File "/home/adonnini1/anaconda3/lib/python3.9/site-packages/torch/_dynamo/symbolic_convert.py", line 2233, in inline_call
    return cls.inline_call_(parent, func, args, kwargs)
  File "/home/adonnini1/anaconda3/lib/python3.9/site-packages/torch/_dynamo/symbolic_convert.py", line 2348, in inline_call_
    tracer.run()
  File "/home/adonnini1/anaconda3/lib/python3.9/site-packages/torch/_dynamo/symbolic_convert.py", line 780, in run
    and self.step()
  File "/home/adonnini1/anaconda3/lib/python3.9/site-packages/torch/_dynamo/symbolic_convert.py", line 743, in step
    getattr(self, inst.opname)(inst)
  File "/home/adonnini1/anaconda3/lib/python3.9/site-packages/torch/_dynamo/symbolic_convert.py", line 462, in wrapper
    return inner_fn(self, inst)
  File "/home/adonnini1/anaconda3/lib/python3.9/site-packages/torch/_dynamo/symbolic_convert.py", line 1231, in CALL_FUNCTION_EX
    self.call_function(fn, argsvars.items, kwargsvars)
  File "/home/adonnini1/anaconda3/lib/python3.9/site-packages/torch/_dynamo/symbolic_convert.py", line 644, in call_function
    self.push(fn.call_function(self, args, kwargs))
  File "/home/adonnini1/anaconda3/lib/python3.9/site-packages/torch/_dynamo/variables/functions.py", line 301, in call_function
    return super().call_function(tx, args, kwargs)
  File "/home/adonnini1/anaconda3/lib/python3.9/site-packages/torch/_dynamo/variables/functions.py", line 255, in call_function
    return super().call_function(tx, args, kwargs)
  File "/home/adonnini1/anaconda3/lib/python3.9/site-packages/torch/_dynamo/variables/functions.py", line 84, in call_function
    return tx.inline_user_function_return(
  File "/home/adonnini1/anaconda3/lib/python3.9/site-packages/torch/_dynamo/symbolic_convert.py", line 650, in inline_user_function_return
    return InliningInstructionTranslator.inline_call(self, fn, args, kwargs)
  File "/home/adonnini1/anaconda3/lib/python3.9/site-packages/torch/_dynamo/symbolic_convert.py", line 2233, in inline_call
    return cls.inline_call_(parent, func, args, kwargs)
  File "/home/adonnini1/anaconda3/lib/python3.9/site-packages/torch/_dynamo/symbolic_convert.py", line 2348, in inline_call_
    tracer.run()
  File "/home/adonnini1/anaconda3/lib/python3.9/site-packages/torch/_dynamo/symbolic_convert.py", line 780, in run
    and self.step()
  File "/home/adonnini1/anaconda3/lib/python3.9/site-packages/torch/_dynamo/symbolic_convert.py", line 743, in step
    getattr(self, inst.opname)(inst)
  File "/home/adonnini1/anaconda3/lib/python3.9/site-packages/torch/_dynamo/symbolic_convert.py", line 462, in wrapper
    return inner_fn(self, inst)
  File "/home/adonnini1/anaconda3/lib/python3.9/site-packages/torch/_dynamo/symbolic_convert.py", line 1190, in CALL_FUNCTION
    self.call_function(fn, args, {})
  File "/home/adonnini1/anaconda3/lib/python3.9/site-packages/torch/_dynamo/symbolic_convert.py", line 644, in call_function
    self.push(fn.call_function(self, args, kwargs))
  File "/home/adonnini1/anaconda3/lib/python3.9/site-packages/torch/_dynamo/variables/nn_module.py", line 307, in call_function
    *proxy_args_kwargs(args, kwargs),
  File "/home/adonnini1/anaconda3/lib/python3.9/site-packages/torch/_dynamo/utils.py", line 576, in proxy_args_kwargs
    raise unimplemented(
  File "/home/adonnini1/anaconda3/lib/python3.9/site-packages/torch/_dynamo/exc.py", line 193, in unimplemented
    raise Unsupported(msg)
torch._dynamo.exc.Unsupported: call_function args: UserDefinedObjectVariable(EasyDict) 

from user code:
   File "/home/adonnini1/Development/ContextQSourceCode/NeuralNetworks/LocationPrediction/models/MHSA.py", line 36, in forward
    emb = self.Embedding(src, context_dict)
  File "/home/adonnini1/anaconda3/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/adonnini1/Development/ContextQSourceCode/NeuralNetworks/LocationPrediction/models/embed.py", line 149, in forward
    emb = self.emb_loc(src)

Set TORCH_LOGS="+dynamo" and TORCHDYNAMO_VERBOSE=1 for more information

cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @chenyang78 @kadeng @chauhang @avikchaudhuri @gmagogsfm @zhxchen17 @tugsbayasgalan @angelayi @suo @ydwu4 @aakhundov

kimishpatel commented 7 months ago

@angelayi can you help take a look?

kimishpatel commented 7 months ago

So it seems that EasyDict is not dynamo traceable.

kimishpatel commented 7 months ago

@angelayi I think EasyDIct is from here https://github.com/makinacorpus/easydict/blob/master/easydict/__init__.py, not sure why it is not traceable

adonnini commented 7 months ago

Does the fact that EasyDict is not dynamo traceable make it impossible to diagnose the problem reported in the traceback log above?

I tried using AttrDict (https://pypi.org/project/attrdict/) instead of EasyDict

Execution failed with the same error

torch._dynamo.exc.Unsupported: call_function args: UserDefinedObjectVariable(AttrDict)

adonnini commented 7 months ago

I also tried using 

class AttrDict(dict):
    def __init__(self, *args, **kwargs):
        super(AttrDict, self).__init__(*args, **kwargs)
        self.__dict__ = self

i.e. not using any external library. Execution failed with the same error

torch._dynamo.exc.Unsupported: call_function args: UserDefinedObjectVariable(AttrDict)
kimishpatel commented 7 months ago

Assigned to export

suo commented 7 months ago

pre_autograd_aten_dialect = capture_pre_autograd_graph(m, (config, config.total_loc_num, device))

Slightly tangential: curious why you are using this API to capture the graph, as it is a private API (in torch._export).

kimishpatel commented 7 months ago

pre_autograd_aten_dialect = capture_pre_autograd_graph(m, (config, config.total_loc_num, device))

Slightly tangential: curious why you are using this API to capture the graph, as it is a private API (in torch._export).

@suo so on executorch side the export flow is, atm:

  1. capture_pre_autograd
  2. to_edge.

Before export was able to produce pre-dispatch IR, for the needs to quantization we needed pre-dispatch IR. For the reason 1 was used. In order to not have two different export workflows, for quant and executorch, it was decided that we will have two step API for edge specific export. Step 1 will provide pre-dispatch IR and passes, such as quant, that need pre-dispatch IR can work on it. Later edge will apply decomps/export to produce core-aten IR. 1 was is supposed to be replaced with equivalent export API but @angelayi might know the status. Then we can start replacing 1 with the new export API

zhxchen17 commented 7 months ago

cc @tugsbayasgalan who is working on migrating capture_pre_autograd_graph to torch.export.

avikchaudhuri commented 7 months ago

@yanboliang offered to look into the the dynamo coverage issue. Meanwhile @kimishpatel any chance you can use non-strict (i.e., strict=False) to unblock?

kimishpatel commented 7 months ago

@avikchaudhuri it is coming from call to export from capture_pre_auto_grad. does it support nonstrict? @adonnini can you try that?

adonnini commented 7 months ago

@kimishpatel I added strict=False to export(pre_autograd_aten_dialect, (enc_input, dec_input, dec_source_mask, dec_target_mask)) When I ran the code, it failed at this line lowered_module = edge_program.to_backend(XnnpackPartitioner()) Telling me it could not find flatc

Then, I had the bright idea (not really) to update executorch installation to the latest nightly build (20240209). Now, execution fails with a brand new error. Please see traceback log below. I searched for information on this error. I did not come up with anything I found useful, not to say that there wasn't relevant information. I just could not decipher whether it was applicable to my case

TRACEBACK LOG

Traceback (most recent call last):
  File "/home/adonnini1/Development/ContextQSourceCode/NeuralNetworks/trajectory-prediction-transformers-master/train-minimum.py", line 21, in <module>
    from executorch.backends.xnnpack.partition.xnnpack_partitioner import XnnpackPartitioner
  File "/home/adonnini1/Development/ContextQSourceCode/NeuralNetworks/trajectory-prediction-transformers-master/executorch/backends/xnnpack/__init__.py", line 8, in <module>
    from .partition.xnnpack_partitioner import (
  File "/home/adonnini1/Development/ContextQSourceCode/NeuralNetworks/trajectory-prediction-transformers-master/executorch/backends/xnnpack/partition/xnnpack_partitioner.py", line 82, in <module>
    class XnnpackOperatorSupport(OperatorSupportBase):
  File "/home/adonnini1/Development/ContextQSourceCode/NeuralNetworks/trajectory-prediction-transformers-master/executorch/backends/xnnpack/partition/xnnpack_partitioner.py", line 236, in XnnpackOperatorSupport
    @_constraint(exir_ops.edge.aten.mean.dim)
TypeError: 'staticmethod' object is not callable
kimishpatel commented 7 months ago

@adonnini so at least with strict=False you are able to get around the export issue and fails elsewhere, right? @mcr229 for xnnpack

adonnini commented 7 months ago

I hope that is the case. It should be. Should I wait to hear from @mcr229 with regards to this latest error? Please let me know if there is anything I can/should do. Thanks

kimishpatel commented 7 months ago

Sorry to ask you to create another issue, but we can go back to executorch repo and create issue there and mark this as "unblocked" since we dont have proper solution but have a workaround. If you dont mind doing that, tag @mcr229 on the issue

adonnini commented 7 months ago

No problem at all. I created a new issue in executorch repo, https://github.com/pytorch/executorch/issues/2025

adonnini commented 6 months ago

Hi,

I cloned the main executorch branch (git clone --branch main https://github.com/pytorch/executorch.git).

Ran executorch installation/set-up following the updated instructions in https://pytorch.org/executorch/stable/getting-started-setup.html

I set strict=False in aten_dialect: ExportedProgram = export(pre_autograd_aten_dialect, (config, config.total_loc_num, device), strict=False)

then ran the training module for my model including the executorch code.

Execution failed once again with NotImplementedError: UserDefinedObjectVariable(EasyDict)

Below you will find the tracebak log

Please let me know what I should do next to help resolve this problem.

Thanks

TRACEBACK LOG

Traceback (most recent call last):
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/_dynamo/utils.py", line 621, in proxy_args_kwargs
    proxy_args = tuple(arg.as_proxy() for arg in args)
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/_dynamo/utils.py", line 621, in <genexpr>
    proxy_args = tuple(arg.as_proxy() for arg in args)
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/_dynamo/variables/base.py", line 304, in as_proxy
    raise NotImplementedError(str(self))
NotImplementedError: UserDefinedObjectVariable(EasyDict)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/adonnini1/Development/ContextQSourceCode/NeuralNetworks/LocationPrediction/main.py", line 68, in <module>
    res_single = single_run(train_loader, val_loader, test_loader, config, device, log_dir)
  File "/home/adonnini1/Development/ContextQSourceCode/NeuralNetworks/LocationPrediction/main.py", line 23, in single_run
    model, perf = get_trainedNets(config, model, train_loader, val_loader, device, log_dir)
  File "/home/adonnini1/Development/ContextQSourceCode/NeuralNetworks/LocationPrediction/utils/utils.py", line 47, in get_trainedNets
    best_model, performance = trainNet(config, model, train_loader, val_loader, device, log_dir=log_dir)
  File "/home/adonnini1/Development/ContextQSourceCode/NeuralNetworks/LocationPrediction/utils/train.py", line 316, in trainNet
    pre_autograd_aten_dialect = capture_pre_autograd_graph(m, (config, config.total_loc_num, device))
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/_export/__init__.py", line 168, in capture_pre_autograd_graph
    m = torch._dynamo.export(
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/_dynamo/eval_frame.py", line 1296, in inner
    result_traced = opt_f(*args, **kwargs)
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1527, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1536, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/_dynamo/eval_frame.py", line 437, in _fn
    return fn(*args, **kwargs)
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1527, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1536, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 919, in catch_errors
    return callback(frame, cache_entry, hooks, frame_state, skip=1)
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 400, in _convert_frame_assert
    return _compile(
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/contextlib.py", line 79, in inner
    return func(*args, **kwds)
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 675, in _compile
    guarded_code = compile_inner(code, one_graph, hooks, transform)
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/_dynamo/utils.py", line 262, in time_wrapper
    r = func(*args, **kwargs)
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 534, in compile_inner
    out_code = transform_code_object(code, transform)
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/_dynamo/bytecode_transformation.py", line 1033, in transform_code_object
    transformations(instructions, code_options)
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 165, in _fn
    return fn(*args, **kwargs)
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 499, in transform
    tracer.run()
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 2175, in run
    super().run()
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 831, in run
    and self.step()
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 794, in step
    getattr(self, inst.opname)(inst)
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 470, in wrapper
    return inner_fn(self, inst)
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 1245, in CALL_FUNCTION
    self.call_function(fn, args, {})
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 695, in call_function
    self.push(fn.call_function(self, args, kwargs))
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/_dynamo/variables/nn_module.py", line 336, in call_function
    return tx.inline_user_function_return(
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 701, in inline_user_function_return
    return InliningInstructionTranslator.inline_call(self, fn, args, kwargs)
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 2311, in inline_call
    return cls.inline_call_(parent, func, args, kwargs)
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 2425, in inline_call_
    tracer.run()
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 831, in run
    and self.step()
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 794, in step
    getattr(self, inst.opname)(inst)
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 470, in wrapper
    return inner_fn(self, inst)
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 1286, in CALL_FUNCTION_EX
    self.call_function(fn, argsvars.items, kwargsvars)
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 695, in call_function
    self.push(fn.call_function(self, args, kwargs))
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/_dynamo/variables/functions.py", line 334, in call_function
    return super().call_function(tx, args, kwargs)
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/_dynamo/variables/functions.py", line 288, in call_function
    return super().call_function(tx, args, kwargs)
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/_dynamo/variables/functions.py", line 89, in call_function
    return tx.inline_user_function_return(
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 701, in inline_user_function_return
    return InliningInstructionTranslator.inline_call(self, fn, args, kwargs)
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 2311, in inline_call
    return cls.inline_call_(parent, func, args, kwargs)
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 2425, in inline_call_
    tracer.run()
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 831, in run
    and self.step()
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 794, in step
    getattr(self, inst.opname)(inst)
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 470, in wrapper
    return inner_fn(self, inst)
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 1245, in CALL_FUNCTION
    self.call_function(fn, args, {})
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 695, in call_function
    self.push(fn.call_function(self, args, kwargs))
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/_dynamo/variables/nn_module.py", line 314, in call_function
    *proxy_args_kwargs(args, kwargs),
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/_dynamo/utils.py", line 628, in proxy_args_kwargs
    raise unimplemented(
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/_dynamo/exc.py", line 190, in unimplemented
    raise Unsupported(msg)
torch._dynamo.exc.Unsupported: call_function args: UserDefinedObjectVariable(EasyDict) 

from user code:
   File "/home/adonnini1/Development/ContextQSourceCode/NeuralNetworks/LocationPrediction/models/MHSA.py", line 36, in forward
    emb = self.Embedding(src, context_dict)
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1536, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/adonnini1/Development/ContextQSourceCode/NeuralNetworks/LocationPrediction/models/embed.py", line 149, in forward
    emb = self.emb_loc(src)
adonnini commented 6 months ago

@kimishpatel

After the failure described above with the executorch main branch installed, I reverted back to v0.1.0 and completed the steps in https://pytorch.org/executorch/stable/getting-started-setup.html The buck2 build failed as described in https://github.com/pytorch/executorch/issues/2198 Next, I ran

rm -rf build

git pull --rebase --no-recurse-submodules -- FAILED
git pull
git submodule sync && git submodule update --init --recursive --depth 1 --jobs 0 -- FAILED
git submodule sync && git submodule update --init --recursive
pip install --force-reinstall --pre torch=="2.3.0.dev20240209" -i https://download.pytorch.org/whl/nightly/cpu
 ./install_requirements.sh
  export PATH="/home/adonnini1/Development/ContextQSourceCode/NeuralNetworks/trajectory-prediction-transformers-master/executorch/third-party/flatbuffers/cmake-out:${PATH}"
  bash /home/adonnini1/Development/ContextQSourceCode/NeuralNetworks/trajectory-prediction-transformers-master/executorch/build/install_flatc.sh

Please note that pip install --force-reinstall --pre torch=="2.3.0.dev20240209" -i https://download.pytorch.org/whl/nightly/cpu completed the installation of torch-2.3.0.dev20240209+cpu but had the following error:

ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
torchaudio 2.2.1 requires torch==2.2.1, but you have torch 2.3.0.dev20240209+cpu which is incompatible.
torchvision 0.17.1 requires torch==2.2.1, but you have torch 2.3.0.dev20240209+cpu which is incompatible.

Next, I ran the training module for the model including the executorch code (with strict=False)

Execution failed producing the traceback log reported below.

TRACEBACK LOG

Traceback (most recent call last):
  File "/home/adonnini1/Development/ContextQSourceCode/NeuralNetworks/LocationPrediction/main.py", line 68, in <module>
    res_single = single_run(train_loader, val_loader, test_loader, config, device, log_dir)
  File "/home/adonnini1/Development/ContextQSourceCode/NeuralNetworks/LocationPrediction/main.py", line 23, in single_run
    model, perf = get_trainedNets(config, model, train_loader, val_loader, device, log_dir)
  File "/home/adonnini1/Development/ContextQSourceCode/NeuralNetworks/LocationPrediction/utils/utils.py", line 47, in get_trainedNets
    best_model, performance = trainNet(config, model, train_loader, val_loader, device, log_dir=log_dir)
  File "/home/adonnini1/Development/ContextQSourceCode/NeuralNetworks/LocationPrediction/utils/train.py", line 211, in trainNet
    from executorch.backends.xnnpack.partition.xnnpack_partitioner import XnnpackPartitioner
  File "/home/adonnini1/Development/ContextQSourceCode/NeuralNetworks/trajectory-prediction-transformers-master/executorch/backends/xnnpack/__init__.py", line 8, in <module>
    from .partition.xnnpack_partitioner import (
  File "/home/adonnini1/Development/ContextQSourceCode/NeuralNetworks/trajectory-prediction-transformers-master/executorch/backends/xnnpack/partition/xnnpack_partitioner.py", line 25, in <module>
    from executorch.backends.xnnpack.passes.fuse_batch_norm_with_conv import (
  File "/home/adonnini1/Development/ContextQSourceCode/NeuralNetworks/trajectory-prediction-transformers-master/executorch/backends/xnnpack/passes/__init__.py", line 9, in <module>
    from executorch.backends.xnnpack.passes.channels_last_tagged_reshape_pass import (
  File "/home/adonnini1/Development/ContextQSourceCode/NeuralNetworks/trajectory-prediction-transformers-master/executorch/backends/xnnpack/passes/channels_last_tagged_reshape_pass.py", line 11, in <module>
    from executorch.backends.xnnpack.utils.utils import is_param_node
  File "/home/adonnini1/Development/ContextQSourceCode/NeuralNetworks/trajectory-prediction-transformers-master/executorch/backends/xnnpack/utils/utils.py", line 20, in <module>
    from torch._export.utils import (
ImportError: cannot import name 'get_lifted_tensor_constant' from 'torch._export.utils' (/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/_export/utils.py)
kimishpatel commented 6 months ago

When you say reverted to 0.1.0, I presume you are working with older branch and not master? @mcr229 if you know anything aobut xnnpack partitionre

adonnini commented 6 months ago

@kimishpatel yes I mean this https://github.com/pytorch/executorch/releases/tag/v0.1.0

kimishpatel commented 6 months ago

@tugsbayasgalan

adonnini commented 6 months ago

@tugsbayasgalan Hi, did you get a chance to take a look at this issue? Is there anything I can do to help resolve it? Please let me know if you need any information from me. Thanks

adonnini commented 6 months ago

@angelayi @jansel I installed executorch from the main branch. I am still getting this error. Please find below the latest traceback log. Please let me know if there is anything I can do to solve this problem or if you need any additional information. Thanks

TRACEBACK LOG

Traceback (most recent call last):
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/_dynamo/utils.py", line 624, in proxy_args_kwargs
    proxy_args = tuple(arg.as_proxy() for arg in args)
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/_dynamo/utils.py", line 624, in <genexpr>
    proxy_args = tuple(arg.as_proxy() for arg in args)
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/_dynamo/variables/base.py", line 246, in as_proxy
    raise NotImplementedError(str(self))
NotImplementedError: UserDefinedObjectVariable(EasyDict)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/adonnini1/Development/ContextQSourceCode/NeuralNetworks/LocationPrediction/main.py", line 68, in <module>
    res_single = single_run(train_loader, val_loader, test_loader, config, device, log_dir)
  File "/home/adonnini1/Development/ContextQSourceCode/NeuralNetworks/LocationPrediction/main.py", line 23, in single_run
    model, perf = get_trainedNets(config, model, train_loader, val_loader, device, log_dir)
  File "/home/adonnini1/Development/ContextQSourceCode/NeuralNetworks/LocationPrediction/utils/utils.py", line 47, in get_trainedNets
    best_model, performance = trainNet(config, model, train_loader, val_loader, device, log_dir=log_dir)
  File "/home/adonnini1/Development/ContextQSourceCode/NeuralNetworks/LocationPrediction/utils/train.py", line 316, in trainNet
    pre_autograd_aten_dialect = capture_pre_autograd_graph(m, (config, config.total_loc_num, device))
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/_export/__init__.py", line 151, in capture_pre_autograd_graph
    m = torch._dynamo.export(
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/_dynamo/eval_frame.py", line 1232, in inner
    result_traced = opt_f(*args, **kwargs)
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1527, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1536, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/_dynamo/eval_frame.py", line 390, in _fn
    return fn(*args, **kwargs)
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1527, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1536, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 939, in catch_errors
    return callback(frame, cache_entry, hooks, frame_state, skip=1)
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 400, in _convert_frame_assert
    return _compile(
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/contextlib.py", line 79, in inner
    return func(*args, **kwds)
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 686, in _compile
    guarded_code = compile_inner(code, one_graph, hooks, transform)
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/_dynamo/utils.py", line 265, in time_wrapper
    r = func(*args, **kwargs)
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 541, in compile_inner
    out_code = transform_code_object(code, transform)
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/_dynamo/bytecode_transformation.py", line 1036, in transform_code_object
    transformations(instructions, code_options)
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 165, in _fn
    return fn(*args, **kwargs)
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 503, in transform
    tracer.run()
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 2106, in run
    super().run()
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 829, in run
    while self.step():
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 743, in step
    self.dispatch_table[inst.opcode](self, inst)
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 471, in wrapper
    return inner_fn(self, inst)
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 1192, in CALL_FUNCTION
    self.call_function(fn, args, {})
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 683, in call_function
    self.push(fn.call_function(self, args, kwargs))
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/_dynamo/variables/nn_module.py", line 341, in call_function
    return tx.inline_user_function_return(
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 689, in inline_user_function_return
    return InliningInstructionTranslator.inline_call(self, fn, args, kwargs)
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 2253, in inline_call
    return cls.inline_call_(parent, func, args, kwargs)
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 2367, in inline_call_
    tracer.run()
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 829, in run
    while self.step():
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 743, in step
    self.dispatch_table[inst.opcode](self, inst)
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 471, in wrapper
    return inner_fn(self, inst)
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 1233, in CALL_FUNCTION_EX
    self.call_function(fn, argsvars.items, kwargsvars)
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 683, in call_function
    self.push(fn.call_function(self, args, kwargs))
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/_dynamo/variables/functions.py", line 342, in call_function
    return super().call_function(tx, args, kwargs)
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/_dynamo/variables/functions.py", line 296, in call_function
    return super().call_function(tx, args, kwargs)
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/_dynamo/variables/functions.py", line 91, in call_function
    return tx.inline_user_function_return(
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 689, in inline_user_function_return
    return InliningInstructionTranslator.inline_call(self, fn, args, kwargs)
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 2253, in inline_call
    return cls.inline_call_(parent, func, args, kwargs)
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 2367, in inline_call_
    tracer.run()
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 829, in run
    while self.step():
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 743, in step
    self.dispatch_table[inst.opcode](self, inst)
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 471, in wrapper
    return inner_fn(self, inst)
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 1192, in CALL_FUNCTION
    self.call_function(fn, args, {})
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 683, in call_function
    self.push(fn.call_function(self, args, kwargs))
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/_dynamo/variables/nn_module.py", line 319, in call_function
    *proxy_args_kwargs(args, kwargs),
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/_dynamo/utils.py", line 631, in proxy_args_kwargs
    unimplemented(
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/_dynamo/exc.py", line 198, in unimplemented
    raise Unsupported(msg) from from_exc
torch._dynamo.exc.Unsupported: call_function args: UserDefinedObjectVariable(EasyDict) 

from user code:
   File "/home/adonnini1/Development/ContextQSourceCode/NeuralNetworks/LocationPrediction/models/MHSA.py", line 36, in forward
    emb = self.Embedding(src, context_dict)
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1536, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/adonnini1/Development/ContextQSourceCode/NeuralNetworks/LocationPrediction/models/embed.py", line 149, in forward
    emb = self.emb_loc(src)
jansel commented 6 months ago

https://github.com/pytorch/pytorch/pull/123318 might fix the issue with EasyDict

adonnini commented 6 months ago

@jansel Thanks. Which branch does https://github.com/pytorch/pytorch/pull/123318 refer to (sorry about this simple question)? pytorch main branch? Thanks

jansel commented 6 months ago

123318 hasn't been merged yet, so you will need to build PyTorch from source to get it and cherry-pick that PR.

Assuming it passes the tests I plan to merge it in the next day or two.

adonnini commented 6 months ago

Thanks. I appreciate your quick response. I have enough on my plate. I think I will wait until the change has been merged in to the main pytorch branch. I'll check back in towards the middle of the week.

When it's ready I can simply download and use the nightly build right?

If you have time, please let me know if anything changes. Thanks

jansel commented 6 months ago

Yes a few days after it is merged it will be in the nightly build

adonnini commented 5 months ago

@jansel Sorry to bother you. I just checked the latest nightly build available is from 04/08.It does not include https://github.com/pytorch/pytorch/pull/123318 right? Thanks

jansel commented 5 months ago

Not sure, you can do python -c "import torch; print(torch.version.git_version)" then looking that version on github to see if it is before or after #123318.

adonnini commented 5 months ago

If I go to https://hud.pytorch.org/hud/pytorch/pytorch/nightly and search for your commit using the following SHA 36b36ab64a09297fa0f0d26199603d09eadea2e4 I don't see any results. What am I doing wrong? Am I using the wrong SHA (what's the right one?)? Am I looking in the wrong place?

if I go to https://hud.pytorch.org/pytorch/pytorch/commit/36b36ab64a09297fa0f0d26199603d09eadea2e4 it shows me details about testing of the commit but no info regarding a nightly build that I could see.

if instead I go to https://hud.pytorch.org/pytorch/pytorch/commit/212e460dcea1777edd57de3cb47d8fac7d35635a which is the link for https://github.com/pytorch/pytorch/pull/123318 in https://hud.pytorch.org/hud/pytorch/pytorch/main/4?per_page=500&name_filter= I think it tells me that commit testing failed

What's the github url for the nightly build?

BTW, when I run a small utility to find out the latest pytorch nightly build it reports https://download.pytorch.org/whl/nightly/cu121/torch-2.4.0.dev20240408%2Bcu121-cp39-cp39-linux_x86_64.whl

jansel commented 5 months ago

The latest nightly build I see is 2024-04-15, which should definitely have it.

adonnini commented 5 months ago

Got it. Thanks

adonnini commented 5 months ago

@angelayi @jansel

Unfortunately, execution of my code still fails with NotImplementedError: UserDefinedObjectVariable(EasyDict)

I reinstalled executorch from main branch and installed the latest pytorch nightly build (confirmed by running pip list) which contains the fix to https://github.com/pytorch/pytorch/pull/123318

In case it is of any use and/or interest to you below you will find the complete traceback log.

Please let me know if you need additional information.

Thanks,

Alex

TRACEBACK LOG

Traceback (most recent call last):
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/_dynamo/utils.py", line 634, in proxy_args_kwargs
    proxy_args = tuple(arg.as_proxy() for arg in args)
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/_dynamo/utils.py", line 634, in <genexpr>
    proxy_args = tuple(arg.as_proxy() for arg in args)
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/_dynamo/variables/base.py", line 246, in as_proxy
    raise NotImplementedError(str(self))
NotImplementedError: UserDefinedObjectVariable(EasyDict)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/adonnini1/Development/ContextQSourceCode/NeuralNetworks/LocationPrediction/main.py", line 68, in <module>
    res_single = single_run(train_loader, val_loader, test_loader, config, device, log_dir)
  File "/home/adonnini1/Development/ContextQSourceCode/NeuralNetworks/LocationPrediction/main.py", line 23, in single_run
    model, perf = get_trainedNets(config, model, train_loader, val_loader, device, log_dir)
  File "/home/adonnini1/Development/ContextQSourceCode/NeuralNetworks/LocationPrediction/utils/utils.py", line 47, in get_trainedNets
    best_model, performance = trainNet(config, model, train_loader, val_loader, device, log_dir=log_dir)
  File "/home/adonnini1/Development/ContextQSourceCode/NeuralNetworks/LocationPrediction/utils/train.py", line 316, in trainNet
    pre_autograd_aten_dialect = capture_pre_autograd_graph(m, (config, config.total_loc_num, device))
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/_export/__init__.py", line 151, in capture_pre_autograd_graph
    m = torch._dynamo.export(
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/_dynamo/eval_frame.py", line 1237, in inner
    result_traced = opt_f(*args, **kwargs)
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1532, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1541, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/_dynamo/eval_frame.py", line 410, in _fn
    return fn(*args, **kwargs)
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1532, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1541, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 978, in catch_errors
    return callback(frame, cache_entry, hooks, frame_state, skip=1)
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 411, in _convert_frame_assert
    return _compile(
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/_utils_internal.py", line 70, in wrapper_function
    return function(*args, **kwargs)
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/contextlib.py", line 79, in inner
    return func(*args, **kwds)
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 700, in _compile
    guarded_code = compile_inner(code, one_graph, hooks, transform)
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/_dynamo/utils.py", line 266, in time_wrapper
    r = func(*args, **kwargs)
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 568, in compile_inner
    out_code = transform_code_object(code, transform)
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/_dynamo/bytecode_transformation.py", line 1116, in transform_code_object
    transformations(instructions, code_options)
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 173, in _fn
    return fn(*args, **kwargs)
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 515, in transform
    tracer.run()
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 2237, in run
    super().run()
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 875, in run
    while self.step():
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 790, in step
    self.dispatch_table[inst.opcode](self, inst)
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 492, in wrapper
    return inner_fn(self, inst)
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 1260, in CALL_FUNCTION
    self.call_function(fn, args, {})
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 730, in call_function
    self.push(fn.call_function(self, args, kwargs))
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/_dynamo/variables/nn_module.py", line 351, in call_function
    return tx.inline_user_function_return(
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 736, in inline_user_function_return
    return InliningInstructionTranslator.inline_call(self, fn, args, kwargs)
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 2404, in inline_call
    return cls.inline_call_(parent, func, args, kwargs)
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 2520, in inline_call_
    tracer.run()
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 875, in run
    while self.step():
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 790, in step
    self.dispatch_table[inst.opcode](self, inst)
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 492, in wrapper
    return inner_fn(self, inst)
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 1301, in CALL_FUNCTION_EX
    self.call_function(fn, argsvars.items, kwargsvars)
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 730, in call_function
    self.push(fn.call_function(self, args, kwargs))
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/_dynamo/variables/functions.py", line 339, in call_function
    return super().call_function(tx, args, kwargs)
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/_dynamo/variables/functions.py", line 293, in call_function
    return super().call_function(tx, args, kwargs)
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/_dynamo/variables/functions.py", line 90, in call_function
    return tx.inline_user_function_return(self, [*self.self_args(), *args], kwargs)
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 736, in inline_user_function_return
    return InliningInstructionTranslator.inline_call(self, fn, args, kwargs)
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 2404, in inline_call
    return cls.inline_call_(parent, func, args, kwargs)
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 2520, in inline_call_
    tracer.run()
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 875, in run
    while self.step():
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 790, in step
    self.dispatch_table[inst.opcode](self, inst)
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 492, in wrapper
    return inner_fn(self, inst)
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 1260, in CALL_FUNCTION
    self.call_function(fn, args, {})
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 730, in call_function
    self.push(fn.call_function(self, args, kwargs))
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/_dynamo/variables/nn_module.py", line 329, in call_function
    *proxy_args_kwargs(args, kwargs),
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/_dynamo/utils.py", line 641, in proxy_args_kwargs
    unimplemented(
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/_dynamo/exc.py", line 211, in unimplemented
    raise Unsupported(msg) from from_exc
torch._dynamo.exc.Unsupported: call_function args: UserDefinedObjectVariable(EasyDict) 

from user code:
   File "/home/adonnini1/Development/ContextQSourceCode/NeuralNetworks/LocationPrediction/models/MHSA.py", line 36, in forward
    emb = self.Embedding(src, context_dict)
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1541, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/adonnini1/Development/ContextQSourceCode/NeuralNetworks/LocationPrediction/models/embed.py", line 149, in forward
    emb = self.emb_loc(src)
kimishpatel commented 5 months ago

@tugsbayasgalan @avikchaudhuri can you guys help take a look?

adonnini commented 5 months ago

@tugsbayasgalan @avikchaudhuri I hope I am not bothering you. When do you think you could let me know about potential resolution for this problem? I am stuck waiting for its resolution. Please let me know if there is anything I can do to help. Thanks

angelayi commented 5 months ago

@adonnini Can you try replacing your capture_pre_autograd_graph call with torch.export._trace._export(..., strict=False, pre_dispatch=True)? And then replace your export call with export(..., strict=False)?

Is there a particular reason against using strict=False?

adonnini commented 5 months ago

@angelayi I made the changes you suggested (I already had strict=False in the export call). Execution fails producing the traceback log reported below. Below, you will also find the code I use to produce the lowered model. Could this problem be cause by the version of executorch I am running (executorch-0.1.0)? Pease let me know what I should do next. Thanks

TRACEBACK LOG

Traceback (most recent call last):
  File "/home/adonnini1/Development/ContextQSourceCode/NeuralNetworks/LocationPrediction/main.py", line 68, in <module>
    res_single = single_run(train_loader, val_loader, test_loader, config, device, log_dir)
  File "/home/adonnini1/Development/ContextQSourceCode/NeuralNetworks/LocationPrediction/main.py", line 23, in single_run
    model, perf = get_trainedNets(config, model, train_loader, val_loader, device, log_dir)
  File "/home/adonnini1/Development/ContextQSourceCode/NeuralNetworks/LocationPrediction/utils/utils.py", line 47, in get_trainedNets
    best_model, performance = trainNet(config, model, train_loader, val_loader, device, log_dir=log_dir)
  File "/home/adonnini1/Development/ContextQSourceCode/NeuralNetworks/LocationPrediction/utils/train.py", line 316, in trainNet
    pre_autograd_aten_dialect = torch.export._trace._export(m, (config, config.total_loc_num, device), strict=False, pre_dispatch=True)
AttributeError: module 'torch.export' has no attribute '_trace'

CODE

        pre_autograd_aten_dialect = torch.export._trace._export(m, (config, config.total_loc_num, device), strict=False, pre_dispatch=True)
        aten_dialect: ExportedProgram = export(pre_autograd_aten_dialect, (config, config.total_loc_num, device), strict=False)
        edge_program: EdgeProgramManager = to_edge(aten_dialect)
        to_be_lowered_module = edge_program.exported_program()

        from executorch.exir.backend.backend_api import LoweredBackendModule, to_backend

        lowered_module = edge_program.to_backend(XnnpackPartitioner())

        print(" - train_minimum - Lowering the Whole Module - lowered_module - ", lowered_module)

        save_path = save_path = "/home/adonnini1/Development/ContextQSourceCode/NeuralNetworks/LocationPrediction/loweredModels/tpt_delegate.pte"
        with open(save_path, "wb") as f:
            f.write(lowered_module.to_executorch().buffer)
adonnini commented 5 months ago

@angelayi I ran the code again after installing the latest executorch (executorch-0.3.0a0+6f0b8e0). It failed with the same error. Below ypou will find the traceback log. The version of pytorch is 2.4.0.dev20240422+cpu

TRACEBACK LOG

Traceback (most recent call last):
  File "/home/adonnini1/Development/ContextQSourceCode/NeuralNetworks/LocationPrediction/main.py", line 68, in <module>
    res_single = single_run(train_loader, val_loader, test_loader, config, device, log_dir)
  File "/home/adonnini1/Development/ContextQSourceCode/NeuralNetworks/LocationPrediction/main.py", line 23, in single_run
    model, perf = get_trainedNets(config, model, train_loader, val_loader, device, log_dir)
  File "/home/adonnini1/Development/ContextQSourceCode/NeuralNetworks/LocationPrediction/utils/utils.py", line 47, in get_trainedNets
    best_model, performance = trainNet(config, model, train_loader, val_loader, device, log_dir=log_dir)
  File "/home/adonnini1/Development/ContextQSourceCode/NeuralNetworks/LocationPrediction/utils/train.py", line 316, in trainNet
    pre_autograd_aten_dialect = torch.export._trace._export(m, (config, config.total_loc_num, device), strict=False, pre_dispatch=True)
AttributeError: module 'torch.export' has no attribute '_trace'
angelayi commented 5 months ago

You might need to manually import torch.export._trace

adonnini commented 5 months ago

@angelayi sorry I should have thought of that. I added the import statement. Execution fails producing the traceback log reported below. As you will see the error is similar but not quite the same as originally reported in this issue. Please let me know what I should do next and if you need any information. Thanks

TRACEBACK LOG

Traceback (most recent call last):
  File "/home/adonnini1/Development/ContextQSourceCode/NeuralNetworks/LocationPrediction/main.py", line 68, in <module>
    res_single = single_run(train_loader, val_loader, test_loader, config, device, log_dir)
  File "/home/adonnini1/Development/ContextQSourceCode/NeuralNetworks/LocationPrediction/main.py", line 23, in single_run
    model, perf = get_trainedNets(config, model, train_loader, val_loader, device, log_dir)
  File "/home/adonnini1/Development/ContextQSourceCode/NeuralNetworks/LocationPrediction/utils/utils.py", line 47, in get_trainedNets
    best_model, performance = trainNet(config, model, train_loader, val_loader, device, log_dir=log_dir)
  File "/home/adonnini1/Development/ContextQSourceCode/NeuralNetworks/LocationPrediction/utils/train.py", line 318, in trainNet
    pre_autograd_aten_dialect = torch.export._trace._export(m, (config, config.total_loc_num, device), strict=False, pre_dispatch=True)
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/export/_trace.py", line 833, in wrapper
    raise e
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/export/_trace.py", line 816, in wrapper
    ep = fn(*args, **kwargs)
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/export/exported_program.py", line 85, in wrapper
    return fn(*args, **kwargs)
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/export/_trace.py", line 988, in _export
    ) = make_fake_inputs(mod, args, kwargs, dynamic_shapes)
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/_export/non_strict_utils.py", line 148, in make_fake_inputs
    fake_args, fake_kwargs = tree_map_with_path(
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/utils/_pytree.py", line 1581, in tree_map_with_path
    return treespec.unflatten(func(*xs) for xs in zip(*all_keypath_leaves))
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/utils/_pytree.py", line 782, in unflatten
    leaves = list(leaves)
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/utils/_pytree.py", line 1581, in <genexpr>
    return treespec.unflatten(func(*xs) for xs in zip(*all_keypath_leaves))
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/_export/non_strict_utils.py", line 149, in <lambda>
    lambda kp, val: fakify(fake_mode, kp, val, t_constraints, sources),
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/_export/non_strict_utils.py", line 72, in fakify
    raise ValueError(f"Unsupported input type {type(t)}")
ValueError: Unsupported input type <class 'easydict.EasyDict'>
angelayi commented 5 months ago

@adonnini I think you have to register the EasyDict as a pytree node with register_pytree_node. More docs on pytree here. Please let me know if anything is unclear!

adonnini commented 5 months ago

@angelayi Thanks. I'll do that ASAP when I get back to the office later today. I already took a quick look at the documentation and code for https://github.com/pytorch/pytorch/blob/394ec2da300e8933d8184ba862daed3a115d7fd9/torch/utils/_pytree.py#L163 a call to register_pytree_node requires that I provide flatten and unflatten functions. Frankly, I have no idea where to find flatten/unflatten functions to use in this instance. What should I use? Should I simply use flatten() and unflatten()? I should call register_pytree_node in the code just before the start of the executorch related call right? Thanks

adonnini commented 5 months ago

@angelayi I made an initial attempt at following your latest suggestion. I installed flatten-dict (https://github.com/ianlini/flatten-dict) pip install flatten-dict

I added these lines to my code:

from flatten_dict import flatten
from flatten_dict import unflatten
torch.utils._pytree.register_pytree_node(edict, flatten, unflatten)

where edict is EasyDict I added the above lines just before the start of the executorch related code.

Code execution failed producing the traceback log reported below.

This might be helpful

        config = edict(config)
        print("LocationPrediction - utils - train.py - config - 1 - ",config)
LocationPrediction - utils - train.py - config - 1 -  {'dataset': 'geolife', 'total_loc_num': 3, 'total_user_num': 2, 'if_embed_user': False, 'if_embed_poi': False, 'if_embed_time': True, 'if_embed_duration': True, 'previous_day': 30, 'verbose': True, 'debug': False, 'batch_size': 2, 'print_step': 20, 'num_workers': 0, 'day_selection': 'default', 'base_emb_size': 4, 'poi_original_size': 2, 'networkName': 'transformer', 'num_encoder_layers': 2, 'nhead': 2, 'dim_feedforward': 16, 'fc_dropout': 0.2, 'optimizer': 'Adam', 'max_epoch': 100, 'lr': 0.001, 'weight_decay': 1e-06, 'beta1': 0.9, 'beta2': 0.999, 'momentum': 0.98, 'num_warmup_epochs': 2, 'num_training_epochs': 50, 'patience': 3, 'lr_step_size': 1, 'lr_gamma': 0.1, 'source_root': './data/', 'save_root': './outputs/'}

Was the problem caused by the flatten/unflatten functions I used? I did some searching for flatten/unflatten functions. It looks like in many instances custom ones need to be created.

What should I do next?

Thanks

TRACEBACK LOG

Traceback (most recent call last):
  File "/home/adonnini1/Development/ContextQSourceCode/NeuralNetworks/LocationPrediction/main.py", line 68, in <module>
    res_single = single_run(train_loader, val_loader, test_loader, config, device, log_dir)
  File "/home/adonnini1/Development/ContextQSourceCode/NeuralNetworks/LocationPrediction/main.py", line 23, in single_run
    model, perf = get_trainedNets(config, model, train_loader, val_loader, device, log_dir)
  File "/home/adonnini1/Development/ContextQSourceCode/NeuralNetworks/LocationPrediction/utils/utils.py", line 47, in get_trainedNets
    best_model, performance = trainNet(config, model, train_loader, val_loader, device, log_dir=log_dir)
  File "/home/adonnini1/Development/ContextQSourceCode/NeuralNetworks/LocationPrediction/utils/train.py", line 320, in trainNet
    pre_autograd_aten_dialect = torch.export._trace._export(m, (config, config.total_loc_num, device), strict=False, pre_dispatch=True)
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/export/_trace.py", line 833, in wrapper
    raise e
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/export/_trace.py", line 816, in wrapper
    ep = fn(*args, **kwargs)
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/export/exported_program.py", line 85, in wrapper
    return fn(*args, **kwargs)
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/export/_trace.py", line 909, in _export
    flat_args, orig_in_spec = pytree.tree_flatten((args, kwargs))
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/utils/_pytree.py", line 853, in tree_flatten
    spec = _tree_flatten_helper(tree, leaves, is_leaf=is_leaf)
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/utils/_pytree.py", line 838, in _tree_flatten_helper
    children_specs = [
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/utils/_pytree.py", line 839, in <listcomp>
    _tree_flatten_helper(child, leaves, is_leaf=is_leaf) for child in child_pytrees
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/utils/_pytree.py", line 838, in _tree_flatten_helper
    children_specs = [
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/utils/_pytree.py", line 839, in <listcomp>
    _tree_flatten_helper(child, leaves, is_leaf=is_leaf) for child in child_pytrees
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/utils/_pytree.py", line 835, in _tree_flatten_helper
    child_pytrees, context = flatten_fn(tree)
ValueError: too many values to unpack (expected 2)
angelayi commented 5 months ago

@adonnini you should probably rewrite the flatten/unflatten functions yourself, based on what is stored inside of the EasyDict. You can follow the example for generic dictionaries: https://github.com/pytorch/pytorch/blob/394ec2da300e8933d8184ba862daed3a115d7fd9/torch/utils/_pytree.py#L421-L433

adonnini commented 5 months ago

@angelayi writing a custom flatten and unflatten for EasyDict is harder than I was hoping. Coming from Java, I am not familiar with Python/Pytorch. I just hope this is not another fruitless attempt at resolving this problem. On the surface, this should not be too complicated to resolve. Why does EasyDict cause https://github.com/pytorch/pytorch/issues/120219#top? Why is EasyDict considered a user defined function? Are all external libraries considered to be user defined functions? Forgive me. I am just a bit frustrated feeling bounced around from one blind alley to another. Thanks for your patience.

adonnini commented 5 months ago

@angelayi As I am still working on flatten/unflatten, I tried a different approach. I defined this class:

class AttrDict(dict):

    def __init__(self, *args, **kwargs):

        super(AttrDict, self).__init__(*args, **kwargs)

        self.__dict__ = self

and used it to create the dictionary. It does do that successfully (config = AttrDict(config) produces the dictionary correctly).

After making this change, code execution fails producing the traceback log reported below. Obviously, I have zero knowledge of how pytorch works internally. However, to my uneducated eye this new error points to an issue in pytorch. What do you think? I could easily be wrong.

Thanks

TRACEBACK LOG

Traceback (most recent call last):
  File "/home/adonnini1/Development/ContextQSourceCode/NeuralNetworks/LocationPrediction/main.py", line 68, in <module>
    res_single = single_run(train_loader, val_loader, test_loader, config, device, log_dir)
  File "/home/adonnini1/Development/ContextQSourceCode/NeuralNetworks/LocationPrediction/main.py", line 23, in single_run
    model, perf = get_trainedNets(config, model, train_loader, val_loader, device, log_dir)
  File "/home/adonnini1/Development/ContextQSourceCode/NeuralNetworks/LocationPrediction/utils/utils.py", line 47, in get_trainedNets
    best_model, performance = trainNet(config, model, train_loader, val_loader, device, log_dir=log_dir)
  File "/home/adonnini1/Development/ContextQSourceCode/NeuralNetworks/LocationPrediction/utils/train.py", line 375, in trainNet
    pre_autograd_aten_dialect = torch.export._trace._export(m, (config, total_loc_num, device), strict=False, pre_dispatch=True)
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/export/_trace.py", line 833, in wrapper
    raise e
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/export/_trace.py", line 816, in wrapper
    ep = fn(*args, **kwargs)
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/export/exported_program.py", line 85, in wrapper
    return fn(*args, **kwargs)
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/export/_trace.py", line 988, in _export
    ) = make_fake_inputs(mod, args, kwargs, dynamic_shapes)
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/_export/non_strict_utils.py", line 148, in make_fake_inputs
    fake_args, fake_kwargs = tree_map_with_path(
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/utils/_pytree.py", line 1581, in tree_map_with_path
    return treespec.unflatten(func(*xs) for xs in zip(*all_keypath_leaves))
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/utils/_pytree.py", line 782, in unflatten
    leaves = list(leaves)
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/utils/_pytree.py", line 1581, in <genexpr>
    return treespec.unflatten(func(*xs) for xs in zip(*all_keypath_leaves))
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/_export/non_strict_utils.py", line 149, in <lambda>
    lambda kp, val: fakify(fake_mode, kp, val, t_constraints, sources),
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/_export/non_strict_utils.py", line 72, in fakify
    raise ValueError(f"Unsupported input type {type(t)}")
ValueError: Unsupported input type <class 'utils.train.AttrDict'>
adonnini commented 5 months ago

@angelayi Next, I added

        from scipy.io import _idl
        config = _idl.AttrDict(config)

I thought that using a scipy utility to create the dictionary would resolve this problem. After making this change, code execution fails producing the traceback log reported below.

I will continue to work on the flatten/unflatten functions. However, to me this increasingly looks like an issue outside of my control.

Thanks

TRACEBACK LOG

Traceback (most recent call last):
  File "/home/adonnini1/Development/ContextQSourceCode/NeuralNetworks/LocationPrediction/main.py", line 68, in <module>
    res_single = single_run(train_loader, val_loader, test_loader, config, device, log_dir)
  File "/home/adonnini1/Development/ContextQSourceCode/NeuralNetworks/LocationPrediction/main.py", line 23, in single_run
    model, perf = get_trainedNets(config, model, train_loader, val_loader, device, log_dir)
  File "/home/adonnini1/Development/ContextQSourceCode/NeuralNetworks/LocationPrediction/utils/utils.py", line 47, in get_trainedNets
    best_model, performance = trainNet(config, model, train_loader, val_loader, device, log_dir=log_dir)
  File "/home/adonnini1/Development/ContextQSourceCode/NeuralNetworks/LocationPrediction/utils/train.py", line 377, in trainNet
    pre_autograd_aten_dialect = torch.export._trace._export(m, (config, total_loc_num, device), strict=False, pre_dispatch=True)
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/export/_trace.py", line 833, in wrapper
    raise e
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/export/_trace.py", line 816, in wrapper
    ep = fn(*args, **kwargs)
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/export/exported_program.py", line 85, in wrapper
    return fn(*args, **kwargs)
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/export/_trace.py", line 988, in _export
    ) = make_fake_inputs(mod, args, kwargs, dynamic_shapes)
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/_export/non_strict_utils.py", line 148, in make_fake_inputs
    fake_args, fake_kwargs = tree_map_with_path(
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/utils/_pytree.py", line 1581, in tree_map_with_path
    return treespec.unflatten(func(*xs) for xs in zip(*all_keypath_leaves))
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/utils/_pytree.py", line 782, in unflatten
    leaves = list(leaves)
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/utils/_pytree.py", line 1581, in <genexpr>
    return treespec.unflatten(func(*xs) for xs in zip(*all_keypath_leaves))
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/_export/non_strict_utils.py", line 149, in <lambda>
    lambda kp, val: fakify(fake_mode, kp, val, t_constraints, sources),
  File "/home/adonnini1/anaconda3/envs/executorch/lib/python3.10/site-packages/torch/_export/non_strict_utils.py", line 72, in fakify
    raise ValueError(f"Unsupported input type {type(t)}")
ValueError: Unsupported input type <class 'scipy.io._idl.AttrDict'>
adonnini commented 5 months ago

@angelayi I apologize in advance if I am being too blunt, and if I come across as being aggressive. I am blunt but I certinaly do not mean to be aggressive. I have a lot on my plate, as I imagine you do too. Before I proceed further with your suggestion to use https://github.com/pytorch/pytorch/blob/394ec2da300e8933d8184ba862daed3a115d7fd9/torch/utils/_pytree.py#L163 I want to ask you. Are you very confident that doing this will resolve the problem I have run into? I am asking for two reasons: 1) It seems that just about any dictionary creating utility I use causes this problem. Based on my experience in similar situations, this would seem to indicate that the problem lies with the code where the error occurs and the error needs to be resolved there. Any other potential solution is at best a (short-term) workaround) 2) Please do not be offended. All potential solutions suggested to date have not made a difference. The problem still persists.

@angelayi you have been very responsive. I really appreciate it. Please let me know what you think. Thanks

angelayi commented 5 months ago

@adonnini Sorry for the frustrating experience! I believe this should fix the issue with the Unsupported input type <class EasyDict>. Here's an example:

import torch.utils._pytree as pytree

class CustomDict(dict):
    def __init__(self, *args, **kwargs):
        super(CustomDict, self).__init__(*args, **kwargs)
        self.__dict__ = self

def _dict_flatten(d):
    return list(d.values()), list(d.keys())

def _dict_flatten_with_keys(d):
    values, context = _dict_flatten(d)
    return [(pytree.MappingKey(k), v) for k, v in zip(context, values)], context

def _dict_unflatten(values, context):
    return dict(zip(context, values))

pytree.register_pytree_node(
    CustomDict,
    flatten_fn=_dict_flatten,
    unflatten_fn=_dict_unflatten,
    serialized_type_name="CustomDict",
    flatten_with_keys_fn=_dict_flatten_with_keys
)

class M(torch.nn.Module):
    def forward(self, d: CustomDict):
        return d["x"] + d["y"]

ep = torch.export.export(M(), (CustomDict({"x": torch.ones(3, 3), "y": torch.ones(3, 3)}),), strict=False)
print(ep)
"""
ExportedProgram:
    class GraphModule(torch.nn.Module):
        def forward(self, d_x: "f32[3, 3]", d_y: "f32[3, 3]"):
            # File: /data/users/angelayi/pytorch/moo.py:39 in forward, code: return d["x"] + d["y"]
            add: "f32[3, 3]" = torch.ops.aten.add.Tensor(d_x, d_y);  d_x = d_y = None
            return (add,)
"""

These are the allowed input types to torch.export: