search

MIT-REALM / neural_clbf

Toolkit for learning controllers based on robust control Lyapunov barrier functions
BSD 3-Clause "New" or "Revised" License
124 stars 43 forks source link

ValueError: Can't apply Jacobian with a quadratic objective #14

Closed JuneJulyAugust closed 1 year ago

JuneJulyAugust commented 1 year ago

Hi, I don't have a gurobi license, so I set disable_hurobi=True to run the training:

python -m neural_clbf.training.train_kinematic_car --gpus=1

But I got this error:

GPU available: True, used: True
TPU available: False, using: 0 TPU cores
Full dataset:
    9001 training
    1000 validation
    ----------------------
    40 goal points
    (3 val)
    3644 safe points
    (406 val)
    5283 unsafe points
    (587 val)
    74 boundary points
    (7 val)
LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]

  | Name                     | Type       | Params
--------------------------------------------------------
0 | differentiable_qp_solver | CvxpyLayer | 0
1 | V_nn                     | Sequential | 8.8 K
--------------------------------------------------------
8.8 K     Trainable params
0         Non-trainable params
8.8 K     Total params
0.035     Total estimated model params size (MB)
Validation sanity check: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 2/2 [00:02<00:00,  1.06it/s]/home/fang/miniconda3/envs/neural_clbf/lib/python3.9/site-packages/pytorch_lightning/core/step_result.py:115: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).
  value = torch.tensor(value, device=device, dtype=torch.float)
Plotting CLF: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 50/50 [01:40<00:00,  2.01s/it]
S-Curve: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 4999/4999 [00:55<00:00, 90.47it/s]
Epoch 0:   0%|                                                                                                                                                                                                               | 0/157 [00:00<?, ?it/s][W python_anomaly_mode.cpp:104] Warning: Error detected in _CvxpyLayerFnFnBackward. Traceback of forward call that caused the error:
  File "/home/fang/miniconda3/envs/neural_clbf/lib/python3.9/runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/home/fang/miniconda3/envs/neural_clbf/lib/python3.9/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/data/projects/fang/neural_clbf/neural_clbf/training/train_kinematic_car.py", line 133, in <module>
    main(args)
  File "/data/projects/fang/neural_clbf/neural_clbf/training/train_kinematic_car.py", line 125, in main
    trainer.fit(clbf_controller)
  File "/home/fang/miniconda3/envs/neural_clbf/lib/python3.9/site-packages/pytorch_lightning/trainer/trainer.py", line 458, in fit
    self._run(model)
  File "/home/fang/miniconda3/envs/neural_clbf/lib/python3.9/site-packages/pytorch_lightning/trainer/trainer.py", line 756, in _run
    self.dispatch()
  File "/home/fang/miniconda3/envs/neural_clbf/lib/python3.9/site-packages/pytorch_lightning/trainer/trainer.py", line 797, in dispatch
    self.accelerator.start_training(self)
  File "/home/fang/miniconda3/envs/neural_clbf/lib/python3.9/site-packages/pytorch_lightning/accelerators/accelerator.py", line 96, in start_training
    self.training_type_plugin.start_training(trainer)
  File "/home/fang/miniconda3/envs/neural_clbf/lib/python3.9/site-packages/pytorch_lightning/plugins/training_type/training_type_plugin.py", line 144, in start_training
    self._results = trainer.run_stage()
  File "/home/fang/miniconda3/envs/neural_clbf/lib/python3.9/site-packages/pytorch_lightning/trainer/trainer.py", line 807, in run_stage
    return self.run_train()
  File "/home/fang/miniconda3/envs/neural_clbf/lib/python3.9/site-packages/pytorch_lightning/trainer/trainer.py", line 869, in run_train
    self.train_loop.run_training_epoch()
  File "/home/fang/miniconda3/envs/neural_clbf/lib/python3.9/site-packages/pytorch_lightning/trainer/training_loop.py", line 489, in run_training_epoch
    batch_output = self.run_training_batch(batch, batch_idx, dataloader_idx)
  File "/home/fang/miniconda3/envs/neural_clbf/lib/python3.9/site-packages/pytorch_lightning/trainer/training_loop.py", line 728, in run_training_batch
    self.optimizer_step(optimizer, opt_idx, batch_idx, train_step_and_backward_closure)
  File "/home/fang/miniconda3/envs/neural_clbf/lib/python3.9/site-packages/pytorch_lightning/trainer/training_loop.py", line 424, in optimizer_step
    model_ref.optimizer_step(
  File "/home/fang/miniconda3/envs/neural_clbf/lib/python3.9/site-packages/pytorch_lightning/core/lightning.py", line 1403, in optimizer_step
    optimizer.step(closure=optimizer_closure)
  File "/home/fang/miniconda3/envs/neural_clbf/lib/python3.9/site-packages/pytorch_lightning/core/optimizer.py", line 214, in step
    self.__optimizer_step(*args, closure=closure, profiler_name=profiler_name, **kwargs)
  File "/home/fang/miniconda3/envs/neural_clbf/lib/python3.9/site-packages/pytorch_lightning/core/optimizer.py", line 134, in __optimizer_step
    trainer.accelerator.optimizer_step(optimizer, self._optimizer_idx, lambda_closure=closure, **kwargs)
  File "/home/fang/miniconda3/envs/neural_clbf/lib/python3.9/site-packages/pytorch_lightning/accelerators/accelerator.py", line 329, in optimizer_step
    self.run_optimizer_step(optimizer, opt_idx, lambda_closure, **kwargs)
  File "/home/fang/miniconda3/envs/neural_clbf/lib/python3.9/site-packages/pytorch_lightning/accelerators/accelerator.py", line 336, in run_optimizer_step
    self.training_type_plugin.optimizer_step(optimizer, lambda_closure=lambda_closure, **kwargs)
  File "/home/fang/miniconda3/envs/neural_clbf/lib/python3.9/site-packages/pytorch_lightning/plugins/training_type/training_type_plugin.py", line 193, in optimizer_step
    optimizer.step(closure=lambda_closure, **kwargs)
  File "/home/fang/miniconda3/envs/neural_clbf/lib/python3.9/site-packages/torch/optim/optimizer.py", line 88, in wrapper
    return func(*args, **kwargs)
  File "/home/fang/miniconda3/envs/neural_clbf/lib/python3.9/site-packages/torch/autograd/grad_mode.py", line 28, in decorate_context
    return func(*args, **kwargs)
  File "/home/fang/miniconda3/envs/neural_clbf/lib/python3.9/site-packages/torch/optim/sgd.py", line 87, in step
    loss = closure()
  File "/home/fang/miniconda3/envs/neural_clbf/lib/python3.9/site-packages/pytorch_lightning/trainer/training_loop.py", line 722, in train_step_and_backward_closure
    result = self.training_step_and_backward(
  File "/home/fang/miniconda3/envs/neural_clbf/lib/python3.9/site-packages/pytorch_lightning/trainer/training_loop.py", line 813, in training_step_and_backward
    result = self.training_step(split_batch, batch_idx, opt_idx, hiddens)
  File "/home/fang/miniconda3/envs/neural_clbf/lib/python3.9/site-packages/pytorch_lightning/trainer/training_loop.py", line 280, in training_step
    training_step_output = self.trainer.accelerator.training_step(args)
  File "/home/fang/miniconda3/envs/neural_clbf/lib/python3.9/site-packages/pytorch_lightning/accelerators/accelerator.py", line 204, in training_step
    return self.training_type_plugin.training_step(*args)
  File "/home/fang/miniconda3/envs/neural_clbf/lib/python3.9/site-packages/pytorch_lightning/plugins/training_type/training_type_plugin.py", line 155, in training_step
    return self.lightning_module.training_step(*args, **kwargs)
  File "/data/projects/fang/neural_clbf/neural_clbf/controllers/neural_clbf_controller.py", line 446, in training_step
    self.descent_loss(x, goal_mask, safe_mask, unsafe_mask, requires_grad=True)
  File "/data/projects/fang/neural_clbf/neural_clbf/controllers/neural_clbf_controller.py", line 346, in descent_loss
    u_qp, qp_relaxation = self.solve_CLF_QP(x, requires_grad=requires_grad)
  File "/data/projects/fang/neural_clbf/neural_clbf/controllers/clf_controller.py", line 409, in solve_CLF_QP
    return self._solve_CLF_QP_cvxpylayers(
  File "/data/projects/fang/neural_clbf/neural_clbf/controllers/clf_controller.py", line 353, in _solve_CLF_QP_cvxpylayers
    result = self.differentiable_qp_solver(
  File "/home/fang/miniconda3/envs/neural_clbf/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1051, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/fang/miniconda3/envs/neural_clbf/lib/python3.9/site-packages/cvxpylayers/torch/cvxpylayer.py", line 152, in forward
    sol = f(*params)
 (function _print_stack)
Traceback (most recent call last):
  File "/home/fang/miniconda3/envs/neural_clbf/lib/python3.9/runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/home/fang/miniconda3/envs/neural_clbf/lib/python3.9/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/data/projects/fang/neural_clbf/neural_clbf/training/train_kinematic_car.py", line 133, in <module>
    main(args)
  File "/data/projects/fang/neural_clbf/neural_clbf/training/train_kinematic_car.py", line 125, in main
    trainer.fit(clbf_controller)
  File "/home/fang/miniconda3/envs/neural_clbf/lib/python3.9/site-packages/pytorch_lightning/trainer/trainer.py", line 458, in fit
    self._run(model)
  File "/home/fang/miniconda3/envs/neural_clbf/lib/python3.9/site-packages/pytorch_lightning/trainer/trainer.py", line 756, in _run
    self.dispatch()
  File "/home/fang/miniconda3/envs/neural_clbf/lib/python3.9/site-packages/pytorch_lightning/trainer/trainer.py", line 797, in dispatch
    self.accelerator.start_training(self)
  File "/home/fang/miniconda3/envs/neural_clbf/lib/python3.9/site-packages/pytorch_lightning/accelerators/accelerator.py", line 96, in start_training
    self.training_type_plugin.start_training(trainer)
  File "/home/fang/miniconda3/envs/neural_clbf/lib/python3.9/site-packages/pytorch_lightning/plugins/training_type/training_type_plugin.py", line 144, in start_training
    self._results = trainer.run_stage()
  File "/home/fang/miniconda3/envs/neural_clbf/lib/python3.9/site-packages/pytorch_lightning/trainer/trainer.py", line 807, in run_stage
    return self.run_train()
  File "/home/fang/miniconda3/envs/neural_clbf/lib/python3.9/site-packages/pytorch_lightning/trainer/trainer.py", line 869, in run_train
    self.train_loop.run_training_epoch()
  File "/home/fang/miniconda3/envs/neural_clbf/lib/python3.9/site-packages/pytorch_lightning/trainer/training_loop.py", line 489, in run_training_epoch
    batch_output = self.run_training_batch(batch, batch_idx, dataloader_idx)
  File "/home/fang/miniconda3/envs/neural_clbf/lib/python3.9/site-packages/pytorch_lightning/trainer/training_loop.py", line 728, in run_training_batch
    self.optimizer_step(optimizer, opt_idx, batch_idx, train_step_and_backward_closure)
  File "/home/fang/miniconda3/envs/neural_clbf/lib/python3.9/site-packages/pytorch_lightning/trainer/training_loop.py", line 424, in optimizer_step
    model_ref.optimizer_step(
  File "/home/fang/miniconda3/envs/neural_clbf/lib/python3.9/site-packages/pytorch_lightning/core/lightning.py", line 1403, in optimizer_step
    optimizer.step(closure=optimizer_closure)
  File "/home/fang/miniconda3/envs/neural_clbf/lib/python3.9/site-packages/pytorch_lightning/core/optimizer.py", line 214, in step
    self.__optimizer_step(*args, closure=closure, profiler_name=profiler_name, **kwargs)
  File "/home/fang/miniconda3/envs/neural_clbf/lib/python3.9/site-packages/pytorch_lightning/core/optimizer.py", line 134, in __optimizer_step
    trainer.accelerator.optimizer_step(optimizer, self._optimizer_idx, lambda_closure=closure, **kwargs)
  File "/home/fang/miniconda3/envs/neural_clbf/lib/python3.9/site-packages/pytorch_lightning/accelerators/accelerator.py", line 329, in optimizer_step
    self.run_optimizer_step(optimizer, opt_idx, lambda_closure, **kwargs)
  File "/home/fang/miniconda3/envs/neural_clbf/lib/python3.9/site-packages/pytorch_lightning/accelerators/accelerator.py", line 336, in run_optimizer_step
    self.training_type_plugin.optimizer_step(optimizer, lambda_closure=lambda_closure, **kwargs)
  File "/home/fang/miniconda3/envs/neural_clbf/lib/python3.9/site-packages/pytorch_lightning/plugins/training_type/training_type_plugin.py", line 193, in optimizer_step
    optimizer.step(closure=lambda_closure, **kwargs)
  File "/home/fang/miniconda3/envs/neural_clbf/lib/python3.9/site-packages/torch/optim/optimizer.py", line 88, in wrapper
    return func(*args, **kwargs)
  File "/home/fang/miniconda3/envs/neural_clbf/lib/python3.9/site-packages/torch/autograd/grad_mode.py", line 28, in decorate_context
    return func(*args, **kwargs)
  File "/home/fang/miniconda3/envs/neural_clbf/lib/python3.9/site-packages/torch/optim/sgd.py", line 87, in step
    loss = closure()
  File "/home/fang/miniconda3/envs/neural_clbf/lib/python3.9/site-packages/pytorch_lightning/trainer/training_loop.py", line 722, in train_step_and_backward_closure
    result = self.training_step_and_backward(
  File "/home/fang/miniconda3/envs/neural_clbf/lib/python3.9/site-packages/pytorch_lightning/trainer/training_loop.py", line 826, in training_step_and_backward
    self.backward(result, optimizer, opt_idx)
  File "/home/fang/miniconda3/envs/neural_clbf/lib/python3.9/site-packages/pytorch_lightning/trainer/training_loop.py", line 859, in backward
    result.closure_loss = self.trainer.accelerator.backward(
  File "/home/fang/miniconda3/envs/neural_clbf/lib/python3.9/site-packages/pytorch_lightning/accelerators/accelerator.py", line 308, in backward
    output = self.precision_plugin.backward(
  File "/home/fang/miniconda3/envs/neural_clbf/lib/python3.9/site-packages/pytorch_lightning/plugins/precision/precision_plugin.py", line 79, in backward
    model.backward(closure_loss, optimizer, opt_idx)
  File "/home/fang/miniconda3/envs/neural_clbf/lib/python3.9/site-packages/pytorch_lightning/core/lightning.py", line 1275, in backward
    loss.backward(*args, **kwargs)
  File "/home/fang/miniconda3/envs/neural_clbf/lib/python3.9/site-packages/torch/_tensor.py", line 255, in backward
    torch.autograd.backward(self, gradient, retain_graph, create_graph, inputs=inputs)
  File "/home/fang/miniconda3/envs/neural_clbf/lib/python3.9/site-packages/torch/autograd/__init__.py", line 147, in backward
    Variable._execution_engine.run_backward(
  File "/home/fang/miniconda3/envs/neural_clbf/lib/python3.9/site-packages/torch/autograd/function.py", line 87, in apply
    return self._forward_cls.backward(self, *args)  # type: ignore[attr-defined]
  File "/home/fang/miniconda3/envs/neural_clbf/lib/python3.9/site-packages/cvxpylayers/torch/cvxpylayer.py", line 344, in backward
    del_param_dict = compiler.apply_param_jac(
  File "/home/fang/miniconda3/envs/neural_clbf/lib/python3.9/site-packages/cvxpy/reductions/dcp2cone/cone_matrix_stuffing.py", line 208, in apply_param_jac
    raise ValueError("Can't apply Jacobian with a quadratic objective.")
ValueError: Can't apply Jacobian with a quadratic objective.
Epoch 0:   0%|          | 0/157 [00:01<?, ?it/s]
JuneJulyAugust commented 1 year ago

The cvxpylayers version installed:

pip list | grep cvxpylayers
cvxpylayers 0.1.5

JuneJulyAugust commented 1 year ago

This is cvxpy version installed, I will try an older version.

pip list | grep cvxpy
cvxpy 1.3.0

JuneJulyAugust commented 1 year ago

I think I figured out a solution by myself. Now the training is running.

Here is my changes of requirements.txt. I used scipy and cvxpy release in the summer of 2021.

diff --git a/requirements.txt b/requirements.txt
index 41f9294..0ae4ddc 100644
--- a/requirements.txt
+++ b/requirements.txt
@@ -1,5 +1,6 @@
 # Standard
 numpy
+scipy==1.7.0
 matplotlib
 seaborn
 pandas
@@ -16,8 +17,8 @@ pytorch-lightning==1.3.4
 torchmetrics==0.5.1

 # Optimization
-cvxpy>=1.1.14
-cvxpylayers
+cvxpy==1.1.14
+cvxpylayers==0.1.4
 gurobipy==9.1.2
 osqp
 ecos
dawsonc commented 1 year ago

Thanks for reporting this bug! I was able to reproduce it on my laptop and it does seem to be a version incompatibility with cvxpy 1.3. I'll fix the version at 1.2.2 (the last known working version) in an upcoming PR

dawsonc commented 1 year ago

This should be resolved as of PR #16, but please re-open if the issue persists and I'll be happy to help.

JuneJulyAugust commented 1 year ago

It works. Thanks for the fix.