fail in reproducing the result of hover

[zhixiongzh]

Hi,

I am running the provided example gym_pybullet_drones/examples/learn.py, but the agent failed to hover at the desired position.

Then I found the provided script is different from the paper then I git checkout master and run the experiments experiments/learning/singleagent.py but there are a lot of errors due to version conflict.

Is there any suggestion to train a good agent with gym_pybullet_drones/examples/learn.py in current main branch?

[RibhavOjha]

@zhixiongzh I am getting exactly the same problem. learn.py doesn't hover properly and just crashes. When I run singleagent.py, it gives this error:

 File "C:\Users\Username\AppData\Roaming\Python\Python311\site-packages\torch\nn\modules\linear.py", line 96, in __init__
    self.weight = Parameter(torch.empty((out_features, in_features), **factory_kwargs))
                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: empty() received an invalid combination of arguments - got (tuple, dtype=NoneType, device=NoneType), but expected one of:
 * (tuple of ints size, *, tuple of names names, torch.memory_format memory_format, torch.dtype dtype, torch.layout layout, torch.device device, bool pin_memory, bool requires_grad)
 * (tuple of ints size, *, torch.memory_format memory_format, Tensor out, torch.dtype dtype, torch.layout layout, torch.device device, bool pin_memory, bool requires_grad)

Any updates?

[zhixiongzh]

@RibhavOjha I only run the code in learn.py not the singleagent.py. I also have not yet reproduced the result using the example

[RibhavOjha]

@zhixiongzh are you also getting the same error as me for singleagent.py ?

[JacopoPan]

Hi @zhixiongzh and @RibhavOjha

to reproduce the work in the paper you should checkout branches master or paper (and I am aware that some of the 3rd party dependencies are hardly maintained/backward compatible, unfortunately).

The current main/default branch is currently worked on to support SITL simulation and after that is done, I will re-introduce the learning examples (this time based on gymnasium and sb3 2.0).

[cyril-data]

I think you could get rid of this error :

 File "C:\Users\Username\AppData\Roaming\Python\Python311\site-packages\torch\nn\modules\linear.py", line 96, in __init__
    self.weight = Parameter(torch.empty((out_features, in_features), **factory_kwargs))
                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: empty() received an invalid combination of arguments - got (tuple, dtype=NoneType, device=NoneType), but expected one of:
 * (tuple of ints size, *, tuple of names names, torch.memory_format memory_format, torch.dtype dtype, torch.layout layout, torch.device device, bool pin_memory, bool requires_grad)
 * (tuple of ints size, *, torch.memory_format memory_format, Tensor out, torch.dtype dtype, torch.layout layout, torch.device device, bool pin_memory, bool requires_grad)

By changing the onpolicy_kwargs in singleagent.py by this :

    # onpolicy_kwargs = dict(activation_fn=torch.nn.ReLU,
    #                        net_arch=[512, 512, dict(vf=[256, 128], pi=[256, 128])]
    #                        ) # or None
    onpolicy_kwargs = dict(activation_fn=torch.nn.ReLU,
                           net_arch=[512, 512, 256, 128]
                           )  # or None

But I face the same difficulty to reproduce the results with ppo and 1D (identical input to all motors) with RPMs ("one_d_rpm")...

[abdul-mannan-khan]

Same. I continued to run the training for 1E10. It did not work.

[JacopoPan]

See #180 for the current status