Closed 4ku closed 5 months ago
Thanks, it should have been PYB_FREQ
, apologies I missed that in the recent refactoring, pushed now
(Please close the issue yourself, it that solved your issue or let me know otherwise)
It also seems that the folder is not created here, so the error occurs when starting the recording. Also it seems a problem with ONBOARD_IMG_PATH folder creation.
ok, let me check, you are on branch main
correct? Can you also show me which commands/examples you are running when incurring in these problems?
Hi @4ku
I realized there is nothing "too" wrong with ONBOARD_IMG_PATH
per se
You need to set this line
https://github.com/utiasDSL/gym-pybullet-drones/blob/e8ae019ba544c6b23416a590b86c75736d6aad40/gym_pybullet_drones/envs/BaseAviary.py#L38
and --record
to True
to create that path.
The actual problem would be that the lines that actually save FPV frames are here, in a subclass of BaseAviary
that I did not move to main
for i in range(self.NUM_DRONES):
if self.step_counter%self.IMG_CAPTURE_FREQ == 0:
self.rgb[i], self.dep[i], self.seg[i] = self._getDroneImages(i)
#### Printing observation to PNG frames example ############
if self.RECORD:
self._exportImage(img_type=ImageType.RGB, # ImageType.BW, ImageType.DEP, ImageType.SEG
img_input=self.rgb[i],
path=self.ONBOARD_IMG_PATH+"drone_"+str(i),
frame_num=int(self.step_counter/self.IMG_CAPTURE_FREQ)
)
Can you explain me a bit better what you are doing so that I can try and help/decide what to modify?
For now, I put those lines back into the step()
method of BaseAviary
(you still need to manually set vision_attributes
to True
)
Yeah, I launch main
branch. So for example if I just change DEFAULT_RECORD_VIDEO
to True
in gym_pybullet_drones/examples/learn.py
and run it, than results
folder will be created, but no any video will be recorded.
Than if I change default observation type to ObservationType.RGB
in gym_pybullet_drones/envs/single_agent_rl/HoverAviary.py
and run gym_pybullet_drones/examples/learn.py
again, than I will have the following error:
(env) ivan@ivanPC:~/Desktop/gym-pybullet-drones/gym_pybullet_drones/examples$ python3 learn.py
pybullet build time: Jul 30 2023 22:42:35
[INFO] BaseAviary.__init__() loaded parameters from the drone's .urdf:
[INFO] m 0.027000, L 0.039700,
[INFO] ixx 0.000014, iyy 0.000014, izz 0.000022,
[INFO] kf 0.000000, km 0.000000,
[INFO] t2w 2.250000, max_speed_kmh 30.000000,
[INFO] gnd_eff_coeff 11.368590, prop_radius 0.023135,
[INFO] drag_xy_coeff 0.000001, drag_z_coeff 0.000001,
[INFO] dw_coeff_1 2267.180000, dw_coeff_2 0.160000, dw_coeff_3 -0.110000
Traceback (most recent call last):
File "/home/ivan/Desktop/gym-pybullet-drones/gym_pybullet_drones/examples/learn.py", line 86, in <module>
run(**vars(ARGS))
File "/home/ivan/Desktop/gym-pybullet-drones/gym_pybullet_drones/examples/learn.py", line 35, in run
env = gym.make("hover-aviary-v0")
File "/home/ivan/Desktop/gym-pybullet-drones/env/lib/python3.10/site-packages/gymnasium/envs/registration.py", line 801, in make
env = env_creator(**env_spec_kwargs)
File "/home/ivan/Desktop/gym-pybullet-drones/gym_pybullet_drones/envs/single_agent_rl/HoverAviary.py", line 51, in __init__
super().__init__(drone_model=drone_model,
File "/home/ivan/Desktop/gym-pybullet-drones/gym_pybullet_drones/envs/single_agent_rl/BaseSingleAgentAviary.py", line 71, in __init__
super().__init__(drone_model=drone_model,
File "/home/ivan/Desktop/gym-pybullet-drones/gym_pybullet_drones/envs/BaseAviary.py", line 145, in __init__
os.makedirs(os.path.dirname(self.ONBOARD_IMG_PATH+"/drone_"+str(i)+"/"), exist_ok=True)
AttributeError: 'HoverAviary' object has no attribute 'ONBOARD_IMG_PATH'
Thanks @4ku
So the first issue, I admit it's counterintuitive but it's by design, frame's are only saved if the GUI is not showing (see BaseAviary
):
#### Save PNG video frames if RECORD=True and GUI=False ####
if self.RECORD and not self.GUI and self.step_counter%self.CAPTURE_FREQ == 0:
For the second, I need to look into it further, that attribute should be created every time self.RECORD
is true in a BaseAviary
(and sub-classes):
if self.RECORD:
self.ONBOARD_IMG_PATH = os.path.join(self.OUTPUT_FOLDER, "recording_" + datetime.now().strftime("%m.%d.%Y_%H.%M.%S"))
Yeah, the first issue is gone, but after DEFAULT_GUI
making False
the second issue is still there.
I see,
ok, I now fixed that by adding the condition on self.RECORD
in BaseAviary
for the creation of paths to save the images (the error was coming from the training environment in learn.py
, not the test one)
if self.RECORD:
for i in range(self.NUM_DRONES):
os.makedirs(os.path.dirname(self.ONBOARD_IMG_PATH+"/drone_"+str(i)+"/"), exist_ok=True)
However, note that the second part of learn.py
would still throw an error because there is some logging that it's hardcoded to expect the observation to contain the state of the system instead of images (if you comment that out it should run as expected)
logger.log(drone=0,
timestamp=i/env.CTRL_FREQ,
state=np.hstack([obs[0:3], np.zeros(4), obs[3:15], np.resize(action, (4))]),
control=np.zeros(12)
)
I update code and comment out logger.log
line in learn.py
, and now have this error:
Traceback (most recent call last):
File "/home/ivan/Desktop/gym-pybullet-drones/gym_pybullet_drones/examples/learn.py", line 86, in <module>
run(**vars(ARGS))
File "/home/ivan/Desktop/gym-pybullet-drones/gym_pybullet_drones/examples/learn.py", line 61, in run
obs, reward, terminated, truncated, info = env.step(action)
File "/home/ivan/Desktop/gym-pybullet-drones/gym_pybullet_drones/envs/BaseAviary.py", line 313, in step
self._exportImage(img_type=ImageType.RGB, # ImageType.BW, ImageType.DEP, ImageType.SEG
File "/home/ivan/Desktop/gym-pybullet-drones/gym_pybullet_drones/envs/BaseAviary.py", line 646, in _exportImage
(Image.fromarray(img_input.astype('uint8'), 'RGBA')).save(os.path.join(path,"frame_"+str(frame_num)+".png"))
File "/home/ivan/Desktop/gym-pybullet-drones/env/lib/python3.10/site-packages/PIL/Image.py", line 2410, in save
fp = builtins.open(filename, "w+b")
FileNotFoundError: [Errno 2] No such file or directory: 'results/recording_08.05.2023_16.23.29/drone_00/frame_0.png'
That just seems a typo in the save path name results/recording_08.05.2023_16.23.29/drone_00/frame_0.png
, it should be results/recording_08.05.2023_16.23.29/drone_0/frame_0.png
, there was an extra +str(i)+
I now removed.
I pull new code and do the same. Now have this error:
Traceback (most recent call last):
File "/home/ivan/Desktop/gym-pybullet-drones/gym_pybullet_drones/examples/learn.py", line 86, in <module>
run(**vars(ARGS))
File "/home/ivan/Desktop/gym-pybullet-drones/gym_pybullet_drones/examples/learn.py", line 75, in run
logger.plot()
File "/home/ivan/Desktop/gym-pybullet-drones/gym_pybullet_drones/utils/Logger.py", line 305, in plot
axs[row, col].plot(t, rdot, label="drone_"+str(j))
File "/home/ivan/Desktop/gym-pybullet-drones/env/lib/python3.10/site-packages/matplotlib/axes/_axes.py", line 1688, in plot
lines = [*self._get_lines(*args, data=data, **kwargs)]
File "/home/ivan/Desktop/gym-pybullet-drones/env/lib/python3.10/site-packages/matplotlib/axes/_base.py", line 311, in __call__
yield from self._plot_args(
File "/home/ivan/Desktop/gym-pybullet-drones/env/lib/python3.10/site-packages/matplotlib/axes/_base.py", line 504, in _plot_args
raise ValueError(f"x and y must have same first dimension, but "
ValueError: x and y must have same first dimension, but have shapes (0,) and (1,)
Ok, but this, as we said, comes from the logger that is intended for KIN observations and won’t work with RGB. Comment out line 75.
yeah, I am sorry. Now everything is ok. Thank you)
132 line only creates self.OUTPUT_FOLDER
:
https://github.com/utiasDSL/gym-pybullet-drones/blob/73f83329b6e011fecf60fd9b41e3bff9cc95b7f5/gym_pybullet_drones/envs/BaseAviary.py#L131-L132
Folder is not created here: https://github.com/utiasDSL/gym-pybullet-drones/blob/73f83329b6e011fecf60fd9b41e3bff9cc95b7f5/gym_pybullet_drones/envs/BaseAviary.py#L533-L535
That's why an errors like that can occur:
results/recording_08.16.2023_11.57.55/output.mp4: No such file or directory
Line 132 creates a folder for self.ONBOARD_IMG_PATH
, subfolder of self.OUTPUT_FOLDER
(only when the FPV images are requested to be saved).
What you are saying is that the folder for self.OUTPUT_FOLDER
might not be created otherwise and that causes an error when only trying to save the scene video, correct?
Does your problem go away just by adding on line 96 of BaseAviary.py
?
os.makedirs(os.path.dirname(self.OUTPUT_FOLDER), exist_ok=True)
If yes, feel free to contribute with a PR or I'll commit the change later.
I mean that this https://github.com/utiasDSL/gym-pybullet-drones/blob/73f83329b6e011fecf60fd9b41e3bff9cc95b7f5/gym_pybullet_drones/envs/BaseAviary.py#L131-L132
is equal to this
os.makedirs('results', , exist_ok=True)
if we are going to launch with default self.OUTPUT_FOLDER
value.
And this lines https://github.com/utiasDSL/gym-pybullet-drones/blob/73f83329b6e011fecf60fd9b41e3bff9cc95b7f5/gym_pybullet_drones/envs/BaseAviary.py#L533-L535 should look something like that:
VIDEO_FOLDER = os.path.join(self.OUTPUT_FOLDER, "recording_" + datetime.now().strftime("%m.%d.%Y_%H.%M.%S"))
os.makedirs(VIDEO_FOLDER, exist_ok=True)
self.VIDEO_ID = p.startStateLogging(loggingType=p.STATE_LOGGING_VIDEO_MP4,
fileName=os.path.join(VIDEO_FOLDER, "output.mp4"),
physicsClientId=self.CLIENT
)
I think it's better to create one folder in the init and use it later:
self.RESULT_FOLDER = os.path.join(self.OUTPUT_FOLDER, "recording_" + datetime.now().strftime("%m.%d.%Y_%H.%M.%S"))
os.makedirs(self.RESULT_FOLDER, exist_ok=True)
Thank you for the detailed discussion, @JacopoPan and @4ku. I am facing the same problem. I have one dumb question. In which python file did you put this section of code?
self.RESULT_FOLDER = os.path.join(self.OUTPUT_FOLDER, "recording_" + datetime.now().strftime("%m.%d.%Y_%H.%M.%S"))
os.makedirs(self.RESULT_FOLDER, exist_ok=True)
https://github.com/utiasDSL/gym-pybullet-drones/blob/fa640e3c0dd3460ea35911a2a0d3144598bf48f5/gym_pybullet_drones/envs/BaseAviary.py#L134C51-L134C51