Closed bara-bba closed 2 years ago
It's difficult for me to debug this with certainty given the information, since it's not clear exactly what the JSON encoder is failing on.
Are you trying to pass the Trainer
as an argument to an experiment function (i.e. a function that has wrap_experiment
applied to it)?
I've tried to make that work, but I wouldn't be surprised if there were corner cases that still didn't work.
The final error here appears to be that isinstance(o, np.number)
is True
, and yet float(o)
crashes with float() argument must be a string or a number, not 'NoneType'
, which is quite strange. My guess is that somehow a np.timedelta64(None)
is being created, but I have no clue where it would be coming from. Perhaps such a value is being created inside your environment and you're passing it as an argument to your experiment function?
I've made a PR (linked above) that should hopefully fix this. Please install it and let me know if it does:
pip uninstall garage && pip install git+https://github.com/rlworkgroup/garage.git@fix-timedelta64-json
Thanks for your reply!
Unfortunately, I don't exactly understand what you mean by passing the Trainer
to the experiment function. I'm using almost the same code as sac_half_cheetah.py
:
#!/usr/bin/env python3
"""This is an example to train a task with SAC algorithm written in PyTorch."""
import numpy as np
import torch
from torch import nn
from torch.nn import functional as F
import warnings
from garage import wrap_experiment
from garage.envs import GymEnv, normalize
from garage.experiment import deterministic
from garage.replay_buffer import PathBuffer
from garage.sampler import FragmentWorker, LocalSampler, DefaultWorker
from garage.torch import set_gpu_mode
from garage.torch.algos import SAC
from garage.torch.policies import TanhGaussianMLPPolicy
from garage.torch.q_functions import ContinuousMLPQFunction
from garage.trainer import Trainer
from panda_env import PandaEnv
"""Snapshotter snapshots training data.
When training, it saves data to binary files. When resuming,
it loads from saved data.
Args:
snapshot_dir (str): Path to save the log and iteration snapshot.
snapshot_mode (str): Mode to save the snapshot. Can be either "all"
(all iterations will be saved), "last" (only the last iteration
will be saved), "gap" (every snapshot_gap iterations are saved),
"gap_and_last" (save the last iteration as 'params.pkl' and save
every snapshot_gap iteration separately), "gap_overwrite" (same as
gap but overwrites the last saved snapshot), or "none" (do not
save snapshots).
snapshot_gap (int): Gap between snapshot iterations. Wait this number
of iterations before taking another snapshot.
"""
@wrap_experiment(snapshot_mode='gap', snapshot_gap=100)
def garage_sac(ctxt=None, seed=1):
"""Set up environment and algorithm and run the task.
Args:
ctxt (garage.experiment.ExperimentContext): The experiment
configuration used by Trainer to create the snapshotter.
seed (int): Used to seed the random number generator to produce
determinism.
"""
deterministic.set_seed(seed)
trainer = Trainer(snapshot_config=ctxt)
env = normalize(GymEnv(PandaEnv(), max_episode_length=300), normalize_obs=True)
policy = TanhGaussianMLPPolicy(
env_spec=env.spec,
hidden_sizes=[256, 256],
hidden_nonlinearity=nn.ReLU,
output_nonlinearity=None,
min_std=np.exp(-20.),
max_std=np.exp(2.),
)
qf1 = ContinuousMLPQFunction(env_spec=env.spec,
hidden_sizes=[512, 512],
hidden_nonlinearity=F.relu)
qf2 = ContinuousMLPQFunction(env_spec=env.spec,
hidden_sizes=[512, 512],
hidden_nonlinearity=F.relu)
replay_buffer = PathBuffer(capacity_in_transitions=int(5e6),
env_spec=env.spec)
sampler = LocalSampler(agents=policy,
envs=env,
max_episode_length=env.spec.max_episode_length,
worker_class=DefaultWorker)
sac = SAC(env_spec=env.spec,
policy=policy,
policy_lr=0.001,
qf1=qf1,
qf2=qf2,
qf_lr=0.001,
sampler=sampler,
optimizer=torch.optim.Adam,
gradient_steps_per_itr=10,
replay_buffer=replay_buffer,
min_buffer_size=1e3,
num_evaluation_episodes=100,
target_update_tau=0.005,
discount=0.99,
buffer_batch_size=512,
initial_log_entropy=0.,
reward_scale=1.,
steps_per_epoch=1)
if torch.cuda.is_available():
set_gpu_mode(True)
else:
set_gpu_mode(False)
sac.to()
trainer.setup(algo=sac, env=env)
trainer.train(n_epochs=10000, batch_size=1024, plot=True)
s = np.random.randint(0, 1000)
garage_sac(seed=521)
I also tried to install your fix but I still get:
/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py:551: UserWarning: Archiving a launch repo larger than 8MiB. This may be slow. Set archive_launch_repo=False in wrap_experiment to disable this behavior.
warnings.warn('Archiving a launch repo larger than 8MiB. This may be '
2022-06-01 11:41:00 | [garage_sac] Logging to /home/bara/PycharmProjects/garage/data/local/experiment/garage_sac
/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/tensorflow_probability/python/__init__.py:57: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
if (distutils.version.LooseVersion(tf.__version__) <
/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/deterministic.py:36: UserWarning: Enabeling deterministic mode in PyTorch can have a performance impact when using GPU.
warnings.warn(
/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/torch/distributed/distributed_c10d.py:126: UserWarning: torch.distributed.reduce_op is deprecated, please use torch.distributed.ReduceOp instead
warnings.warn("torch.distributed.reduce_op is deprecated, please use "
Traceback (most recent call last):
File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 637, in _default_inner
return json.JSONEncoder.default(self, o)
File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/json/encoder.py", line 179, in default
raise TypeError(f'Object of type {o.__class__.__name__} '
TypeError: Object of type Trainer is not JSON serializable
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 637, in _default_inner
return json.JSONEncoder.default(self, o)
File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/json/encoder.py", line 179, in default
raise TypeError(f'Object of type {o.__class__.__name__} '
TypeError: Object of type NormalizedEnv is not JSON serializable
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 637, in _default_inner
return json.JSONEncoder.default(self, o)
File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/json/encoder.py", line 179, in default
raise TypeError(f'Object of type {o.__class__.__name__} '
TypeError: Object of type GymEnv is not JSON serializable
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 637, in _default_inner
return json.JSONEncoder.default(self, o)
File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/json/encoder.py", line 179, in default
raise TypeError(f'Object of type {o.__class__.__name__} '
TypeError: Object of type PandaEnv is not JSON serializable
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 637, in _default_inner
return json.JSONEncoder.default(self, o)
File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/json/encoder.py", line 179, in default
raise TypeError(f'Object of type {o.__class__.__name__} '
TypeError: Object of type module is not JSON serializable
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 637, in _default_inner
return json.JSONEncoder.default(self, o)
File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/json/encoder.py", line 179, in default
raise TypeError(f'Object of type {o.__class__.__name__} '
TypeError: Object of type module is not JSON serializable
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 637, in _default_inner
return json.JSONEncoder.default(self, o)
File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/json/encoder.py", line 179, in default
raise TypeError(f'Object of type {o.__class__.__name__} '
TypeError: Object of type module is not JSON serializable
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 637, in _default_inner
return json.JSONEncoder.default(self, o)
File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/json/encoder.py", line 179, in default
raise TypeError(f'Object of type {o.__class__.__name__} '
TypeError: Object of type module is not JSON serializable
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 637, in _default_inner
return json.JSONEncoder.default(self, o)
File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/json/encoder.py", line 179, in default
raise TypeError(f'Object of type {o.__class__.__name__} '
TypeError: Object of type dict is not JSON serializable
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 637, in _default_inner
return json.JSONEncoder.default(self, o)
File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/json/encoder.py", line 179, in default
raise TypeError(f'Object of type {o.__class__.__name__} '
TypeError: Object of type module is not JSON serializable
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 637, in _default_inner
return json.JSONEncoder.default(self, o)
File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/json/encoder.py", line 179, in default
raise TypeError(f'Object of type {o.__class__.__name__} '
TypeError: Object of type module is not JSON serializable
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 637, in _default_inner
return json.JSONEncoder.default(self, o)
File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/json/encoder.py", line 179, in default
raise TypeError(f'Object of type {o.__class__.__name__} '
TypeError: Object of type module is not JSON serializable
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 637, in _default_inner
return json.JSONEncoder.default(self, o)
File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/json/encoder.py", line 179, in default
raise TypeError(f'Object of type {o.__class__.__name__} '
TypeError: Object of type module is not JSON serializable
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 637, in _default_inner
return json.JSONEncoder.default(self, o)
File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/json/encoder.py", line 179, in default
raise TypeError(f'Object of type {o.__class__.__name__} '
TypeError: Object of type module is not JSON serializable
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 637, in _default_inner
return json.JSONEncoder.default(self, o)
File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/json/encoder.py", line 179, in default
raise TypeError(f'Object of type {o.__class__.__name__} '
TypeError: Object of type module is not JSON serializable
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 637, in _default_inner
return json.JSONEncoder.default(self, o)
File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/json/encoder.py", line 179, in default
raise TypeError(f'Object of type {o.__class__.__name__} '
TypeError: Object of type module is not JSON serializable
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "scripts/garage_sac.py", line 112, in <module>
garage_sac(seed=521)
File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 369, in __call__
result = self.function(ctxt, **kwargs)
File "scripts/garage_sac.py", line 108, in garage_sac
trainer.train(n_epochs=10000, batch_size=1024, plot=True)
File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/trainer.py", line 394, in train
dump_json(summary_file, self)
File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 464, in dump_json
json.dump(data,
File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/json/__init__.py", line 179, in dump
for chunk in iterable:
File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/json/encoder.py", line 438, in _iterencode
o = _default(o)
File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 611, in default
return self._default_inner(o)
File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 674, in _default_inner
data = {
File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 675, in <dictcomp>
k: self.default(v)
File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 611, in default
return self._default_inner(o)
File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 674, in _default_inner
data = {
File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 675, in <dictcomp>
k: self.default(v)
File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 611, in default
return self._default_inner(o)
File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 674, in _default_inner
data = {
File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 675, in <dictcomp>
k: self.default(v)
File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 611, in default
return self._default_inner(o)
File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 674, in _default_inner
data = {
File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 675, in <dictcomp>
k: self.default(v)
File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 611, in default
return self._default_inner(o)
File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 674, in _default_inner
data = {
File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 675, in <dictcomp>
k: self.default(v)
File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 611, in default
return self._default_inner(o)
File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 674, in _default_inner
data = {
File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 675, in <dictcomp>
k: self.default(v)
File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 611, in default
return self._default_inner(o)
File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 674, in _default_inner
data = {
File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 675, in <dictcomp>
k: self.default(v)
File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 611, in default
return self._default_inner(o)
File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 674, in _default_inner
data = {
File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 675, in <dictcomp>
k: self.default(v)
File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 611, in default
return self._default_inner(o)
File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 644, in _default_inner
data[k] = self.default(v)
File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 611, in default
return self._default_inner(o)
File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 674, in _default_inner
data = {
File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 675, in <dictcomp>
k: self.default(v)
File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 611, in default
return self._default_inner(o)
File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 674, in _default_inner
data = {
File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 675, in <dictcomp>
k: self.default(v)
File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 611, in default
return self._default_inner(o)
File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 674, in _default_inner
data = {
File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 675, in <dictcomp>
k: self.default(v)
File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 611, in default
return self._default_inner(o)
File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 674, in _default_inner
data = {
File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 675, in <dictcomp>
k: self.default(v)
File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 611, in default
return self._default_inner(o)
File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 674, in _default_inner
data = {
File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 675, in <dictcomp>
k: self.default(v)
File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 611, in default
return self._default_inner(o)
File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 674, in _default_inner
data = {
File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 675, in <dictcomp>
k: self.default(v)
File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 611, in default
return self._default_inner(o)
File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 674, in _default_inner
data = {
File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 674, in <dictcomp>
data = {
RuntimeError: dictionary changed size during iteration
Ok, I installed a previous version of gym
and now it is working. I think the main problem is in the changes with _mujoco_bindings
in gym==0.24.0
Ah, yeah. The recent gym changes have broken a lot of things. It appears I forgot about another one of the calls to dump_json
, which dumps the Trainer
, hence my confusion earlier. I've made a version that's even more robust now, so hopefully this issue shouldn't come up in the future.
Many thanks for the help!
Hi everyone!
I installed garage on my new pc but in doing so I encountered this error. My environment hasn't changed at all so I think it is due to a version incompatibility of packages.
pip list looks like this:
And this is the error:
Thanks in advance for the help!