DanielTakeshi commented 4 years ago

Contents:

RGBD (This post)
RGB only
Depth only

RGBD Stuff

Commands to run. We should test at the point when it's 5e4 time steps (for fairness with the earlier trials) but I may want to run just a little longer to see if performance will hit a limit. We need to use similar hyperparameters from the pre-print. For these, use https://github.com/DanielTakeshi/gym-cloth/commit/66de2eeff83ba15be4e7b0b3147bbded0a8213db (edit: https://github.com/DanielTakeshi/baselines-fork/commit/4bd7474fc41eb15231b32aeff7e140642df0716f) as that has all the configuration files set up appropriately, with use_rgbd='True', force_grab=False and so forth. We probably don't need 500 behavior cloning epochs, and I'm only doing this for consistency. Don't forget to compile gym-cloth!!

We should use policies that are stored on nfs:

(py3-iros-2020) seita@hermes1:/nfs/diskstation/seita/clothsim/policies $ ls -lh 
total 0
drwxr-xr-x 1 nobody nogroup 784 Aug 26 12:26 openai-2019-08-20-19-33-07-396753
drwxr-xr-x 1 nobody nogroup 784 Aug 26 12:27 openai-2019-08-20-19-36-05-376245
drwxr-xr-x 1 nobody nogroup 784 Aug 26 12:28 openai-2019-08-23-10-47-57-805603
drwxr-xr-x 1 nobody nogroup 784 Aug 26 12:29 openai-2019-08-23-11-05-29-629583
drwxr-xr-x 1 nobody nogroup 784 Aug 29 21:42 openai-2019-08-28-09-21-16-672897
drwxr-xr-x 1 nobody nogroup 194 Aug 29 20:44 openai-2019-08-29-19-37-22-836707
drwxr-xr-x 1 nobody nogroup 194 Aug 29 20:45 openai-2019-08-29-19-50-50-101687
drwxr-xr-x 1 nobody nogroup 194 Aug 29 21:07 openai-2019-08-29-20-14-36-620560
drwxr-xr-x 1 nobody nogroup 482 Sep  1 09:56 openai-2019-08-30-21-39-05-868283
drwxrwxr-x 1 nobody nogroup 482 Sep  2 14:40 openai-2019-08-31-22-42-10-834308
drwxrwxr-x 1 nobody nogroup 494 Sep  2 14:42 openai-2019-09-01-11-10-28-087733
drwxr-xr-x 1 nobody nogroup 494 Sep  4 10:03 openai-2019-09-01-20-37-45-609860
drwxrwxr-x 1 nobody nogroup 482 Sep  4 08:56 openai-2019-09-02-19-25-40-802588
drwxrwxr-x 1 nobody nogroup 494 Sep  4 08:57 openai-2019-09-02-19-30-13-323241
drwxrwxr-x 1 nobody nogroup 482 Sep  4 18:40 openai-2019-09-02-23-01-55-146225
drwxrwxr-x 1 nobody nogroup 494 Sep  4 18:42 openai-2019-09-02-23-03-41-443793
drwxrwxr-x 1 nobody nogroup 482 Sep  6 14:45 openai-2019-09-04-12-56-54-072948
drwxrwxr-x 1 nobody nogroup 814 Oct 24 08:32 openai-2019-10-17-11-40-45-049240
drwxrwxr-x 1 nobody nogroup 814 Oct 24 08:34 openai-2019-10-21-13-06-42-294761
(py3-iros-2020) seita@hermes1:/nfs/diskstation/seita/clothsim/policies $

The above has the policies that were trained before these three, and these three will go in the same folder above.

Physical rollout results are stored here:

(py2-clothenv) seita@hermes1:~/dvrk-python-iros2020 (master) $ ls -lh results/
total 140K
drwxr-xr-x 2 seita seita 4.0K Feb 16 10:50 img_tier1_color
drwxr-xr-x 2 seita seita 4.0K Feb 16 10:50 img_tier1_color_yellowcloth
drwxr-xr-x 2 seita seita  12K Feb 16 10:50 img_tier1_depth
drwxr-xr-x 2 seita seita 4.0K Feb 16 10:50 img_tier1_depth_yellowcloth
drwxr-xr-x 2 seita seita  16K Feb 16 10:50 img_tier2_color
drwxr-xr-x 2 seita seita  20K Feb 16 10:50 img_tier2_depth
drwxr-xr-x 2 seita seita  20K Feb 16 10:50 img_tier3_color
drwxr-xr-x 2 seita seita  20K Feb 16 10:50 img_tier3_depth
-rw-r--r-- 1 seita seita  107 Feb 16 10:46 README.md
drwxr-xr-x 8 seita seita 4.0K Feb 16 10:46 results
drwxr-xr-x 2 seita seita 4.0K Feb 16 10:46 tier1_color
drwxr-xr-x 2 seita seita 4.0K Feb 16 10:46 tier1_color_yellowcloth
drwxr-xr-x 2 seita seita 4.0K Feb 16 10:46 tier1_depth
drwxr-xr-x 2 seita seita 4.0K Feb 16 10:46 tier1_depth_yellowcloth
drwxr-xr-x 2 seita seita 4.0K Feb 16 10:46 tier2_color
drwxr-xr-x 2 seita seita 4.0K Feb 16 10:46 tier2_depth
drwxr-xr-x 2 seita seita 4.0K Feb 16 10:46 tier3_color
drwxr-xr-x 2 seita seita 4.0K Feb 16 10:46 tier3_depth

so we should be adding more to the above, with tier_{1,2,3}_rgbd folders. Also, run python analysis.py from https://github.com/DanielTakeshi/dvrk-python (requires a separate Python 2.7 virtualenv, sorry) to analyze results. The IMAGES are also saved above, that's how we can check the episodes.

Tier 1 [Experiment 5.0.0a]

Running on mason Feb 11, after https://github.com/DanielTakeshi/baselines-fork/commit/4bd7474fc41eb15231b32aeff7e140642df0716f

See openai-2020-02-11-15-52-56-391653

python -m baselines.run --alg=imit --env=Cloth-v0 --num_env=10 --num_timesteps=6e4 \
        --cloth_config=../gym-cloth/cfg/t1_rgbd.yaml  --rb_size=50000  --bc_epochs=500 \
        --demos_path=../gym-cloth/logs/demos-2020-02-09-16-31-pol-oracle-seed-1336_to_1340-obs-blender-depth-False-rgbd-True-tier1_epis_2000_COMBO.pkl   \
        --actor_l2_reg=1e-5  --nb_rollout_steps=20  --nb_train_steps=240

LGTM.

-------------------------------------
| memory/nb_entries      | 5e+04    |
| rollout/actions_mean   | 8.18e-05 |
| rollout/actions_std    | 0.456    |
| rollout/episode_steps  | 4.1      |
| rollout/episodes       | 65       |
| rollout/return         | 4.28     |
| rollout/return_history | 4.84     |
| total/duration         | 3.55e+05 |
| total/episodes         | 1.46e+04 |
| total/epochs           | 300      |
| total/steps_per_env    | 6000     |
| total/steps_per_second | 0.0169   |
| train/loss_actor       | 0.00893  |
| train/loss_actor_l2    | 0.00532  |
-------------------------------------

We are now saving stuff!!
Saving model checkpoint to:  /tmp/openai-2020-02-11-15-52-56-391653/checkpoints/00299
(py3-iros-2020) seita@mason:~/baselines-fork (master) $

Tier 2 [Experiment 5.0.0b]

Running on mason, Feb 12, see openai-2020-02-12-14-11-03-092908. Same commit as Tier1.

python -m baselines.run --alg=imit --env=Cloth-v0 --num_env=10 --num_timesteps=6e4 \
        --cloth_config=../gym-cloth/cfg/t2_rgbd.yaml --rb_size=50000 --bc_epochs=500 \
        --demos_path=../gym-cloth/logs/demos-2020-02-10-15-02-pol-oracle-seed-1336_to_1340-obs-blender-depth-False-rgbd-True-tier2_epis_2000_COMBO.pkl    \
        --actor_l2_reg=1e-5  --nb_rollout_steps=20  --nb_train_steps=240

and after forever, it gave me:

-------------------------------------
| memory/nb_entries      | 5e+04    |
| rollout/actions_mean   | -0.0031  |
| rollout/actions_std    | 0.414    |
| rollout/episode_steps  | 6.22     |
| rollout/episodes       | 43       |
| rollout/return         | 3.63     |
| rollout/return_history | 4.36     |
| total/duration         | 4.52e+05 |
| total/episodes         | 9.64e+03 |
| total/epochs           | 300      |
| total/steps_per_env    | 6000     |
| total/steps_per_second | 0.0133   |
| train/loss_actor       | 0.00882  |
| train/loss_actor_l2    | 0.0053   |
-------------------------------------

We are now saving stuff!!
Saving model checkpoint to:  /tmp/openai-2020-02-12-14-11-03-092908/checkpoints/00299

real    7572m3.825s
user    25535m21.114s
sys     403m52.547s
(py3-iros-2020) seita@mason:~/baselines-fork (master) $ time python -m baselines.run --alg=imit --env=Cloth-v0 --num_env=10 --num_timesteps=6e4         --cloth_config=../gym-cloth/cfg/t2_rgbd.yaml --rb_size=50000 --bc_epochs=500         --demos_path=../gym-cloth/logs/demos-2020-02-10-15-02-pol-oracle-seed-1336_to_1340-obs-blender-depth-False-rgbd-True-tier2_epis_2000_COMBO.pkl            --actor_l2_reg=1e-5  --nb_rollout_steps=20  --nb_train_steps=240

Tier 3 [Experiment 5.0.0c]

Running on mason, evening of Feb 15, see openai-2020-02-15-19-37-59-883528. Same commit as Tiers 1, 2.

python -m baselines.run --alg=imit --env=Cloth-v0 --num_env=10 --num_timesteps=6e4 \
        --cloth_config=../gym-cloth/cfg/t3_rgbd.yaml --rb_size=50000 --bc_epochs=500 \
        --demos_path=../gym-cloth/logs/demos-2020-02-10-15-05-pol-oracle-seed-1336_to_1340-obs-blender-depth-False-rgbd-True-tier3_epis_2000_COMBO.pkl  \
        --actor_l2_reg=1e-5  --nb_rollout_steps=20  --nb_train_steps=240

and gave me this at the end:

-------------------------------------
| memory/nb_entries      | 5e+04    |
| rollout/actions_mean   | -0.00309 |
| rollout/actions_std    | 0.413    |
| rollout/episode_steps  | 8.39     |
| rollout/episodes       | 25       |
| rollout/return         | 2.74     |
| rollout/return_history | 3.27     |
| total/duration         | 2.93e+05 |
| total/episodes         | 7.14e+03 |
| total/epochs           | 300      |
| total/steps_per_env    | 6000     |
| total/steps_per_second | 0.0205   |
| train/loss_actor       | 0.00731  |
| train/loss_actor_l2    | 0.00492  |
-------------------------------------

We are now saving stuff!!
Saving model checkpoint to:  /tmp/openai-2020-02-15-19-37-59-883528/checkpoints/00299

real    4930m28.847s
user    19662m56.939s
sys     405m6.901s
(py3-iros-2020) seita@mason:~/baselines-fork (master) $ time python -m baselines.run --alg=imit --env=Cloth-v0 --num_env=10 --num_timesteps=6e4         --cloth_config=../gym-cloth/cfg/t3_rgbd.yaml --rb_size=50000 --bc_epochs=500         --demos_path=../gym-cloth/logs/demos-2020-02-10-15-05-pol-oracle-seed-1336_to_1340-obs-blender-depth-False-rgbd-True-tier3_epis_2000_COMBO.pkl          --actor_l2_reg=1e-5  --nb_rollout_steps=20  --nb_train_steps=240

DanielTakeshi commented 4 years ago

Training Run Plots

These don't go in the paper and are strictly for debugging / inspection.

DanielTakeshi commented 4 years ago

Simulated Results

Here, I roll out policies to get simulated results.

We may also consider rolling out the prior results, if we have changed the domain randomization settings.

For RGBD rollouts, use this: https://github.com/DanielTakeshi/gym-cloth/commit/d102476e16c7ecd2bce59894aa18ca43dc531431 with exact configs for each tier.
And use this: https://github.com/DanielTakeshi/baselines-fork/commit/89ee8f11738cbc65ede5128b970e5af73e21eb44

Run ./scripts/test_5-0-0.sh and comment out any scripts that I don't need.

Here is Tier 1:

 ************ FINISHED EPISODE, done: [ True  True  True False  True  True False False  True  True] ************
episode_rews: [ 5.08868793  5.10905308  5.20396504  0.07435006  5.21350688  5.03646509 -0.03652244  0.17665543  5.12530883  5.28954401]
And epinfo: ({'num_steps': 1, 'num_sim_steps': 1508, 'actual_coverage': 0.9337404333584286, 'start_coverage': 0.8450524993096872, 'variance_inv': 2.0050895362737817, 'start_variance_inv': 1.0623698871627956, 'have_tear': False, 'out_of_bounds': False, 'episode': {'r': 5.088688, 'l': 1, 't': 508.738849}}, {'num_steps': 1, 'num_sim_steps': 1596, 'actual_coverage': 0.9380414189552327, 'start_coverage': 0.8289883428027073, 'variance_inv': 3.0295910299598274, 'start_variance_inv': 0.7672108787604458, 'have_tear': False, 'out_of_bounds': False, 'episode': {'r': 5.109053, 'l': 1, 't': 502.757832}}, {'num_steps': 2, 'num_sim_steps': 3079, 'actual_coverage': 0.9362099325319958, 'start_coverage': 0.7322448936193338, 'variance_inv': 3.369133576709013, 'start_variance_inv': 0.8532728892553803, 'have_tear': False, 'out_of_bounds': False, 'episode': {'r': 5.203965, 'l': 2, 't': 508.63038}}, {'num_steps': 2, 'num_sim_steps': 2989, 'actual_coverage': 0.9043328995143208, 'start_coverage': 0.829982834609221, 'variance_inv': 1.5429741495466127, 'start_variance_inv': 1.0278553548815021, 'have_tear': False, 'out_of_bounds': False}, {'num_steps': 2, 'num_sim_steps': 3119, 'actual_coverage': 0.9766314477331864, 'start_coverage': 0.7631245677043283, 'variance_inv': 6.439669545946714, 'start_variance_inv': 0.6597400596296513, 'have_tear': False, 'out_of_bounds': False, 'episode': {'r': 5.213507, 'l': 2, 't': 509.453544}}, {'num_steps': 1, 'num_sim_steps': 1523, 'actual_coverage': 0.9226612436163785, 'start_coverage': 0.8861961578996195, 'variance_inv': 3.283224602309535, 'start_variance_inv': 1.5661466352340538, 'have_tear': False, 'out_of_bounds': False, 'episode': {'r': 5.036465, 'l': 1, 't': 507.135245}}, {'num_steps': 3, 'num_sim_steps': 4758, 'actual_coverage': 0.6319296683851909, 'start_coverage': 0.6684521060748212, 'variance_inv': 1.4130080961582794, 'start_variance_inv': 0.6460046779299087, 'have_tear': False, 'out_of_bounds': False}, {'num_steps': 2, 'num_sim_steps': 3120, 'actual_coverage': 0.8541753242390293, 'start_coverage': 0.677519891616005, 'variance_inv': 2.197493666152759, 'start_variance_inv': 0.5763676515286619, 'have_tear': False, 'out_of_bounds': False}, {'num_steps': 1, 'num_sim_steps': 1567, 'actual_coverage': 0.9803064455441693, 'start_coverage': 0.8549976204091334, 'variance_inv': 10.622549849939706, 'start_variance_inv': 1.035190432645554, 'have_tear': False, 'out_of_bounds': False, 'episode': {'r': 5.125309, 'l': 1, 't': 507.95574}}, {'num_steps': 4, 'num_sim_steps': 6187, 'actual_coverage': 0.9766279134580443, 'start_coverage': 0.6870839043637678, 'variance_inv': 26.655565300772274, 'start_variance_inv': 0.7148369007649874, 'have_tear': False, 'out_of_bounds': False, 'episode': {'r': 5.289544, 'l': 4, 't': 509.024559}}) (len 10)

Stats only for 52 completed episodes:
Play rewards: 5.076 +/- 0.7
rewards max/min/median: 5.319, 0.197, 5.159
Num steps : 2.365 +/- 1.7
Start inv-var:  0.898 +/- 0.4
Final inv-var:  9.894 +/- 17.0
Start coverage: 0.778 +/- 0.1
Final coverage: 0.950 +/- 0.0
Final coverage max/min/median: 0.991, 0.879, 0.947
Out of bounds total: 0

Num exceeding coverage thresh: 51 / 52

DONE w/50 epis, breaking ...

saving at: logs/policy-rollout-imit-50-epis-tier1-seed-1600-depth-False-forcegrab-True-stats-2020-02-17-18-10.pkl

real    198m58.494s
user    1024m14.751s
sys     8m30.392s
(py3-iros-2020) seita@hermes1:~/baselines-fork-iros2020 (master) $

T2:

 ************ FINISHED EPISODE, done: [False  True False  True False False  True False False False] ************
episode_rews: [-0.06039422  5.38926302 -0.07673253  0.42935294  0.19306459  0.02002805  5.35780898 -0.20735234  0.02979321  0.12697387]
And epinfo: ({'num_steps': 1, 'num_sim_steps': 1722, 'actual_coverage': 0.5346742820326489, 'start_coverage': 0.5950685033018015, 'variance_inv': 0.6252027287930368, 'start_variance_inv': 0.8519691408755706, 'have_tear': False, 'out_of_bounds': False}, {'num_steps': 2, 'num_sim_steps': 3207, 'actual_coverage': 0.9674070647229567, 'start_coverage': 0.5781440417573387, 'variance_inv': 8.30131751457255, 'start_variance_inv': 0.9555113551623438, 'have_tear': False, 'out_of_bounds': False, 'episode': {'r': 5.389263, 'l': 2, 't': 1365.796101}}, {'num_steps': 4, 'num_sim_steps': 6494, 'actual_coverage': 0.6289036579845914, 'start_coverage': 0.7056361839501307, 'variance_inv': 0.7883408545906985, 'start_variance_inv': 1.316286808924844, 'have_tear': False, 'out_of_bounds': False}, {'num_steps': 10, 'num_sim_steps': 15935, 'actual_coverage': 0.918898988780477, 'start_coverage': 0.489546051621854, 'variance_inv': 2.3144836655915917, 'start_variance_inv': 0.7832482430989415, 'have_tear': False, 'out_of_bounds': False, 'episode': {'r': 0.429353, 'l': 10, 't': 1365.941111}}, {'num_steps': 7, 'num_sim_steps': 11065, 'actual_coverage': 0.8480271318373896, 'start_coverage': 0.6549625442420514, 'variance_inv': 1.5253044960650155, 'start_variance_inv': 0.7085242888123907, 'have_tear': False, 'out_of_bounds': False}, {'num_steps': 2, 'num_sim_steps': 3461, 'actual_coverage': 0.4850773170396436, 'start_coverage': 0.4650492698861566, 'variance_inv': 0.7091268993912823, 'start_variance_inv': 0.6144289825868093, 'have_tear': False, 'out_of_bounds': False}, {'num_steps': 3, 'num_sim_steps': 4806, 'actual_coverage': 0.93906649440613, 'start_coverage': 0.5812575105134888, 'variance_inv': 11.24396758733423, 'start_variance_inv': 0.7755086587699376, 'have_tear': False, 'out_of_bounds': False, 'episode': {'r': 5.357809, 'l': 3, 't': 1365.88492}}, {'num_steps': 6, 'num_sim_steps': 9899, 'actual_coverage': 0.3548518132953206, 'start_coverage': 0.5622041566836247, 'variance_inv': 0.6493919522795469, 'start_variance_inv': 0.6222631905153421, 'have_tear': False, 'out_of_bounds': False}, {'num_steps': 7, 'num_sim_steps': 11716, 'actual_coverage': 0.5191718544585274, 'start_coverage': 0.4893786410675045, 'variance_inv': 0.6948412823410318, 'start_variance_inv': 0.544111065031615, 'have_tear': False, 'out_of_bounds': False}, {'num_steps': 5, 'num_sim_steps': 8218, 'actual_coverage': 0.7530946300801848, 'start_coverage': 0.6261207592740208, 'variance_inv': 1.0921586253153996, 'start_variance_inv': 0.877554249954281, 'have_tear': False, 'out_of_bounds': False}) (len 10)

Stats only for 52 completed episodes:
Play rewards: 3.962 +/- 2.3
rewards max/min/median: 5.489, -0.190, 5.322
Num steps : 5.731 +/- 3.1
Start inv-var:  0.750 +/- 0.1
Final inv-var:  5.580 +/- 7.1
Start coverage: 0.568 +/- 0.1
Final coverage: 0.876 +/- 0.1
Final coverage max/min/median: 0.972, 0.329, 0.934
Out of bounds total: 0

Num exceeding coverage thresh: 38 / 52

DONE w/50 epis, breaking ...

saving at: logs/policy-rollout-imit-50-epis-tier2-seed-1600-depth-False-forcegrab-True-stats-2020-02-18-19-15.pkl

real    326m48.553s
user    1567m41.955s
sys     10m20.773s

And T3:

 ************ FINISHED EPISODE, done: [False False False  True False False False  True  True False] ************
episode_rews: [0.42377173 0.17474674 0.15924494 5.54163373 0.42816038 0.10072608 0.14432273 5.5241958  0.42936058 0.51953218]
And epinfo: ({'num_steps': 4, 'num_sim_steps': 6468, 'actual_coverage': 0.8092379938451001, 'start_coverage': 0.3854662598783713, 'variance_inv': 1.1549245702645747, 'start_variance_inv': 0.2574274840512516, 'have_tear': False, 'out_of_bounds': False}, {'num_steps': 1, 'num_sim_steps': 1672, 'actual_coverage': 0.608875044962432, 'start_coverage': 0.43412830277624553, 'variance_inv': 0.5281474022441482, 'start_variance_inv': 0.15368056768235155, 'have_tear': False, 'out_of_bounds': False}, {'num_steps': 1, 'num_sim_steps': 1658, 'actual_coverage': 0.5085732966976535, 'start_coverage': 0.34932835257567757, 'variance_inv': 0.39995000917701806, 'start_variance_inv': 0.2247842061138652, 'have_tear': False, 'out_of_bounds': False}, {'num_steps': 6, 'num_sim_steps': 9357, 'actual_coverage': 0.9406900618778342, 'start_coverage': 0.3990563330048989, 'variance_inv': 3.40791541454813, 'start_variance_inv': 0.2478578946053148, 'have_tear': False, 'out_of_bounds': False, 'episode': {'r': 5.541634, 'l': 6, 't': 1153.498817}}, {'num_steps': 6, 'num_sim_steps': 9343, 'actual_coverage': 0.8598790424468741, 'start_coverage': 0.4317186634766831, 'variance_inv': 1.3396568882658633, 'start_variance_inv': 0.13498994711123852, 'have_tear': False, 'out_of_bounds': False}, {'num_steps': 1, 'num_sim_steps': 1672, 'actual_coverage': 0.5169579954082666, 'start_coverage': 0.41623191228173606, 'variance_inv': 0.5242268609992154, 'start_variance_inv': 0.11849796115619161, 'have_tear': False, 'out_of_bounds': False}, {'num_steps': 3, 'num_sim_steps': 4832, 'actual_coverage': 0.5956802589915232, 'start_coverage': 0.4513575297145415, 'variance_inv': 0.555546676515546, 'start_variance_inv': 0.2397815698311404, 'have_tear': False, 'out_of_bounds': False}, {'num_steps': 9, 'num_sim_steps': 14074, 'actual_coverage': 0.9400771815360134, 'start_coverage': 0.41588138515411543, 'variance_inv': 3.882631019144772, 'start_variance_inv': 0.2075805841419052, 'have_tear': False, 'out_of_bounds': False, 'episode': {'r': 5.524196, 'l': 9, 't': 1159.316844}}, {'num_steps': 10, 'num_sim_steps': 15307, 'actual_coverage': 0.8441721694555943, 'start_coverage': 0.41481158936984125, 'variance_inv': 1.4940638741003875, 'start_variance_inv': 0.24913519923198943, 'have_tear': False, 'out_of_bounds': False, 'episode': {'r': 0.429361, 'l': 10, 't': 1153.637789}}, {'num_steps': 9, 'num_sim_steps': 14144, 'actual_coverage': 0.9060128845661931, 'start_coverage': 0.3864807067161985, 'variance_inv': 2.4709858473301014, 'start_variance_inv': 0.22986491192909586, 'have_tear': False, 'out_of_bounds': False}) (len 10)

Stats only for 51 completed episodes:
Play rewards: 4.436 +/- 2.1
rewards max/min/median: 5.629, 0.364, 5.519
Num steps : 7.353 +/- 2.1
Start inv-var:  0.230 +/- 0.1
Final inv-var:  6.392 +/- 10.5
Start coverage: 0.409 +/- 0.0
Final coverage: 0.923 +/- 0.0
Final coverage max/min/median: 0.972, 0.794, 0.940
Out of bounds total: 0

Num exceeding coverage thresh: 40 / 51

DONE w/50 epis, breaking ...

saving at: logs/policy-rollout-imit-50-epis-tier3-seed-1600-depth-False-forcegrab-True-stats-2020-02-19-20-36.pkl

real    267m56.694s
user    1634m29.193s
sys     12m38.246s
(py3-iros-2020) seita@hermes1:~/baselines-fork-iros2

The Data Logs

The logs can be found here:

-rw-r--r-- 1 seita seita 5.6K Feb 17 15:11 policy-rollout-imit-50-epis-tier1-seed-1600-depth-False-forcegrab-True-stats-2020-02-17-15-11.pkl
-rw-r--r-- 1 seita seita 5.6K Feb 17 15:30 policy-rollout-imit-50-epis-tier1-seed-1600-depth-False-forcegrab-True-stats-2020-02-17-15-30.pkl
-rw-r--r-- 1 seita seita 5.6K Feb 17 15:48 policy-rollout-imit-50-epis-tier1-seed-1600-depth-False-forcegrab-True-stats-2020-02-17-15-48.pkl
-rw-r--r-- 1 seita seita 5.8K Feb 17 16:06 policy-rollout-imit-50-epis-tier1-seed-1600-depth-False-forcegrab-True-stats-2020-02-17-16-06.pkl
-rw-r--r-- 1 seita seita 5.6K Feb 17 16:23 policy-rollout-imit-50-epis-tier1-seed-1600-depth-False-forcegrab-True-stats-2020-02-17-16-23.pkl
-rw-r--r-- 1 seita seita 5.9K Feb 17 16:41 policy-rollout-imit-50-epis-tier1-seed-1600-depth-False-forcegrab-True-stats-2020-02-17-16-41.pkl
-rw-r--r-- 1 seita seita 5.6K Feb 17 16:55 policy-rollout-imit-50-epis-tier1-seed-1600-depth-False-forcegrab-True-stats-2020-02-17-16-55.pkl
-rw-r--r-- 1 seita seita 5.6K Feb 17 17:10 policy-rollout-imit-50-epis-tier1-seed-1600-depth-False-forcegrab-True-stats-2020-02-17-17-10.pkl
-rw-r--r-- 1 seita seita 5.7K Feb 17 17:24 policy-rollout-imit-50-epis-tier1-seed-1600-depth-False-forcegrab-True-stats-2020-02-17-17-24.pkl
-rw-r--r-- 1 seita seita 5.6K Feb 17 17:37 policy-rollout-imit-50-epis-tier1-seed-1600-depth-False-forcegrab-True-stats-2020-02-17-17-37.pkl
-rw-r--r-- 1 seita seita 5.7K Feb 17 17:49 policy-rollout-imit-50-epis-tier1-seed-1600-depth-False-forcegrab-True-stats-2020-02-17-17-49.pkl
-rw-r--r-- 1 seita seita 5.7K Feb 17 18:01 policy-rollout-imit-50-epis-tier1-seed-1600-depth-False-forcegrab-True-stats-2020-02-17-18-01.pkl
-rw-r--r-- 1 seita seita 5.8K Feb 17 18:10 policy-rollout-imit-50-epis-tier1-seed-1600-depth-False-forcegrab-True-stats-2020-02-17-18-10.pkl
-rw-r--r-- 1 seita seita 5.8K Feb 18 14:15 policy-rollout-imit-50-epis-tier2-seed-1600-depth-False-forcegrab-True-stats-2020-02-18-14-15.pkl
-rw-r--r-- 1 seita seita 5.6K Feb 18 14:41 policy-rollout-imit-50-epis-tier2-seed-1600-depth-False-forcegrab-True-stats-2020-02-18-14-41.pkl
-rw-r--r-- 1 seita seita 5.7K Feb 18 15:08 policy-rollout-imit-50-epis-tier2-seed-1600-depth-False-forcegrab-True-stats-2020-02-18-15-08.pkl
-rw-r--r-- 1 seita seita 5.6K Feb 18 15:34 policy-rollout-imit-50-epis-tier2-seed-1600-depth-False-forcegrab-True-stats-2020-02-18-15-34.pkl
-rw-r--r-- 1 seita seita 5.6K Feb 18 15:59 policy-rollout-imit-50-epis-tier2-seed-1600-depth-False-forcegrab-True-stats-2020-02-18-15-59.pkl
-rw-r--r-- 1 seita seita 5.6K Feb 18 16:24 policy-rollout-imit-50-epis-tier2-seed-1600-depth-False-forcegrab-True-stats-2020-02-18-16-24.pkl
-rw-r--r-- 1 seita seita 5.7K Feb 18 16:51 policy-rollout-imit-50-epis-tier2-seed-1600-depth-False-forcegrab-True-stats-2020-02-18-16-51.pkl
-rw-r--r-- 1 seita seita 5.7K Feb 18 17:18 policy-rollout-imit-50-epis-tier2-seed-1600-depth-False-forcegrab-True-stats-2020-02-18-17-18.pkl
-rw-r--r-- 1 seita seita 5.9K Feb 18 17:42 policy-rollout-imit-50-epis-tier2-seed-1600-depth-False-forcegrab-True-stats-2020-02-18-17-42.pkl
-rw-r--r-- 1 seita seita 5.7K Feb 18 18:06 policy-rollout-imit-50-epis-tier2-seed-1600-depth-False-forcegrab-True-stats-2020-02-18-18-06.pkl
-rw-r--r-- 1 seita seita 6.2K Feb 18 18:30 policy-rollout-imit-50-epis-tier2-seed-1600-depth-False-forcegrab-True-stats-2020-02-18-18-30.pkl
-rw-r--r-- 1 seita seita 5.6K Feb 18 18:52 policy-rollout-imit-50-epis-tier2-seed-1600-depth-False-forcegrab-True-stats-2020-02-18-18-52.pkl
-rw-r--r-- 1 seita seita 5.8K Feb 18 19:15 policy-rollout-imit-50-epis-tier2-seed-1600-depth-False-forcegrab-True-stats-2020-02-18-19-15.pkl
-rw-r--r-- 1 seita seita 5.6K Feb 19 16:26 policy-rollout-imit-50-epis-tier3-seed-1600-depth-False-forcegrab-True-stats-2020-02-19-16-26.pkl
-rw-r--r-- 1 seita seita 5.7K Feb 19 16:47 policy-rollout-imit-50-epis-tier3-seed-1600-depth-False-forcegrab-True-stats-2020-02-19-16-47.pkl
-rw-r--r-- 1 seita seita 5.6K Feb 19 17:07 policy-rollout-imit-50-epis-tier3-seed-1600-depth-False-forcegrab-True-stats-2020-02-19-17-07.pkl
-rw-r--r-- 1 seita seita 5.7K Feb 19 17:27 policy-rollout-imit-50-epis-tier3-seed-1600-depth-False-forcegrab-True-stats-2020-02-19-17-27.pkl
-rw-r--r-- 1 seita seita 5.6K Feb 19 17:48 policy-rollout-imit-50-epis-tier3-seed-1600-depth-False-forcegrab-True-stats-2020-02-19-17-48.pkl
-rw-r--r-- 1 seita seita 5.8K Feb 19 18:10 policy-rollout-imit-50-epis-tier3-seed-1600-depth-False-forcegrab-True-stats-2020-02-19-18-10.pkl
-rw-r--r-- 1 seita seita 5.8K Feb 19 18:31 policy-rollout-imit-50-epis-tier3-seed-1600-depth-False-forcegrab-True-stats-2020-02-19-18-31.pkl
-rw-r--r-- 1 seita seita 5.6K Feb 19 18:54 policy-rollout-imit-50-epis-tier3-seed-1600-depth-False-forcegrab-True-stats-2020-02-19-18-54.pkl
-rw-r--r-- 1 seita seita 5.6K Feb 19 19:14 policy-rollout-imit-50-epis-tier3-seed-1600-depth-False-forcegrab-True-stats-2020-02-19-19-14.pkl
-rw-r--r-- 1 seita seita 5.6K Feb 19 19:35 policy-rollout-imit-50-epis-tier3-seed-1600-depth-False-forcegrab-True-stats-2020-02-19-19-35.pkl
-rw-r--r-- 1 seita seita 5.8K Feb 19 19:56 policy-rollout-imit-50-epis-tier3-seed-1600-depth-False-forcegrab-True-stats-2020-02-19-19-56.pkl
-rw-r--r-- 1 seita seita 5.6K Feb 19 20:16 policy-rollout-imit-50-epis-tier3-seed-1600-depth-False-forcegrab-True-stats-2020-02-19-20-16.pkl
-rw-r--r-- 1 seita seita 5.7K Feb 19 20:36 policy-rollout-imit-50-epis-tier3-seed-1600-depth-False-forcegrab-True-stats-2020-02-19-20-36.pkl

I also put them on NFS in seita/clothsin/rollouts_5.0.0 under (a,b,c).

DanielTakeshi commented 4 years ago

RGB only -- Use branch w/three channel RGB (62eca3109656d6a161795dccda0a0d9411536f76)

If I do this, then use the commands, and ensure that the dataset is the SAME as what we are using for RGBD. Here we use 50k steps, since I only did 60k earlier for RGBD to see how well it'd do with more steps (I still evaluate with 50k).

T1 RGB (see `openai-2020-02-19-16-35-04-032759`)

python -m baselines.run --alg=imit --env=Cloth-v0 --num_env=10 --num_timesteps=5e4 \
        --cloth_config=../gym-cloth/cfg/demo_baselines_fixed_t1_color.yaml --rb_size=50000 \
        --demos_path=../gym-cloth/logs/demos-2020-02-09-16-31-pol-oracle-seed-1336_to_1340-obs-blender-depth-False-rgbd-True-tier1_epis_2000_COMBO.pkl   \
        --bc_epochs=500  --actor_l2_reg=1e-5  --nb_rollout_steps=20  --nb_train_steps=240

T2 RGB (see `openai-2020-02-19-16-39-21-629526`)

python -m baselines.run --alg=imit --env=Cloth-v0 --num_env=10 --num_timesteps=5e4 \
        --cloth_config=../gym-cloth/cfg/demo_baselines_fixed_t2_color.yaml --rb_size=50000 \
        --demos_path=../gym-cloth/logs/demos-2020-02-10-15-02-pol-oracle-seed-1336_to_1340-obs-blender-depth-False-rgbd-True-tier2_epis_2000_COMBO.pkl    \
        --bc_epochs=500  --actor_l2_reg=1e-5  --nb_rollout_steps=20  --nb_train_steps=240

T3 RGB (see `openai-2020-02-20-21-35-16-544757`)

python -m baselines.run --alg=imit --env=Cloth-v0 --num_env=10 --num_timesteps=5e4 \
        --cloth_config=../gym-cloth/cfg/demo_baselines_fixed_t3_color.yaml --rb_size=50000 \
        --demos_path=../gym-cloth/logs/demos-2020-02-10-15-05-pol-oracle-seed-1336_to_1340-obs-blender-depth-False-rgbd-True-tier3_epis_2000_COMBO.pkl  \
        --bc_epochs=500  --actor_l2_reg=1e-5  --nb_rollout_steps=20  --nb_train_steps=240

DanielTakeshi commented 4 years ago

...

DanielTakeshi commented 4 years ago

...

DanielTakeshi commented 4 years ago

Depth Only

T1 D

python -m baselines.run --alg=imit --env=Cloth-v0 --num_env=10 --num_timesteps=5e4 \
        --cloth_config=../gym-cloth/cfg/demo_baselines_fixed_t1_depth.yaml --rb_size=50000 \
        --demos_path=../gym-cloth/logs/demos-2020-02-09-16-31-pol-oracle-seed-1336_to_1340-obs-blender-depth-False-rgbd-True-tier1_epis_2000_COMBO.pkl   \
        --bc_epochs=500  --actor_l2_reg=1e-5  --nb_rollout_steps=20  --nb_train_steps=240

T2 D

python -m baselines.run --alg=imit --env=Cloth-v0 --num_env=10 --num_timesteps=5e4 \
        --cloth_config=../gym-cloth/cfg/demo_baselines_fixed_t2_depth.yaml --rb_size=50000 \
        --demos_path=../gym-cloth/logs/demos-2020-02-10-15-02-pol-oracle-seed-1336_to_1340-obs-blender-depth-False-rgbd-True-tier2_epis_2000_COMBO.pkl    \
        --bc_epochs=500  --actor_l2_reg=1e-5  --nb_rollout_steps=20  --nb_train_steps=240

T3 D

python -m baselines.run --alg=imit --env=Cloth-v0 --num_env=10 --num_timesteps=5e4 \
        --cloth_config=../gym-cloth/cfg/demo_baselines_fixed_t3_depth.yaml --rb_size=50000 \
        --demos_path=../gym-cloth/logs/demos-2020-02-10-15-05-pol-oracle-seed-1336_to_1340-obs-blender-depth-False-rgbd-True-tier3_epis_2000_COMBO.pkl  \
        --bc_epochs=500  --actor_l2_reg=1e-5  --nb_rollout_steps=20  --nb_train_steps=240

DanielTakeshi commented 4 years ago

...

DanielTakeshi commented 4 years ago

DanielTakeshi / baselines-fork

Experiments, RGBD #1

RGBD Stuff

Tier 1 [Experiment 5.0.0a]

Tier 2 [Experiment 5.0.0b]

Tier 3 [Experiment 5.0.0c]

Training Run Plots

Simulated Results

The Data Logs

RGB only -- Use branch w/three channel RGB (62eca3109656d6a161795dccda0a0d9411536f76)

T1 RGB (see `openai-2020-02-19-16-35-04-032759`)

T2 RGB (see `openai-2020-02-19-16-39-21-629526`)

T3 RGB (see `openai-2020-02-20-21-35-16-544757`)

...

...

Depth Only

T1 D

T2 D

T3 D

...

...

DanielTakeshi / baselines-fork

Experiments, RGBD #1

RGBD Stuff

Tier 1 [Experiment 5.0.0a]

Tier 2 [Experiment 5.0.0b]

Tier 3 [Experiment 5.0.0c]

Training Run Plots

Simulated Results

The Data Logs

RGB only -- Use branch w/three channel RGB (62eca3109656d6a161795dccda0a0d9411536f76)

T1 RGB (see openai-2020-02-19-16-35-04-032759)

T2 RGB (see openai-2020-02-19-16-39-21-629526)

T3 RGB (see openai-2020-02-20-21-35-16-544757)

...

...

Depth Only

T1 D

T2 D

T3 D

...

...

T1 RGB (see `openai-2020-02-19-16-35-04-032759`)

T2 RGB (see `openai-2020-02-19-16-39-21-629526`)

T3 RGB (see `openai-2020-02-20-21-35-16-544757`)