transfer learning notebook

[hyunjimoon]

• meaning of num_transfer_steps in https://github.com/hyunjimoon/24_transpo/blob/f47120b11d764bf07b0340f22358f55cfe058041/CP3/analysis/utils.py#L95

episode 한번 policy update (Q-learning에서 Q matrix, ) , rollout = episode (

R is from env (Q is from agent)

• how is GaussianProcessRegressor used?

  from sklearn.gaussian_process import GaussianProcessRegressor
  from sklearn.gaussian_process.kernels import RBF

• by "without cheating" do you mean not knowing oracle? could you explain how get_baseline_performance works and what "oracle_transfer, exhaustive_training, sequential_oracle_training" means?

def get_baseline_performance(data_transfer, num_transfer_steps):
    deltas = data_transfer.columns.values.astype(float)

    # Oracle transfer
    oracle_transfer = [data_transfer.max(axis=0).mean()] * num_transfer_steps

    # Exhaustive training
    data_transfer_diagonal = np.zeros(len(deltas))
    for i in range(len(deltas)):
        data_transfer_diagonal[i] = data_transfer.iloc[i][i]

    exhaustive_training = [data_transfer_diagonal.mean()] * num_transfer_steps

    # Sequential oracle training
    sequential_oracle_training = []
    sot_deltas = []

    # 1st step
    sot_deltas.append(data_transfer.mean(axis=1).argmax())
    sequential_oracle_training.append(data_transfer.iloc[data_transfer.mean(axis=1).argmax(),:].mean())
    for _ in range(num_transfer_steps-1):
        candidate_indices = [x for x in range(len(deltas)) if x not in sot_deltas]
        index_tmp = [data_transfer.T[sot_deltas+[i]].max(axis=1).mean() for i in candidate_indices].index(max([data_transfer.T[sot_deltas+[i]].max(axis=1).mean() for i in candidate_indices]))
        sot_deltas.append(candidate_indices[index_tmp])
        sequential_oracle_training.append(data_transfer.T[sot_deltas].max(axis=1).mean())

    return oracle_transfer, exhaustive_training, sequential_oracle_training

[hyunjimoon]

• how to read Figure 6: Transfer matrix heatmap

---

resolved

• how to read file tags?

  
  For instance, you can run the following code if you want to do a zero-shot transfer from the source task trained for 13.89 m/s to 13 m/s.

python transfer_main.py --speed 13.0 --model_num 1 --source_path_name "results/intersection_reward-waittime_flow1000_lane4.0_length750_speed13.89_left0.25/" --num_episodes 50

parser = argparse.ArgumentParser(description='Arguments')
parser.add_argument('--flow', type=int, default=1000, help='Flow of cars')
parser.add_argument('--lane', type=float, default=4.0, help='Number of lanes')
parser.add_argument('--length', type=float, default=750, help='Length of lanes')
parser.add_argument('--speed', type=float, default=13.89, help='Speed limit')
parser.add_argument('--left', type=float, default=0.25, help='Left turn ratio')
parser.add_argument('--model_num', type=int, default=1, help='Model number')
parser.add_argument('--source_path_name', type=str, default="intersection_flow1000_lane4.0_length750.0_speed13.89_left0.25/", help='pathname')
parser.add_argument('--num_episodes', type=int, default=50, help='Number of episodes')
parser.add_argument('--reward', type=str, default='waittime', help='We only support wait time reward for transferring now.')
args = parser.parse_args()