Add meta learning framework

ai4co / rl4co

A PyTorch library for all things Reinforcement Learning (RL) for Combinatorial Optimization (CO)

https://rl4.co

MIT License

381 stars 70 forks source link

Add meta learning framework #183

Closed jieyibi closed 2 months ago

jieyibi commented 3 months ago

Description

Add a new training framework based on meta learning. Details refer to Zhou et al. 2023.

Motivation and Context

To address the generalization issue.

Types of changes

What types of changes does your code introduce? Remove all that do not apply:

[ ] Bug fix (non-breaking change which fixes an issue)
[x] New feature (non-breaking change which adds core functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)
[ ] Documentation (update in the documentation)
[x] Example (update in the folder of examples)

Checklist

Go over all the following points, and put an x in all the boxes that apply. If you are unsure about any of these, don't hesitate to ask. We are here to help!

[x] My change requires a change to the documentation.
[ ] I have updated the tests accordingly (required for a bug fix or a new feature).
[ ] I have updated the documentation accordingly.

fedebotu commented 3 months ago

It seems there was a mistake here - I noticed that the "tasks" are a bit hardcoded, i.e. the "size" and "capacity" (I guess for TSP and CVRP, right?)

Speaking of testing: you can make sure things work on your device before committing by using pytest tests

fedebotu commented 3 months ago

Routine check, how is progress going? I think the multi-distribution generators in particular should be included, since they are part of MDPOMO, right?

jieyibi commented 3 months ago

The updated commit supports training on multiple mixed distributions by changing the argument loc_distribution to "mix_distribution" in the generator_params of the environment, e.g.

env = CVRPEnv(generator_params={'loc_distribution': "mix_distribution"})

The rest of the arguments remain the same.

Note that this mixed distribution setting follows the # setting in Bi et al., 2022.

fedebotu commented 3 months ago

Great! Should the code be merged?

jieyibi commented 2 months ago

Yes. I think it is ready to be merged. The newly reproduced performance (by changing some key parameters) seems similar to that in the literature.

fedebotu commented 2 months ago

Great!

How about the generalization experiments, i.e., MDPOMO? Do you have yaml configuration for them like this, or a training script?

jieyibi commented 2 months ago

Yeah, already added to the main branch:)

fedebotu commented 2 months ago

Awesome! Then we can go ahead and merge :)