microsoft / maro

Multi-Agent Resource Optimization (MARO) platform is an instance of Reinforcement Learning as a Service (RaaS) for real-world resource optimization problems.
https://maro.readthedocs.io/en/latest/
MIT License
855 stars 153 forks source link

💡 Add Depots inside the CIM scenario #380

Closed Ahmedest61 closed 3 years ago

Ahmedest61 commented 3 years ago

Summary

Currently, the CIM environment considers only the ports Vs vessels interaction. However, depots also play an important role for ports in the storage/use of containers (empty/laden) in the shipping network.

Motivation

Once the cargo arrives, the port unloads it but may or may not directly send those to the consignee. The cargo can be placed (stored) preferably at some nearby depot(s). Similarly, after the cargo reaches to the consignee through other mean of transportation (e.g., intermodal), the empty container(s) that returned back to the port may need to be stored at some depot. It is because the empty containers may not loaded immediately on the vessel. There are factors like next ship arrival day/time, orders from consignor at the ports and so on. Therefore. the container movement between ports and depots shouldn't be ignored while carrying empty container repositioning (ECR).

Description

References

Alternatives

Additional context

Jinyu-W commented 3 years ago

Hi @Ahmedest61 , in CIM scenario, we use the full_return.buffer_ticks and the empty_return.buffer_ticks (parameters in the config.yml) to replace the process between the (empty/full) container release and the (full/empty) container return. No more other movements are simulated.

BTW, what do you want to do by introducing the depot? Do you want them to act as a kind of decision agent? If so, I don't think it is the problem we build this scenario for. If not, I think this kind of simplification is acceptable.

Ahmedest61 commented 3 years ago

Hi @Jinyu-W, I think utilizing full_return.buffer_ticks & empty_return.buffer_ticks for replacing other transportations (port to depot/port to consignee/port to consigner and vise versa) make sense. Correct me if I'm wrong MARO CIM only considers the empties loading & discharging cost? - it does not take account the storage and transportation costs as mentioned in this paper.

Jinyu-W commented 3 years ago

Hi @Ahmedest61, you are right about the cost part. Currently, MARO CIM only counts the number of empties loaded and discharged and report it as the operation num in the env metrics.

I agree that taking account the storage and transportation costs together would be better, but it is not easy for us to define a general cost calculation formula in the simulator since the needs may vary among different people/organizations. If you have some good ideas about such general cost formula, please let me know.

For your need:

  1. You can add some processing logic into the business engine to track the cost;
  2. Or you can count how many containers are being transported and how many storages each port is maintaining outside the business engine
Ahmedest61 commented 3 years ago

Great to hear back! I'm relatively new in playing with MARO CIM so if you could pin point the areas where I can embed the potentially innovative ideas of adding storage/transportation costs, please. I would really appreciate that.

riccardopoiani commented 3 years ago

@Ahmedest61 as previously said, I think that business_engine.py could be a good starting point. You might need some extra attributes in the different entities (ports.py and vessels.py) so that this information is directly retrievable from the environment snapshots, so that they can be used in the reward function.

Jinyu-W commented 3 years ago

@Ahmedest61 as previously said, I think that business_engine.py could be a good starting point. You might need some extra attributes in the different entities (ports.py and vessels.py) so that this information is directly retrievable from the environment snapshots, so that they can be used in the reward function.

Thank you all @Ahmedest61 @riccardopoiani for your nice suggestions and issues! For the features that you think are important and useful, we encourage you to create a pull request to directly contribute to this project :). Also, for the potential problems or bugs you found, you are always welcome to create a pull request if you find the corresponding fixing method.

Ahmedest61 commented 3 years ago

Having the full_return.buffer_ticks & empty_return.buffer_ticks in there for every port to replace the other type of transportations, we assume that all the containers (either filled/empties) always come back to the same port but this might not hold all the time, e.g., a laden once reaches to the consignee (that can be a port too!) gets empty now, it might be sent to a different port considering the shortest transport distance (cost). Similarly, an empty container ordered by a consigner once gets filled, it can send the filled container to some other port.

Jinyu-W commented 3 years ago

You are right about the actual business logic. Here in the CIM scenario, we just assume that all the containers will be returned to the port where they were released.

If you want the support for returning to another port, you need to modify the businees_engine.py by yourself, maybe to the functions _on_order_generated or _on_full_return, _on_empty_return.