search

DLR-RM / stable-baselines3

PyTorch version of Stable Baselines, reliable implementations of reinforcement learning algorithms.
https://stable-baselines3.readthedocs.io
MIT License
9.15k stars 1.7k forks source link

[Feature Request] GraphFeatureExtractor #1723

Open EloyAnguiano opened 1 year ago

EloyAnguiano commented 1 year ago

🚀 Feature

Graph feature extractor for the new observation space that gymnassium provides.

Motivation

I've been using a Graph2Vec aproximation using GraphTransformers and CLS tokens on a non-publishable custom environment. As Gymnassium has implemented the Graph observation space, I would like to code a Feature Extractor for that kind of observation spaces.

Pitch

I want agents to use graph as observations

Alternatives

The alternative is to create a custom feature extractor that handles a dictionary space of the matrixes and attributes of graphs, but when the graph is very big this is not very optimal. The final goal would be to work with batches of graphs and sparse tensors.

Additional context

The only remaining piece is an official or publicly available gym environment to test this Feature Extractor. Does anyone know about this?

Checklist

araffin commented 1 year ago

Probably a duplicate of https://github.com/DLR-RM/stable-baselines3/issues/1280

EloyAnguiano commented 1 year ago

I think that one thing is that the whole SB3 framework can handle graphs (batches of graphs, rollout buffers of graphs, etc...) and another thing would be making a Feature Extractor once that happens. Would it be verý difficult to adapt?