ManifoldRG / NEKO

In Progress Implementation of GATO style Generalist Multimodal model capable of image, text, RL and Robotics tasks
https://discord.gg/brsPnzNd8h
GNU General Public License v3.0
46 stars 10 forks source link

Introduce text modality into the repo #14

Closed bhavul closed 1 year ago

bhavul commented 1 year ago

Currently the repo is focused on control tasks. This is an issue so we can work on including pure text based task and use autoregressive pre-training on the same.

This likely means :

As for how this could be done, I wrote a doc couple of weeks back explaining different parts of the repo and how and what are the places where code changes are to be made : https://docs.google.com/document/d/16rNVqRucDMz4yqiYJz8NZFPHMGvzO0emb0KkxjAlfXo/edit#heading=h.huk7442i8c1n

bhavul commented 1 year ago

The code has been merged for addition of text modality now, closing the issue. Feel free to reopen if needed.