VU Amsterdam - CLTL - Robot Framework

Repository for Robot Applications created as part of the Computational Lexicology & Terminology Lab (CLTL) at the Vrije Universiteit, Amsterdam.

This is the original Python 2 version of the Leolani platform. This version was intended for the robot use as it supports the NAOqi backend. However, development on this repository has stopped due to the sunsetting of Python 2. For the latest version of this software please see the Python 3 version of the repository.

Pepper Robot Leolani

Features

A framework for creating interactive Robot Applications using Python, to enable:
- Human-Robot conversation using Speech-to-Text and Text-to-Speech
- Recognising friends by face and learning about them and the world through conversation
- Recognising and positioning the people and objects in its enviroment.
Natural Language Understanding through Syntax Trees (Grammars)
Knowledge Representation of all learned facts through a RDF Graph: the robot's Brain!
Curiosity based on Knowledge Gaps and Conflicts resulting from learned facts
Realtime visualisation in web browser

Getting started

Check out our WIKI for information on how it works and how to set up.

Check out our API Reference and Sample Applications!

Prerequisites

NAOqi Python SDK (version 2.5.10) from the Softbank robotics download page
Python 2.7.10
An application on Wolfram Alpha with API access
A project on the Google Cloud Platform supporting Text-To-Speech and Speech-To-Text APIs
Docker Engine
OpenFace
Pepper's sister-project: pepper_tensorflow, which includes the Tensorflow services within Python 3.6.
GraphDB with a repository named leolani

Detailed instructions on how to set up.

More information

More information on the Make Robots Talk project at CLTL can be found on its website

Citation

Please cite (at least one of) the following papers if you use this software in your research:

Leolani: a reference machine with a theory of mind for social communication

@inproceedings{vossen2018leolani,
      title={Leolani: a reference machine with a theory of mind for social communication},
      author={Vossen, Piek and Baez, Selene and Bajc̆eti{\'c}, Lenka and Kraaijeveld, Bram},
      booktitle={International conference on text, speech, and dialogue},
      pages={15--25},
      year={2018},
      organization={Springer},
}

A communicative robot to learn about us and the world

@inproceedings{vossen2019communicative,
      title={A communicative robot to learn about us and the world},
      author={Vossen, Piek and Santamaria, Selene Baez and Bajc̆eti{\'c}, Lenka and Ba{\v{s}}i{\'c}, Suzana and Kraaijeveld, Bram},
      booktitle={2019 Annual International Conference on Computational Linguistics and Intellectual Technologies, Dialogue 2019},
      pages={728--743},
      year={2019}
}

Leolani: A robot that communicates and learns about the shared world

@inproceedings{vossen2019leolani,
      title={Leolani: A robot that communicates and learns about the shared world},
      author={Vossen, Piek and Baez, Selene and Baj{\v{c}}eti{\'c}, Lenka and Ba{\v{s}}i{\'c}, Suzana and Kraaijeveld, Bram},
      booktitle={2019 ISWC Satellite Tracks (Posters and Demonstrations, Industry, and Outrageous Ideas), ISWC 2019-Satellites},
      pages={181--184},
      year={2019},
      organization={CEUR-WS}
}

Modelling context awareness for a situated semantic agent

@inproceedings{vossen2019modelling,
      title={Modelling context awareness for a situated semantic agent},
      author={Vossen, Piek and Baj{\v{c}}eti{\'c}, Lenka and Baez, Selene and Ba{\v{s}}i{\'c}, Suzana and Kraaijeveld, Bram},
      booktitle={International and Interdisciplinary Conference on Modeling and Using Context},
      pages={238--252},
      year={2019},
      organization={Springer}
}

License

Distributed under the MIT License. See LICENSE for more information.

cltl / pepper

readme