• Rose-Hulman AI XPrize Neural Network [[https://travis-ci.org/RHIT-XPrize/rhit-xprize-neural-network.svg?branch=master]]

This repository hosts the neural network component of Rose-Hulman's AI XPrize robot. For more detailed information of the project as a whole, see [[https://github.com/RHIT-XPrize/rhit-xprize-pipeline][this repo]].

** Instructions To get started, first =pip3 install --user -r requirements.txt=. Run =generate_text_instructions.py= to create sample input and output files for the model to train on.

Next, run each of =text_flip_trainer.py=, =text_colors_trainer.py=, and =text_letters_trainer.py= on the appropriate output files to train the model.

Finally, either run =model_reply.py= to hop into a REPL and test the model, or use the =model_runner.py= interface in another program to use the generated models. ** Usage *** Generate Text Instructions =generate_text_instructions.py =

Generates == sample instructions and meanings to the four .csv files ==, ==, ==, and ==.

• == holds the raw text (in all caps to reduce the search space)

• == holds a boolean matrix of instruction types: | Flip | Move | |------+------|

• == holds a boolean matrix of colors:

| Red | Green | Blue | Orange | Yellow | None | |-----+-------+------+--------+--------+------|

• == holds a similar boolean matrix of letters:

| A | B | C | D | E | F | G | H | None | |---+---+---+---+---+---+---+---+------|

These each signify the desired meaning of the corresponding line of text. The 'None' columns mean that no color/letter was detected (e.g. "Move the blue block" has no letter). * Trainers ** Flip Trainer =text_flip_trainer.py ()=

Trains a neural network and stores it in the HDF5 file ==. The output is written out once every ten epochs. If == is provided, then it will load the weights stored in that HDF5 file before continuing training.

== is a csv file with a list of text inputs from Generate Text Instructions.

== is a csv file with a boolean matrix of expected flips outputs from Generate Text Instructions. **** Colors Trainer =text_colors_trainer.py ()=

Trains a neural network and stores it in the HDF5 file ==. The output is written out once every ten epochs. If == is provided, then it will load the weights stored in that HDF5 file before continuing training.

== is a csv file with a list of text inputs from Generate Text Instructions.

== is a csv file with a boolean matrix of expected colors outputs from Generate Text Instructions. **** Letters Trainer =text_letters_trainer.py ()=

Trains a neural network and stores it in the HDF5 file ==. The output is written out once every ten epochs. If == is provided, then it will load the weights stored in that HDF5 file before continuing training.

== is a csv file with a list of text inputs from Generate Text Instructions.

== is a csv file with a boolean matrix of expected letters outputs from Generate Text Instructions. * Runners ** Model REPL =model_repl.py =

Loads up the model files in ==, ==, and == and spins up a simple REPL instance to test the model. Sample usage looks like this:

+BEGIN_SRC

Move the red a here ('Move', 0.99999213) ('red', 0.9981108) ('A', 0.9987594) Flip the blue block ('Flip', 0.9937981) ('blue', 0.99789387) ('None', 0.9977901)

+END_SRC

It outputs three tuples, each of which holds the guessed meaning and the output (out of one) at that output neuron. The first is whether to move or to flip, the second is the color, and the third is the letter.