jn-jairo
commented
3 months ago Alright, installing from github fixed it.
Mix.install([
{:axon, github: "elixir-nx/axon", override: true},
{:nx, github: "elixir-nx/nx", sparse: "nx", override: true},
{:exla, github: "elixir-nx/nx", sparse: "exla", override: true},
{:kino_vega_lite, ">= 0.1.6"}
])So it is just the version on hex that has the problem.
The "Modeling XOR with a neural network" example don't work.
The loss increases until it become
NaNmodeling_xor_with_a_neural_network.livemd.zip
Modeling XOR with a neural network
Introduction
In this notebook we try to create a model and learn it the logical XOR.
Even though XOR seems like a trivial operation, it cannot be modeled using a single dense layer (single-layer perceptron). The underlying reason is that the classes in XOR are not linearly separable. We cannot draw a straight line to separate the points $(0,0)$, $(1,1)$ from the points $(0,1)$, $(1,0)$. To model this properly, we need to turn to deep learning methods. Deep learning is capable of learning non-linear relationships like XOR.
The model
Let's start with the model. We need two inputs, since XOR has two operands. We then concatenate them into a single input vector with
Axon.concatenate/3. Then we have one hidden layer and one output layer, both of them dense.Note: the model is a sequential neural network. In Axon, we can conveniently create such a model by using the pipe operator (
|>) to add layers one by one.Training data
The next step is to prepare training data. Since we are modeling a well-defined operation, we can just generate random operands and compute the expected XOR result for them.
The training works with batches of examples, so we repeatedly generate a whole batch of inputs and the expected result.
Here's how a sample batch looks:
Training
It's time to train our model. In this case we use binary cross entropy for the loss and stochastic gradient descent as the optimizer. We use binary cross entropy because we can consider the task of computing XOR the same as a binary classification problem. We want our output to have a binary label
0or1, and binary cross entropy is typically used in these cases. Having defined our training loop, we run it withAxon.Loop.run/4.Trying the model
Finally, we can test our model on sample data.
Try other combinations of $x_1$ and $x_2$ and see what the output is. To improve the model performance, you can increase the number of training epochs.
Visualizing the model predictions
The original XOR we modeled only works with binary values $0$ and $1$, however our model operates in continuous space. This means that we can give it $x_1 = 0.5$, $x_2 = 0.5$ as input and we expect some output. We can use this to visualize the non-linear relationship between inputs $x_1$, $x_2$ and outputs that our model has learned.
From the plot we can clearly see that during training our model learnt two clean boundaries to separate $(0,0)$, $(1,1)$ from $(0,1)$, $(1,0)$.