sofenn: Self-Organizing Fuzzy Neural Network

Welcome to sofenn (sounds like soften)!

This package is a Keras/TensorFlow implementation of a Self-Organizing Fuzzy Neural Network. The sofenn framework consists of two models:
(1) Fuzzy Network - Underlying fuzzy network whose parameters are optimized during training
(2) Self-Organizer - Meta-model that optimizes the architecture of the Fuzzy Network

FuzzyNetwork

Underlying neural network model that contains parameters to be optimized during training

SelfOrganizer

Meta-model to optimize architecture of underlying fuzzy network

Model Description

The model is implemented per the description in:

'An on-line algorithm for creating self-organizing fuzzy neural networks\'
Leng, Prasad, McGinnity (2004)

Fuzzy Neural Network Architecture

Credit: Leng, Prasad, McGinnity (2004)

Layers

Inputs Layer (0)

Input layer of network

• input :
  • shape : (*, features)

Fuzzy Layer (1)

Radial (Ellipsoidal) Basis Function Layer

• each neuron represents "if-part" or premise of a fuzzy rule

• individual Membership Functions (MF) are applied to each feature for each neuron

• output is product of Membership Functions

• each MF is Gaussian function:

  

  • for i features and j neurons:

  • = ith MF of jth neuron

  • = center of ith MF of jth neuron

  • = width of ith MF of jth neuron

• output for Fuzzy Layer is:

  

• input :

  • shape: (*, features)

• output :

  • shape : (*, neurons)

Information flow of r features within neuron j

Credit: Leng, Prasad, McGinnity (2004)

Normalized Layer (2)

Normalization Layer

• output of each neuron is normalized by total output from previous layer

• number of outputs equal to previous layer (# of neurons)

• output for Normalized Layer is:

  

  = output of Fuzzy Layer neuron j

• input :

  • shape : (*, neurons)

• output :

  • shape : (*, neurons)

Weighted Layer (3)

Weighting of ith MF of each feature

• yields the "consequence" of the jth fuzzy rule of fuzzy model
• each neuron has two inputs:
  • = output of previous related neuron
  • = weighted bias

• with:

  = number of original input features

  

  

  = output of jth neuron from normalized layer

-output for weighted layer is:

• inputs :
  • shape : [(*, 1+features), (*, neurons)]
• output :
  • shape : (*, neurons)

Output Layer (4)

Final Output

• Unweighted sum of each output of previous layer ()

• output for fuzzy layer is:

  

  for u neurons

• final layer for regression problems

• input :

  • shape : (*, neurons)

• output :

  • shape : (*,)

Softmax Layer (5)

Classification Output

• (Optional) Softmax layer for classification problems

• input :

  • shape : (*, )

• output :

  • shape : (*, classes)