Artificial Android Life - Artificial Neural Networks Overview

So I started thinking about Artificial Life, Genetic Algorithms, and Neural Networks. Most of the concepts are not actually that hard to grasp it just seems that there is a lot of obfuscation when it comes to the subject. So hopefully I can write about what I am doing without making it too complicated.

Artificial Neural Networks(ANN)

You can check out Wikipedia for a nice explanation of what an artificial neural network is or you could just take my word that it is just function machine or better yet a collection of function machines. So in short take a bunch of function machines and point them at one another and you have a artificial neural network.

Function Machine

A function machine has three parts input, rule, and output. the input can be what ever you want it to be, the rule is any function that transforms the input, and the output is the result of that transformation.

Example:
input = 10
rule = input * 2
output = 20

 Artificial Neurons

Again Wikipedia has a good explanation of what an artificial neuron is. But again basically this is a set of transfer function machines that point to one activation function machine.

Transfer Function

Now my definition here might be a little harder to find because transfer function and activation function are usually synonymous, but I consider this as the function that transforms inputs before passing the them to a summation function. usually this function takes two inputs.

Example:

public transfer(float input, float bias) {
 return input * bias;
}

Summation Function

Anywhere you look this should be the same it is the sum of several inputs.

Example:

public sum(float[] input) {
 float result = 0;
 for(int i = 0; i < input.length; i++) {
  result += input[i];
 }
 return result;
}

Activation Function

There are a lot of different functions to choose from and I would definitely check out Wikipedia on this one but this function is actually going to return the output of the artificial neuron.

Example:

public activate(float input) {
 return input < 0 ? 0 : 1;
}

Java Function Implementations

On the off chance you really don't want to read mathematical notation to write your own implementation of some of the functions found on Wikipedia I have written them out in Java.

I have not included the functions that I know to be included in the Java Math class. if you want a very very tiny speed boost use the StrictMath class incited of Math class.

public double identity(double input) {
 return input;
}

public double binaryStep(double input) {
 return input < 0.0 ? 0.0 : 1.0;
}

public double softStep(double input) {
 return 1.0 / (1.0 + (Math.pow(Math.E, input)));
}

public double softSign(double input) {
 return input / (1.0 + Math.abs(input));
}

public double reLU(double input) {
 return input < 0.0 ? 0.0 : input;
}

public double pReLU(double input, double bias) {
 return input < 0.0 ? bias * input : input;
}

public double eLU(double input, double bias) {
 return input < 0.0 ? bias * (Math.pow(Math.E, input) - 1.0) : input;
}

public double softPlus(double input) {
 return Math.log(1.0 + Math.pow(Math.E, input));
}

public double bentIdentity(double input) {
 return ((Math.sqrt((input * input) + 1.0) - 1.0) / 2) + input;
}

public double softExponential(double input, double bias) {
 if(bias < 0.0) {
  return -(Math.log(1.0 - (bias * (input + bias))) / bias);
 }
 
 if(bias > 0.0) {
  return ((Math.pow(Math.E, bias * input) - 1.0) / bias) + bias;
 }
 
 return input;
}

public double sinc(double input) {
 return input == 0.0 : 1.0 : Math.sin(input) / input;
}

public double gaussian(double input) {
 return Math.pow(Math.E, -(input * input));
}