Introduction to Neural Network

What is Neural Network?
Neural Network is a computer model that mimic what brain do for processing data. Brain uses Neurons in order to process data and get a predictions. Have you ever thought about what was like when you are in childhood i bet you have asked soo many questions from your parents. That is how our brain acquire data. Our brain can process huge amount of data for the first time you hear a voice of a bird in the second time you can definitely understand who is the bird.

below video contain a sound of a bird lets consider that bird as coco

see below 2 clips and guess which bird is coco

Number 1	Number 2

Its Number 2 Right? see how powerful our brain is acquiring processing and predicting is accurate to the last drop.Theory of this power can be applied to a computer,For this bran neurons has to be mathematically modeled
to computer neuron. This theory was founded by Donald Hebb and his theory was called Hebbian theory
Theory
This below equation is the fundamental equation of neurons explained by Donald Hebb

let me explain the terms
y_i=output of a neuron,
x_j=j^thinput,
w_ij=weight to input to output
α is learning rate

if we get a graphical representation of neural network it looks like below this is called Perceptron Diagram

Figure of neural network graphical representation

here the bias term is used for activating the neural network when there is no data inputs like keeping the network alive,Output function is a liner function that has limited applications in real world

Figure of Data classification methods linear and non-linear

For convert linear output to nonlinear we use function called activation functions lets get this activation function as g(XW+b)
X=x₁,x₂,x₃,x₄,x₅....
W=w₁,w₂,w₃,w₄,w₅....
b=bias

There are 3 activation functions they are,

Sigmoid Activation
TanH Activation
ReLU Activation

Figure of activation functions

Above we talked about Perceptron Diagram for a single neuron. For multiple neurons there is a model called Multi-Output Perceptron

Figure of Multi-Output Pereceptron

using above building block we create a multi layer neural network called as deep learning.

deep learning network mean there is a input layer hidden layer and output layer,hidden layer is the difference between neural network and deep neural network is this middle layer.

Figure of Deep Neural Network

what can be the uses of this neural networks?

Neural networks and financial prediction

Currency prediction
Futures prediction
Bond ratings
Business failure prediction
Debt risk assessment
Credit approval
Bank theft
Bank failure

Medicine
Security

eg. Consider there is a trained model that can be predict the flight delay by inputting temperature and wind speed. first in the initial process we give already known data to the model and get a predicted value and compare it to actual value.

Figure of flight delay Neural network and its prediction

predicted Value is 0.05 and actual value is 1 difference between that is called loss we need to minimize the loss in-order to make a reliable prediction.
There are 3 systems of loss quantifying.

Total Loss
Binary Cross Entropy Loss
Mean Squared Error Loss

1.Total Loss

2.Binary Cross Entropy Loss

3.Mean Square Error Loss

Training the neural networking objective is to reduce the loss.
If we over train the neural network model this will leads to over fitting the data set.

Figure of function fitting to dataset

For prevent over fitting and under fitting there are best practices those are called Regularization Techniques.
1. Dropou

During training, randomly set some activations to 0

2. Early Stopping

Don’t give the network time to overfit

3. Weight Regularization

Large weights typically mean model is overfitting
Add the size of the weights to our loss function
Perform well on task + keep weights small