How machines learn to replicate the decision making and thinking process of humans. To understand neural networks for dummies, the best way is to observe how thoughts occur in the brain.
Thoughts only originate from the data or say memories in our brain that we have collected through our five senses. Whenever something related directly or indirectly to the memory occurs, thoughts occur as a pop in response to that data. Thinking is a quite complex phenomenon because as the memories increase the number of possibilities for a new data increases.
That is why human brain is considered so complex in thinking and decision making. Evolution of millions of years has resulted in so much of memories resulting in the most complex neurological structure. Another question arises how thoughts connect each other?
Our experiences create the memories in brain and these memories shape the patterns of thoughts. A similar situation is experienced differently by different people resulting in different memories inside the brain and hence a completely different perception. Hence the mind is trained differently resulting in different patterns of thoughts.
A famous experiment in which students were given a work and they had to tell the first two words that occur in their mind in response to that word. Red was the first word given. One student gave the answer blood,ambulance and another answered apple,doctor. Clearly the answers are completely different but there are patterns in answer. This experiment gave a practical demonstration how thoughts occur and connect each other.
This idea of how thoughts work led to the revolutionary era of deep learning and neural networks. Now we’ll share some fascinating visualization about neural networks for dummies.
How neural networks work:
Decision making of mind is completely based on memories and connections between the memories. Memories are stored in biological units called neurons. To replicate this complex thinking process of human brain by a computer we build neural networks.
The name neural networks itself signifies neurons and connections between them. Data is the most important and primary aspect to build an artificial neuron. Once data is collected we have to form the necessary connections and patterns between the data to build the experiences that will give the results we desire. Same data can be trained differently resulting in a different neural network and hence giving different result. Hence the computer is given data, connections are built between the data that creates an experience for the machine and based on this experience it results the output for new data.
The best part of neural networks is you do not have to build each connection yourself. This task is next to impossible. By applying some simple and interesting math one can easily build these networks. There are very big libraries like keras in which you can directly create neural networks very easily. One doesn’t need to apply the math all the time in the code. But a beginner must at least visualize how these connections are built between data and the network is working. These visualizations fascinate a beginner and inspire to think how to build new networks of their own. This article about neural networks for dummies at neuralai.co contains many interesting visualizations in the later section to understand neural networks.
Neural Networks: Motivation Problem
The human visual system is one of the biggest wonders of the complete system. It is so powerful in recognizing and detecting objects that when we try to replicate that on a computer, we understand the beauty and complexity of it. For example, below is an image of hand-written numbers. It takes almost no or least effort to recognize that this is 3, 4, etc. The human visual cortex contain millions of neurons with billions of connections between them making the recognition process easy and fast.
When we try to train a computer to recognize digits the real challenge lies is to make computer learn what a digit may look like even if it is handwritten . For example ,9 consists of a loop and a line but there is no hard and fast rule to make a loop as show in above figure. Because there are many possibilities for manually writing a digit, it becomes difficult as there are many possibilities. Here comes the idea of a neural network, which tries to replicate the visual system of humans.
These are some interesting visualization by 3 BLUE 1 BROWN YouTube channel which shows a neural network that recognize a digit. Before making the final prediction among 10 possible outputs the patterns like loops are detected just in the previous layer as shown. And in the layer before the pattern for the corresponding loops are detected.
Building a basic neural network:
The basic architecture of a neural network consists of neurons that contain information and further connections between them. The architecture is explained for digit recognition system.
Initially we build an artificial neuron also known as a perceptron in which we feed the data(here the handwritten image). But one thing to keep in mind is that in a simple neural network there is usually no visualization of what information is passed to next layer. For that we have to go with convolutional neural networks in further articles.
- The artificial neurons are stores in layers as shown in figure. The first layer is the input layer in which there are 784 neurons. It means the input image has 28 X 28 pixels in image.These pixels are the data to 784 neurons in the first layer.
- In the next layer which is the first hidden layer containing 256 neurons we pass on the data from neurons of first layer. Each neuron in the hidden layer is connected to all the neurons in the first layer. The connections are made through weights which is the mathematical aspect discussed in another article. But the interesting question is what information is passed to the next layer. It is shown in the visualization.
- In the next hidden layer which contains 128 neurons information is further forwarded.
- Before reaching the last layer we have to detect the patterns of loops or lines in the input image.
- The final prediction is between 10 neurons from the digits 0 to 9.
Based on the input specific neurons activate and lighten up in the next layer and they further activate neurons in the next layer until a prediction occurs in final layer.
But wait how did the network learn what features to extract in the next layer? How the perfect connections are built between hidden layers and the output. Here comes the concept of learning of neural network!
How neural networks learn:
- Initially the networks contains random connections between neurons that has no significance. Information is passed from one layer to others to make the prediction.
- To make a neural network learn what answer it should predict, we also give it the correct answer as an input. The final prediction by the output layer is compared to the correct answer. The difference is the cost or say loss in our correct prediction.
- Hence to minimize the difference we reflect back this loss in previous layers. This means connections between neurons of corresponding layers are tightened and loosened according to the loss from the next layer.
- The process is known as back propagation as we are back-propagating losses and adjusting the connections between neurons. The process is repeated again and again till loss is minimized to a threshold value.
This is an amazing visualization of learning of neurons between layers.
Neural Networks for dummies: A visualization
Neural networks are the basic architecture for advance deep learning. Convolutional neural networks, Recurrent neural networks, Deep Neural Networks are further fascinating advance architectures of NN.
On image mostly convolutional neural networks are been used for processing. Image classification, object detection, art generation and many interesting applications are done by CNN’s.
Recurrent neural networks work extremely well in audio data. For music generation, caption generation, music genre classification, poetry generation and many amazing applications use RNN’s.
This was the basis of neural networks for dummies. More visualizations of these networks with real life interesting projects can be found on other sections of the website.
In other articles you will find the appropriate explanation with some cool projects. Do share your comments and reviews. Thank you for reading.