Effective Usage of Activation Functions for Data Classification with TensorFlow in Deep Neural Networks

Abstract

Artificial neural networks can be known as a computer system modeled on the human brain and neural system. In data classification, neural network provides fast and efficient results. Neural Network models are trained by using sets of labeled data. Neural networks have the ability to work with data, based on the training. There are thousands of interconnected nodes that belong to interconnected hidden layers inside the neural network. Activation function that have included in the neural network provides the output based on given an input or set of inputs. This research work focused on the comparison of the effects of using several activation functions on multiple hidden layers for classification using MNIST (Mixed National Institute of Standards and Technology) data set. Data classification was made using TensorFlow library. Tensorflow library with the help of keras used to build the neural network model. The experiment results of Rectified Linear Unit (ReLu), Leaky ReLU, Hyperbolic Tangent (tanH), Exponential Linear Unit (eLu), sigmoid, softplus, softmax and softsign activation functions. Data have been collected for the experiment in two different methodologies. There is a hidden layer with one activation function and multiple hidden layers with multiple activation function. The result of the study shows that the higher accurate rate than 88% for training and testing when it uses multiple hidden layers with multiple activation functions.