A deep neural network (DNN) is an artificial neural network (ANN) with multiple hidden layers of units between the input and output layer. DNN can learning complex non-linear relationship between input patterns and can generate compositional models where the input data(iamges) is expressed as a layred composition.
Stack Auto-Encoder is one kind of Simple Neural Network Architecture where The input and output are same. The objective of Stack Auto-Encoder is to enforce the network to learn a model that will give the similar output as input .
The strategy of fed , Stack Auto-Encoder to Deep Neural Network is first , dropping the connections of output layer and plug directly into the Deep Neural Network .
Another Strategy, first drop the whole the first hidden layer to rest of the network of the Stack AutoEncoder and plug the remaining input and trained weights to the Deep Neural Network.
In this tutorial , i use deepnet package which is available in CRAN Project.
1.Train Data: 17500 (28*28 pixel)
2. Test Data : 2500(28*28 pixel)
2. Deep Neural Network
In neural network training , we generally choose random weights in input layer to first hidden layer connection. Instead of using the random weights , we can prefer some weights that is influenced by input data . Neural Network with lot of parameters needs significant amount of computation for optimization parameters . So pre-training is an efficient way to start neural network training that minimizes the computation time in further step .
First the input data in pre-trained by stack auto-encoder which provides initial weights to deep neural network. The DNN needs some information to overlap the default value . I randomly choose some values.
train <- read.delim(“~/bangla_data/train.csv”, header=FALSE)
test <- read.delim(“~/bangla_data/test.csv”, header=FALSE)
trainlabel <- read.table(“~/bangla_data/trainlabel.csv”, quote=”\””, comment.char=””)
testlabel <- read.table(“~/bangla_data/testlabel.csv”, quote=”\””, comment.char=””)
train = as.matrix(train)
test = as.matrix(test)
trainlabel = as.vector(trainlabel)
testlabel = as.as.vector(testlabel)
#Train deep neural network with weights initialized by stack autoencoder
dnn =sae.dnn.train(train, trainlabel, hidden = c(10,10), activationfun = “sigm”, learningrate = 0.8,
momentum = 0.5, learningrate_scale = 1, output = “sigm”, sae_output = “linear”,
numepochs = 50, batchsize = 100, hidden_dropout = 0, visible_dropout = 0)
nn.test(dnn, test, testlabel)
begin to train sae ……
training layer 1 autoencoder …
training layer 2 autoencoder …
sae has been trained.
begin to train deep nn ……
The DataSet and Source Code are available in https://github.com/robi56/BHCR-DNN-SAE