Bangla Handwritten Character Recognition using Deep Neural Network in R

A deep neural network (DNN) is an artificial neural network (ANN) with multiple hidden layers of units between the input and output layer.  DNN can learning complex non-linear relationship between input patterns and can generate compositional models where the input data(iamges)   is expressed as a layred composition.

Stack Auto-Encoder is one kind of Simple Neural Network Architecture where The input and output are same. The objective of Stack Auto-Encoder  is to enforce the network to learn a model that will give the similar output as input .

The strategy of fed , Stack Auto-Encoder to Deep Neural Network  is first , dropping  the connections of output layer and plug directly  into the Deep Neural Network .

Another Strategy, first  drop the whole  the first hidden layer to rest of the network of the Stack AutoEncoder and plug the remaining input and trained weights to the Deep Neural Network.

In this tutorial  , i use  deepnet  package which is available in  CRAN Project.

Data Description:

1.Train Data: 17500 (28*28 pixel)

2. Test Data : 2500(28*28 pixel)

 

Methods :

1.Stack Auto-Encoder

2. Deep Neural Network

Description:

In neural network training , we generally choose random weights in input layer to first hidden layer connection. Instead  of using the random weights  , we can prefer some weights that is influenced by input data . Neural Network with lot of parameters needs significant amount of computation  for optimization parameters . So pre-training is an efficient way to start neural network training that minimizes the computation time in further step .

First the input data in pre-trained by stack auto-encoder which provides initial weights to deep neural network.  The DNN needs some information to overlap the default value . I randomly choose some values.

 

library(“deepnet”, lib.loc=”~/R/x86_64-pc-linux-gnu-library/3.0″)
#Data Loading
train <- read.delim(“~/bangla_data/train.csv”, header=FALSE)
test <- read.delim(“~/bangla_data/test.csv”, header=FALSE)
trainlabel <- read.table(“~/bangla_data/trainlabel.csv”, quote=”\””, comment.char=””)
testlabel <- read.table(“~/bangla_data/testlabel.csv”, quote=”\””, comment.char=””)

#Data Preprocessing
train = as.matrix(train)
test = as.matrix(test)

trainlabel = as.vector(trainlabel)
testlabel = as.as.vector(testlabel)

#Train deep neural network with weights initialized by stack autoencoder

dnn =sae.dnn.train(train, trainlabel, hidden = c(10,10), activationfun = “sigm”, learningrate = 0.8,
momentum = 0.5, learningrate_scale = 1, output = “sigm”, sae_output = “linear”,
numepochs = 50, batchsize = 100, hidden_dropout = 0, visible_dropout = 0)

#test
nn.test(dnn, test, testlabel)

 

#………………………………..

begin to train sae ……
training layer 1 autoencoder …
training layer 2 autoencoder …
sae has been trained.
begin to train deep nn ……

The DataSet and Source Code are available in https://github.com/robi56/BHCR-DNN-SAE

 

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s