pp.29–36
Rakesh Kumar Bhujade, Amitkant Pandit, Stuti Asthana
1,3Phd Scholar, Department of CSE, Suresh GyanVihar University, Jaipur- 302025, Rajasthan, India
2SECE, Shri Mata Vaishno Devi University, Jammu, India Corresponding Author email: rakesh.bhujade@gmail.com
Abstract-Process of feature extraction and the extracted feature has to be classified and for the same Artificial Neural Network (ANN) has been used. There are numerous classifiers but BPNN is most preferable because it is easy to handle and less error prone and apart from that its accuracy is much higher compared to other classifier. The classification has beenanalyzed individually with the two extracted features and finally the features were cascaded to increase the accuracy.To recognize the Handwritten English Character using a multilayer perceptron with one hidden layer has been considered. The feature extracted from the handwritten character is Boundary tracing along with Fourier Descriptor. Character is identified by analyzing its shape and comparing its features that distinguishes each character. An analysis has been done to determine the number of hidden layer nodes to achieve high performance of backpropagation network in the recognition of handwritten English characters.
Keywords: Back Propagation Neural Network (BPNN), Hidden layer, Perceptron, Character recognition.
INTRODUCTION
Neural Networks are recently being used in various kind of pattern recognition. Handwritings of the different human being are different; therefore it is very difficult to recognize the handwritten characters. The Handwritten Character recognition is an area of pattern recognition that has become the subject of research during the last several decades. Neural system is playing an important role in handwritten character appreciation. several information of character recognition in English have been published but still high recognition accuracy and minimum training time of handwritten English characters using neural network is an open problem. Consequently, it is a great important to develop an automatic handwritten character recognition system for English language [1]. In this work, efforts have been made to develop automatic handwritten character recognition system for English language with high recognition accuracy and minimum training and classification time. Experimental result shows that the approach used in this paper for English character recognition is giving high recognition accuracy and minimum training time.
Character Modeling
As the English language consists of 26 characters (5 vowels, 21 consonants) and is written from left to right correctly A place of hand written English characters is shown in Fig. 1.
Fig. 1A Set of Handwritten English Characters
Scanning and Skeletonization
Handwritten characters are scanned and it has been converted into 1024 (32X32) binary pixels. The skeletonization process may be used to binary pixel image and the extra pixels which are not belonging to the backbone of the character has been deleted and the broad strokes has been reduced to thin lines [1]. Skeletonization is illustrated in Fig. 2.
Fig. 2 Skeletonization of an English Character
Normalization
There are lots of variations in handwritings of dissimilar people. consequently, after skeletonization procedure it normalization of characters is performed so that all characters could become in equal dimensions of environment. In this study, characters are normalized into 30X30 pixel character and move to the top left corner of pixel window.
Bhujade et. al.SGVU Int J Env Sci Technol,Vol1,Issue2,29-36
Handwritten Characters – Scanning of Characters – Skeletionization – Normalization Feature Extraction –Neural Network Classification – Output (Recognized Characters)
Fig. 3: Block Diagram of the Recognition System
Character Recognition System
Classification:- For classification or recognition
The obstruct figure of Character recognition system is shown in following Fig 3.
The process of handwritten English character recognition is as follows:
Acquire the sample by scanning.
Normalization and Skeletionization process are execute.
- relate Boundary Detection characteristic removal techniqu
- Neural network Classification.
- Recognized Char
Character Recognition Procedure
- Pre-processing:- The pre-processing stage yield a clean document in the sense that maximal shape information with maximal compression and minimal noise on normalized image is obtained.
- Segmentation: – Segmentation is ansignificant stage because the extent one can reach in separation of words, lines or characters directly affects the recognition rate of the scr
- Feature removal:- After segmenting the character, extraction of feature like height, width, horizontal line, vertical line, and top and bottom detection is don
back propagation algorithm is used. Output:- Output is saved in form of text format.
Input Image
Pre-Processing
Segmentation
Feature Extraction
Recognition
Output Text
Fig. 4: Character recognition steps
As discussed above there are many methods for classification such as Support vector machine (SVM), Hidden Markov’s Model (HMM), Artificial Neural Network (ANN) etc. In the case of ANN for the purpose of classification.
Inpu
Neural Network Including Connections (Called Weights) Between Neurons
Output
Target
Compare
Inpu
Hidden
Output
Adjust Weights
Fig. 5: A basic feedback neural network
A neural network classifier was designed, trained and tested using the extracted feature sets. The training dataset consists of 2000 images, 200 sample each 10 class. The testing dataset consists of 500 images i.e. 50 for each individual class.
In ANN classification the empirically chosen learning rate parameter, momentum constant, number of epochs 0.5, 0.9 and 10000 respectively. The single hidden layer and number of nodes in hidden layer are empirically chosen 200. Number of nodes in input layer equals to feature size (412 and 64 for contour signature and tchebichef moments respectively). Number of nodes in output layer equals to number of classes. Here our output layer is 10 as have 10 classes.
TRAINING: The network was trained using 2000 images, 200 samples of each class.
TESTING: A total of 500 images was used 50 from each class were tested.
TWO-FOLD OPERATION: A two-fold operation is performed to evaluate the generalized performance of the system in user-independent condition. In two fold operation, testing and training dataset are altered, and the average values of all the performance parameters are taken.
Feed-forward multilayer perceptron
Multilayer perceptron is a neural network with one or more hidden layers. It is a feed forward artificial neural network that can be assumed as a connected graph between input, hidden and output nodes. Every nodes of multilayer perceptron is connected to each one except the input nodes. Each node in the multilayer perceptron is known as neuron. Back propagation learning algorithm is used in these types of artificial neural network. Fig 6 shows the graphical representation of multilayer perceptron, which consists two
Fig. 6 Graphical representation of Multilayer Perceptron Neural Network with 2 hidden layer
hidden layers and one output layer. Here neurons are present in both the hidden and output layers. Hidden layer neurons take a very important role in the operation of multilayer perceptron. Hidden neurons are used to recover and extract features from the training data.It is done by a non-linear function called activation function. Multilayer perceptron uses Back-propagation algorithm [27] for supervised learning.
Designing Of Multilayer Neural Network for Recognition
There are two basic methods used for OCR: Matrix matching and feature removal. Of the two behaviors to identify characters, medium matching is the simpler and additional ordinary. But immobile the Feature Extraction is used to make the product more robust and accurate. Feature Extraction is much more versatile than matrix matching. Here the use of Matrix matching for Recognition of character. The Process of Character Recognition of the document image mainly involves six phases:
- Acquisition of Grayscale Image
- Digitization/Binarization
- Line and Boundary Detection
- Feature Extraction
- Feed onward reproduction Neural Network based Matching.
- appreciation of Character support on matching sco
Fig. 7: Multilayer perceptron neural network
The scanned image must be [4, 5] a grayscale image or binary image, wherever binary image is a contrast stretched grayscale image. Grayscale image is then undergoes digitization.
In digitization [12] a rectangular matrix of 0s and 1s are formed as of the image. anywhere 0-black and 1-white and all RGB values are converted into 0s and 1s.The matrix of dots represents two dimensional arrays of bits. Digitization is also called binarization as it converts grayscale image into binary image using adaptive threshold. Line and Boundary detection is the procedure of identifying points in a digital image at which the character top, bottom, left and right are calculated. Feed Forward Neural Network approach is used to combine all the unique features, which are taken as inputs, one hidden layer is used to integrate and collaborate[9] similar features and if required adjust the inputs by adding or subtracting weight values, lastly oneoutput layer has been used to find the overall matching score of the network.
Multi-Layer Feed-forward Neural Network
Feed-forward ANNs transmits signals in one way only from input to outpu Advice system is not to be had in such type of networks or be capable of say that the output is self-determining. Feed-forward networks emerge to be straight ahead networks that correlate inputs through outputs. These category of networks are generally utilized for pattern detectiontion.
Fig. 8: Three layered neural network
In a three-layer network, the first layer known as input layer connects the input variables. The last layer that is known as output layer. The layer present in between the input layers and output layers are known as the hidden layer. In an ANN can utilize more than one hidden layer.
Multi-Layer Feed-back Neural Networks.
In a feedback networks the signals can travel directionally by using loops in the network. Feedback networks are a bit complicated and at the same time very dominant. Feedback networks are dynamic i.e. their state change constantly till they reach a stability point. They stay at the Stable point till the input changes and a new stability wants to be establish.criticism neural network are as well known to be Normalized and Skeletonized
Fourier Descript
A Normalization
Binarization
Boundary Tracing Features (32X32) Bits
Classifier MLPN
Output
Scanned Handwritten Character
Character Conversion into (1 of 0)
Pattern Recognition
communicative or persistent, still though the last term is often utilized to represent feedback relations in single- layer organizations.
Fig. 9: Feedback neural network
This type of network comprises of two condition First one is the forward phase were the activation spread from the input layer to output layer. The second phase is the backward phase where the difference between the detected actual value and the minimal value in the output layer is circulated backwards so that it can amend the values of weights and bias values.
SYSTEM MODEL
A whole procedure of handwritten English character recognition is given below
- Capture the scanned characters.
- Perform the Normalization process.
- Perform Binarization.
- Apply Feature Extraction Methods (Boundary tracing technique).
- Implement Neural Network Classif
- Get the recognized character. A complete flowchart of handwritten English character recognition is given below in Fig.10
Fig. 10: A System for English Character Recognition
Extraction of handwritten character recognition
In this paper, to extract the information of the boundary of a handwritten character, the eight-neighbor adjacent method has been adopted. This scans the binary image until it finds the boundary. The searching follows according to clockwise direction. For a few foreground pixel p, the set of all foreground pixels connected to it is called connected component containing p. The pixel p and its 8-neighbors have been shown in Fig.10. Once a white pixel may be detected, it checks another new white pixel. The tracing follows the boundary automatically. As the first pixel has been found, the program will be assigned the coordinates of that position to indicate that this is an origin of the boundary. The new found pixel would be assigned as a new reference point and starts the eight-neighbor searching. This way, the coordinates of the initial point are varied according to the position. As the tracer moves along the boundary of the image, the analogous coordinates will be stored in an array for the computation of Fourier Descriptors. Throughout the boundary tracing process, the program always check the condition whether the first coordinates of the boundary are equal to the last coordinates. Once it is obtained; means the whole boundary has been traced and boundary tracing process completes [2].
Fig. 11: Pixel P and Its 8-neighbor
Fig. 12 Boundary Tracing of a Character
Fourier Descriptors
Fourier Descriptors have involved in finding the
Discrete Fourier coefficients a[k] and b[k] for 0 ≤ k ≤ L – 1,
a[k] = 1/L S x[m]e– jk(2p/L)m (1)
b[k] = 1/L S y[m]e– jk(2p/L)m (2)
Fourier coefficients derived according to equations (1) and (2) are not rotational or shift invariant but Fourier Descriptors that have the invariant property with respect to rotation and shift,the following operations are defined. For each n compute a set of invariant descriptors r(n)
r(n) = [|a(n)|2 + |b(n)|2]1/2 (3)
Computing new set of descriptors s(n) by eliminating
the size of character from r(n)
s(n) = r(n)/r(1) (4)
a(n), b(n) and invariant descriptors s(n), n = 1,2,…..(L –
1) were derived for all of the characters.
Recognition
Recognition of handwritten characters is a very complex problem. The characters could be written in different sizes, thickness,orientation, format and dimension. This will give infinite variations. The capability of neural network to generalize and insensitive to the missing data would be very beneficial in recognizing handwritten characters. In this paper, for English handwritten character recognition in neural Feed Forward Multi-Layer Perceptron network (MLPN) with one hidden layer has been used. For training, back-propagation algorithm hasbeen implemented [1, 5].
Multilayer Perceptron Network
The multilayer perceptron neural networks with the EBP algorithm have been applied to the wide variety of problems. In this paper, two-layer perceptron i.e., one of the hidden layers and one output layer has been used [5]. Structure of MLP network for English character recognition is shown in Figure 2.4.
Fig. 13Multilayer perceptron Network (MLPN)
In MLPN with Backpropagation training algorithm, the procedure and calculations as follows:
fj(x) = 1/(1 + e–net) and net = ∑ wijoi
whereoiis the output of unit i, wijis the weight from unit i to unit j. The generalized delta rule algorithm was used to update the weights of the neural network in order to minimize the cost function:
E = ½(∑(Dpk– Opk))2
whereDpkand Opkare the desired and actual values respectively, of the output unit k and training pair p.Convergence is achieved by updating the weights using following formulas:
Wij(n + l) = Wij(n) + ∆Wij(n)
(5)
∆Wij(n) = ηδXj + α(Wij(N) – Wij(n – 1))(6)
where h is the learning rate, a is the momentum, Wij(n) is weight from hidden node i or from an input to node j at nth iteration, Xi is also the output of unit i or is an input, and djis an error term for unit j [1]. If unit j is an output unit, then
δj = Oj( 1 – Oj)(Dj– Oj)
If unit j is an internal hidden unit
δj = Oj( 1 – Oj) ∑ δkWkj
Back-Propagation Algorithm
The back-propagation learning algorithm has mainly two steps: i) propagation ii) weight update.
Propagation is of two types:
Forward propagation- In forward propagation function signals flow from input layer to hidden and output layers. Forward propagation of training input is used to generate the propagation output activations.
Backward propagation- In backward propagation error signals propagate from output to hidden and input layers. Backward propagation is used to generate details of all output and hidden neurons in input layer. It is used as feedback in input layers. This type of propagation generally has higher accuracy compared to forward propagation because here comparison is done with the desired output and the error present if any is corrected by updating the weights. Weight update is done by the following steps a) multiply local gradient and input signal of neuron b) subtract a portion of the gradient from the weight. It can be expressed by the following formula
Weight correction=(learning rate parameter)X(local gradient)X(input signal of neuron) b
LITERATURE REVIEW
Jia-ShingSheu; Guo-Shing Huang; Ya-Ling Huang, [1] Investigated on The HCI (Human-Computer Interface) technology development has been an important topic because of the popularity of digital technology and the innovation of 3C product. The interactive controlsystem of this paper is included both of dynamic and handwriting appreciation. The active gesture uses the Kinect Sensor to achieve the real-time communication. The script digital character could be used to select TV channel and use the BPNN (Back Propagation Neural Network)
Ouchtati, S.; Redjimi, M.; Bedda, M., [2] In this study has been proposed an off line system for the recognition of the handwritten numeric chains. the study is extended for the reading of the handwritten numeric chains constituted of a variable number of digits. Vertical projection had been used to segment the numeric chain at isolated digits and every digit (or segment) was presented separately to the entry of the system achieved in the first part (recognition system of the isolated handwritten digits).
Soni, R.; Puja, D., [3] In this study the performance of Gradient descent with momentum & adaptive backpropagation (TRAINGDX) and BFGS quasi- Newton backpropagation (TRAINBFG) of Backpropagation Algorithm in multilayer feed forward Neural Network for Handwritten English Characters of
Vowels. Those are the variant of Backpropagation learning algorithm namely Quasi-Newton backpropagation learning algorithm and Gradient descent with momentum and adaptive backpropagation learning algorithm for training set of the Handwritten English Characters of Vowels.There is constraint of gradient descent learning algorithm for convergence due to the problem of local minima which is inherit problem of backpropagation learning algorithm.
Kaensar, C., [4] Investigated on “Analysis on the Parameter of Back Propagation Algorithm with Three Weight Adjustment Structure for Hand Written Digit Recognition, [4] Recently a commonly used method for Recognition of Handwritten Digit Application based on Back Propagation Neural Network (BPNN) has been widely applied.
Zhu Dan; Chen Xu, [5] In the Study of Offline handwriting recognition has become one of the hottest directions in the field of image processing and pattern recognition. On the other hand, authors make comparison of the recognition rate among three methods: neural system technique, thirteen features technique and Fisher discriminant study technique. To finish, they succeed in porting these algorithms to Android.
Budiwati, S.D.; Haryatno, J.; Dharma, E.M., [6] Investigated on Japanese language has complex writing systems, Kanji and Kana (Katakana and Hiragana). Each one has different way of writing. One easy way to differentiate is Kanji have more strokes comparatively Kana. Meanwhile, it needs a lot of effort to remember characters of Katakana and Hiragana,therefore it will be very difficult to distinguish handwritten Katakana and Hiragana, because there are a lot of similar characters. This is the reason why authors need pattern recognition.
- PROBLEM FORMULATION
The problem Originally, Neural Networks are recently being used in various kind of pattern recognition. Handwritings of different person are different; therefore it is very difficult to recognize the handwritten characters. The Handwritten Character recognition is an area of pattern recognition that has become the subject of research during the last some decades. That efforts have been made to develop automatic handwritten character recognition system for English language with high recognition accuracy and minimum training and classification time. Experimental result shows that the approach used in this paper for English character recognition is giving high recognition accuracy and minimum training time.
- PROPOSED METHODOLOGY
Main goal of It has been observed in character recognition problems and also in other pattern recognition problems that, in many cases, the distinguishing features are located in small parts of a group of similar characters while the other parts are indistinguishable. These smaller parts containing the distinguishing features can be discriminated with lower order moments in comparison to the order of moments needed for the complete images. We have explored this observation in the recognition of printed characters of the handwritten English characters and found that considerable improvement can be achieved in the recognition of characters through the proposed method. By using skeletonizationprocess , the problems associated to continuous orthogonal moments are eliminated by using a discrete orthogonal basis function. Apart from that mapping is also not required to compute skeletonization process due to the orthogonality property of the skeletonization polynomials.
- CONCLUSIONS
For the process, a system for recognizing handwritten English characters has been developed. An experimental result shows that Fourier descriptors with back propagation network yields good recognition accuracy. The skeletonized and normalized binary pixels of English characters were used as the inputs of the network. The structure analysis shows that if the number of hidden nodes increases the number of epoches taken to recognize the handwritten character is also increases. A lot of efforts have been made to get higher accuracy but stil there are tremendous scope of improving recognition accuracy by developing new feature extraction techniques or modifying the existing feature extraction techniques.
REFERENCES
Jia-ShingSheu; Guo-Shing Huang; Ya-Ling Huang, “Design an Interactive User Interface with Integration of Dynamic Gesture and Handwritten Numeral Recognition,” Computer, Consumer and Control (IS3C), 2014 International Symposium on , vol., no., pp.1295,1298, 10-12 June 2014.
Ouchtati, S.; Redjimi, M.; Bedda, M., “Realization of an offline system for the recognition of the handwritten numeric chains,” Information Systems and Technologies (CISTI), 2014 9th Iberian Conference on , vol., no., pp.1,6, 18-21 June 2014.
Soni, R.; Puja, D., “Performance evaluation of multilayer feed forward neural network for handwritten English Vowels Characters,” Information Systems and Computer Networks (ISCON), 2013 International Conference on , vol., no., pp.82,87, 9-10 March 2013.
Structure for Hand Written Digit Recognition,” Service Systems and Service Management (ICSSSM), 2013 10th International Conference on , vol., no., pp.18,22, 17-19 July 2013.
Zhu Dan; Chen Xu, “The Recognition of Handwritten Digits Based on BP Neural Network and the Implementation on Android,” Intelligent System Design and Engineering Applications (ISDEA), 2013 Third International Conference on , vol., no., pp.1498,1501, 16-18 Jan. 2013.
Budiwati, S.D.; Haryatno, J.; Dharma, E.M., “Japanese character (Kana) pattern recognition application using neural network,” Electrical Engineering and Informatics (ICEEI), 2011 International Conference on , vol., no., pp.1,6, 17-19 July 2011.
Ramzi, A.; Zahary, A., “Online Arabic handwritten character recognition using online-offline feature extraction and back-propagation neural network,” Advanced Technologies for Signal and Image Processing (ATSIP), 2014 1st International Conference on , vol., no., pp.350,355, 17-19 March 2014.
Phienthrakul, T.; Chevakulmongkol, W., “Handwritten recognition on Pali cards of BuddhadasaIndapanno,” Computer Science and Engineering Conference (ICSEC), 2013 International , vol., no., pp.191,195, 4-6 Sept. 2013.
Selvi, P.P.; Meyyappan, T., “Recognition of Arabic numerals with grouping and ungrouping using back propagation neural network,” Pattern Recognition, Informatics and Mobile Engineering (PRIME), 2013 International Conference on , vol., no., pp.322,327, 21-22 Feb. 2013.
Sahu, N.; Raman, N.K., “An efficient handwritten Devnagari character recognition system using neural network,” Automation, Computing, Communication, Control and Compressed Sensing (iMac4s), 2013 International Multi-Conference on , vol., no., pp.173,177, 22-23 March 2013.
V.K. Govindan and A.P. Shivaprasad, “Character Recognition – A review,” Pattern Recognition, Vol. 23, no. 7, pp. 671- 683, 1990.
PlamondonRejean, Sargur N. Srihari, “On line and off line handwriting recognition: A comprehensive survey”, IEEE Transactions on PAMI, vol. 22, No. 1, 2000.
Anil K. Jain, Robert P. W. Duin, Jianchang Mao, “Statistical Pattern Recognition:A Review”, IEEE Transactions on PAMI, Vol. 22, No. 1, pp. 4-37, January 2000.
- R. Duda, P. E. Hart, D. G. Stork, “Pattern Classification”, second ed., Wiley- Interscience, New York 2001.
R.G. Casey and E.Lecolinet, “A Survey of Methods and Strategies in Character Segmentation,” IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 18, No.7, July 1996, pp. 690-706
C.-L. Liu, K. Nakashima, H. Sako, andH. Fujisawa, “Handwritten digit recognition: Bench-marking of state-of-the-art techniques”, Pattern Recognition. 36 (10), 2271–2285, 2003.
A. Rajavelu, T. Musavi, and M. V. Shirvaikar, “ANeural Network Approach to Character Recognition,” Neural Networks, 2, pp. 387-393,1989.
J. Dong, A. Krzyzak, C. Y. Suen, “An improved handwritten Chinese character recognition system using support vector machine”, Pattern Recognition Letters. 26(12), 1849–1856, 2005
S.C.W. Ong, S. Ranganath, “Automatic sign language analysis: a survey and the future beyond lexical meaning”, IEEE Transactions on Pattern Analysis and Machine Intelligence 27 (6) (2005) 873–891.
S. Zhang and G. Lu, “A comparative study on shape retrieval using fourier descriptors with different shape signatures”, in Proc. International Conference on Intelligent Multimedia and Distance Education (ICIMADE01), 2001
Rafael C. Gonzalez, Richard E. woods and Steven L.Eddins, “Digital Image Processing using MATLAB”, Pearson Education, Dorling Kindersley, South Asia, 2004.
Kimura, T. Wakahara, and A. Tomono, “Combination of statistical and neural classifiers for a high-accuracy recognition of large character sets”, Transactions of IEICE Japan. J83-D-II (10), 1986– 1994, 2000 (in Japanese).