Developing A Text Classification System Using Svm Computer Science Essay

Data excavation is a engineering that has been at that place from old ages. Many companies used these techniques to analyse market tendencies. It allows users and companies to measure the informations from a figure of different positions and to set it into assorted classs. They try to happen out correlativities among the shopping tendencies and to derive more and more consumer focal point. It is chiefly nil but happening similarities in a immense relational database. Data excavation can supply companies with all kinds of consumer related factors such as demographic values, pricing etc. This can be truly helpful as it can be used to supply each client with specific trades and merchandises advertizement. e.g. Credit card companies frequently offer mortgage trades and loan ads based on their disbursals, balance in their histories and their recognition history. Amazon offers merchandises based on what client purchased late and what points are they seeking for these yearss. Similarly Netflix and Blockbuster recommend leases to each and every person based on their rental history.

Support vector machines are really utile when it comes to the categorization, arrested development and supervising of informations. It can execute informations suiting and categorization over really conventional informations sets every bit good as if there is any abstract information given. SVMs perform categorization by taking inputs and making hyper planes. That manner it is easy to screen out classs of each information inputs. Simplest signifier of division of informations is classed additive categorization where information is separated between two categories. Greater the distance between those two categories is, the better hyperplane it is. Each Support Vector Machine defines boundaries utilizing determination plane which means it splits the objects belonging to different categories and draws a determination boundary to clearly sort them.

The end of our undertaking is to understand the supervised acquisition methods and to accomplish text categorization utilizing Support Vector Machines ( SVMs ) . In order to accomplish this, we have used SVM algorithms via the LIBSVM negligee in Weka environment. LIBSVM makes it easy to utilize assorted tools utilizing map calls/APIs to accomplish control over categorization parametric quantities. Advanced capablenesss are besides achieved with the aid of a GUI rich Weka environment. We have developed two frames to accept a scope of input and supply classified end product in a graphical mode exposing right classified values and many others. We are utilizing pre-processed preparation and trial informations sets in ARFF format to accomplish our intent.

Support Vector Machine ( SVM )

In formal footings, a support vector machine creates a hyperplane ( or a set of hyperplanes ) for the intent of categorization of informations. It can besides be used for arrested development or other intents.

Figure 1 Writer: Cyc ( Wikipedia ) SVM theoretical account in a finite infinite is a set of points that are mapped in a manner that separate classs are split by a spread that must be every bit broad as possible. The most effectual hyperplane is the 1 that divides the classs in a manner that the distance between the hyperplane and the closest informations point to that plane is maximal. Better generalisation by the classier is normally achieved if this border is larger.

E.g. In figure 1, H3 does non split the classs at all. H2, even though it divides the classs right, the border is excessively little. H2 on the other manus is the best hyperplane that provides the maximal border and hence a better categorization. Such a plane is called maximum-margin hyperplane and the additive classifier defined by it is called maximal border classifier.

It ‘s non so straightforward nevertheless. Notice the undermentioned figure:

Figure 2 Kernel Function function

In a finite input infinite ( left ) , the information points seldom could be separated by a consecutive line. This finite infinite can be mapped to a significantly higher infinite so that the separation could be achieved much easy. The information points on the left demand to be someway rearranged so that they become linearly dissociable ( right ) . This procedure of rearrangement is called Mapping. This is done utilizing the meat maps. Cross merchandises need to be calculated by SVM in footings of the variables in the primary finite infinite.

These cross merchandises are defined in footings of a map known as Kernel Function K ( x, Y ) . Different Kernel maps use different methods in order to accomplish effectual categorization. In simple footings, all the SVM has to make is to happen an optimal line that good separates the information categories and categorizes them efficaciously.

Types of Kernel Functions:

The figure below lists the normally used meats maps:

Figure 3 Kernel Functions ( Apr 2010, Chih-Wei Hsu, Chih-Chung Chang, and Chih-Jen Lin )

The most widely used meat is RBF meat. Linear meat is instance of RBF meat for specific property values. When the figure of cases is less than figure of characteristics so additive meat is the best pick. When there is non any additive relationship between properties and cases so RBF meat is used to map the values to a higher dimensional infinite. The Sigmoid and Polynomial meats have really specific usage instances.


LIBSVM is a library for Support Vector Machines ( SVMs ) . I can besides be called as a negligee that provides the SVM algorithms to be used with the aid of certain maps. This facilitates the users to utilize SVM as a tool.

The LIBSVM library provides assorted maps in the signifier of bids. Each bid has certain figure of parametric quantities that could be passed to it. Depending on the parametric quantities and the bid that is being executed, we could accomplish different types of categorizations. The library is non restricted nevertheless to merely categorization related maps. E.g. svm-scale is a tool provided in the LIBSVM library that is used for scaling input informations file. Following is a elaborate illustration of the svm-predict bid in LIBSVM that predicts the mark values of the trial informations:

Use: svm-predict [ options ] test_file model_file output file


-b probability_estimates: whether to foretell chance estimations, 0 or 1 ( default 0 ) ; for one-class SVM merely 0 is supported

model_file is the theoretical account file generated by svm-train.

test_file is the trial informations you want to foretell.

svm-predict will bring forth end product in the output file.

In our undertaking, we have used assorted LIBSVM maps in order to do anticipations and achieve text categorization in the trial information. The major maps are svm_train and svm_predict.


It stands for Waikato Environment for Knowledge Analysis and is a suite that contains package for machine acquisition. It provides an assembly of visual image tools and algorithms for informations analysis and prognostic mold. These tools and algorithms are available for disposal via a graphical user interface ( GUI ) , besides a portion of Weka.

Weka provides support for informations excavation activities that includes categorization, visual image, preprocessing of informations, bunch, arrested development and characteristic choice. The Weka adventurer is the primary GUI for interfacing with the user. In simple footings, Weka provides an environment to lade the LIBSVM tools and use the power of those tools for text categorization. The rich GUI capabilities the Weka work bench besides facilitates us to expose the end product of our text categorization in a clear ocular mode.

In our undertaking, we have loaded the LIBSVM tools in Weka through the jar file. This file consists of all the tools required for text categorization. Then we use the API/function calls ( e.g. svm-predict ) to treat our text as per the demands. We have created two frames. One is for uploading the preparation informations set and trial informations set along with a screen to expose assorted end products and consequences of our text categorization. The other frame is used to alter the different input parametric quantities like meat map, capacity etc. and supply a graphical end product that displays the assorted categories, their properties and the categorizations.

Input signal

The inputs to our application are ARFF ( Attribute Relation File Format ) files. We upload a preparation information set every bit good as a trial information set as input to our classifier. Based on the preparation informations set, the LIBSVM classifier generates a theoretical account and so analyses the trial informations file harmonizing to this theoretical account. It makes anticipation based on what it has “ learnt ” and so generates matching end product.

ARFF files are apparent text files that contain the description of assorted dealingss, their properties and all their cases. Each property has a name, informations type and a value scope.

Figure 4 Sample ARFF file ( Holmes, G. , Donkin, A. , and Witten, I. H. Weka: A Machine Learning Workbench ) In the first base on balls, our application goes through all the cases of this preparation informations set ARFF file and “ learns ” . In the 2nd base on balls, when it encounters an unanticipated case in the trial information set, it makes a judgement based on the theoretical account that was developed after the first base on balls. Depending on the meat map, cost, gamma and other factors, it may right or falsely sort that case.

End product

The end product of our application comprises of two graphs:

SVM truth categorization

In this graph, we show the truth of the each single category in footings of right or falsely classified properties.

Fig 5. shows the Results spectator frame which has a graph incorporating values of each category. The ruddy 1s are the falsely classified cases and the right classified in blue.

Fig 5. Shows the Accuracy spectator frame which has a graph dwelling of True Positives ( TP ) , False Positives ( FP ) , Precision and Recall for single categories. Graphs for different categories can be viewed by snaping the left manus side tabular array.

Figure 5 Result Viewer – la1s

Figure 6 Accuracy Viewer – la1s

Classs and properties

This graph has two sub-types:

Nominal ( categorical ) value graph:

A nominal value means that a category can function as an identifier for a set of informations. E.g. A category named “ conditions ” can dwell of values: “ sunny ” , “ windy ” , “ rainy ” etc.

This graph displays the properties, values and counts of every case of a category.

Figure 7 Instance Visualizer – Nominal Properties

The left manus tabular array displays the properties for the selected case, entire figure of cases and the entire figure of properties. Right manus side contains the tabular array consisting of all the labels for a specific selected property and graph incorporating the counts for each.

Non nominal value graph:

A non nominal value means that the informations category is absolute and does non stand for a set other values.

This graph displays the mean, standard divergence, minimal value and maximal value of a peculiar category.

Figure 8 Instance Visualizer – Non Nominal

Procedure Flow

The preparation and trial informations sets are uploaded on Weka through one of the developed frames. The LIBSVM tools are accessed via our developed codification ( API/function calls ) . The classifier so generates a theoretical account after treating the preparation informations set ( input ) . The anticipation is so made by the SVM based on the theoretical account generated by the preparation informations set. The comparing is eventually made between the existent value and the classified value of the category. The graph based on these stats is generated which high spots these classified values and deduces the truth of the SVM. Fig. 9 below shows the procedure flow.

Text Classification Tweaks:

As mentioned earlier, there are certain parametric quantities that affect the truth of the text categorization. There are two of import factors that could be tweaked in order to acquire variable degrees of truth in text categorization are:

Cost ( C )

Gamma ( I? )

Figure 9 Procedure flow diagram

These parametric quantities straight affect the meat maps. So if we tweak their values it changes the meat map and accordingly the functions for the values of the datasets.

We experimented with these values and observed that the truth degree alterations inordinately with alterations in these two values. We have tabulated the consequences for a scope of input values for both Cost ( C ) and Gamma ( I? ) and their comparative Accuracy ( A ) for two of the datasets taken under consideration as under:

Dataset: La1s.wc Table 5-1 – LA Times

Dataset: Oh05.wc Table 5-2 – Oshomed

We followed a really simple attack to happen out the most suited values for Cost and Gamma. As per the observation done by libSVM Godheads the values of Cost and Gamma normally are in powers of 2. So we followed a grid hunt attack to hold different combinations of Cost and Gamma for a specific dataset and look into the truth for the related values. The consequences of that grid hunt are tabulated above every bit mentioned before.

Cost ( C )


Accuracy ( % )











3.0517578 E -05



3.0517578 E -05



4.8828125 E -04


Table 5aˆ‘1 La2s.wc

Cost ( C )


Accuracy ( % )





3.0517578 E -05






4.8828125 E -04






4.8828125 E -04


Table 5aˆ‘2 Oh05.wc

Datasets used

The dataset being used for this undertaking is from Geroge Forman from HP Labs. They contain 19 multi-class text datasets, whose word count characteristic vectors have been extracted. The jobs come from LA Times, TREC, OHSUMED, etc. and the informations were originally converted to word counts by George Forman.

The datasets range from newspapers to medical related text paperss. Almost all of the datasets have one characteristic in common, Number of properties are manner higher than figure of cases. As mentioned above in the meat subdivision we used additive meat for the creative activity of the theoretical account as the figure of properties are manner greater than the figure of cases.


One of the most of import points we would wish to pull from our experiment is the generalised theoretical account developed from our trial informations matched the sensitiveness or information. This was portrayed in graphical charts. We besides identified important alterations in SVM end product on altering subsequent input values.

We developed a GUI application based on Java Swings to demo the graphical end products after executing SVM preparation and categorization. Few of the Kernel maps have besides been included in this survey.

Data excavation is a widely used construct today and every company needs to implement algorithms. These vary in assorted sizes from a individual local computing machine to clients and waiters stretching over immense platforms. And Support vector machines have proven powerful categorization algorithms.