Data mining is the practise of sifting through vast amounts of data to find relevant or important information. Decision-makers, on the other hand, require access to smaller, more specialised pieces of data. Facts mining is used by businesses to gain business information and to uncover specific data that can help them make better leadership and management decisions.
Data mining is the process of discovering solutions to problems you were not aware you were looking for. Exploring new data sources, for example, may lead to the discovery of causes for financial problems, underperforming personnel, and other issues. Quantifiable data reveals information that would otherwise be hidden from ordinary observation.
Many data analysts believe they are missing important information that will help their companies perform better as a result of information overload. Data mining experts sift through massive amounts of data to find trends and patterns.
Data mining can be done using a variety of software packages and analytical tools. The procedure can be automated or performed manually. Individual workers can use data mining to send customised requests for information to archives and databases, resulting in tailored results.
Data Mining Techniques
The extraction of hidden patterns in data using various data mining approaches can be divided into two categories:
- Description methods
- Prediction methods
The data description approaches concentrate on understanding and interpreting the data through the use of examples and the way the underlying data links to its components.
According to the study, the goal of prediction-oriented models is to build a behavioural model with new samples that can forecast values connected to the sample.
The data mining techniques that are utilised for data analysis are as follows:
1. Association
The discovery of association rules indicating attribute-value conditions that occur frequently together in a given set of data is referred to as association analysis.
For a market basket or transaction data analysis, association analysis is commonly utilised. Association rule mining is a key and rapidly evolving area of data mining study.
Associative classification is one approach of association-based categorization that consists of two parts:
- Apriori – a modified version of the traditional association rule mining technique, used to produce association instructions.
- Build a classifier – based on the identified association rules.
2. Classification
- Classification is the process of locating a set of models (or functions) that explain and separate data classes or concepts in order to use the model to forecast the class of unknown items.
- The model that is determined is based on an examination of a set of training data facts (data objects whose class label is known).
- The resulting model can be expressed in a variety of ways, including classification rules, decision trees, and neural networks.
Methods include:
- Decision Tree
- SVM (Support Vector Machine)
- Generalized Linear Models
- Bayesian Classification
- Classification by Backpropagation
- K-NN Classifier
- Rule-Based Classification
- Frequent-Pattern Based Classification
- Rough Set Theory
- Fuzzy Logic
Decision Trees:
A decision tree is a flowchart-like structure in which:
- Each node represents a test on an attribute value.
- Each branch represents a test outcome.
- Leaves indicate classes or class distributions.
Decision trees are nonparametric and simple to read, especially when smaller. They work well for discrete-valued functions but do not simplify easily to some Boolean problems.
3. Prediction
- Prediction, like classification, is a two-step procedure.
- We do not use the phrase “class label attribute” because prediction deals with continuous values rather than categorical ones.
- The property is also known as the projected attribute.
Prediction = development and use of a model to:
- Determine the class of an unlabelled object.
- Estimate the value or value ranges of an attribute.
4. Clustering
Unlike classification and prediction, clustering analyses data objects without class labels.
- Training data usually has no class labels.
- Labels can be generated through clustering.
- The goal is to maximise intra-class similarity while reducing inter-class similarity.
Clustering can:
- Group similar events together.
- Help build classification models.
- Organise observations into a hierarchy of classes.
5. Regression
Regression is a statistical modelling strategy that uses previously obtained data to predict a continuous quantity for fresh observations.
- Also called the Continuous Value Classifier.
- Two main types: Linear RegressionMultiple Linear Regression
6. Artificial Neural Network (ANN) Classifier Method
An artificial neural network (ANN), or neural network, is a process model inspired by biological neural networks.
- Made of interconnected input/output units with weights.
- Learns by adjusting weights during training.
- Also called connectionist learning.
Key features:
- Require long training cycles.
- Network topology often defined empirically.
- Low interpretability (black box problem).
Advantages:
- High tolerance for noisy input.
- Can classify unseen patterns.
- Rule extraction methods are improving their usefulness.
Common types:
- Perceptron
- Multilayer Perceptron
7. Outlier Detection
Some data objects do not conform to overall behaviour — these are outliers.
Outlier Mining can be done using:
- Statistical tests (distribution/probability-based).
- Distance measures (few neighbours = outlier).
- Deviation-based strategies (focus on unusual variances).
8. Genetic Algorithm
Genetic algorithms are adaptive heuristic search algorithms, inspired by natural selection and genetics.
- Intelligent random search guided by historical data.
- Frequently used for optimisation and search problems.
- Mimic “survival of the fittest” in successive generations.
Each generation:
- Population of individuals created.
- Each represents a potential solution.
- Represented by a string (like a chromosome).
Data Mining Tools and Platforms
- Data mining tools have existed for a long time, but with big data analytics, their importance has grown.
- The market offers a wide variety of tools, ranging from basic to advanced.
Examples:
- Simple tools like MS Excel vs. advanced tools like IBM SPSS Modeler.
- Stand-alone tools or embedded in ERP/transaction processing systems.
- Open-source tools (e.g., Weka) vs. commercial products.
- Text-based tools (need coding) vs. GUI-based drag-and-drop tools.
- Some tools work only with proprietary formats, others support multiple standard data formats.
