Machine learning

Machine learning (ML) is a subfield of artificial intelligence in which algorithms enable computer systems to learn patterns from data and make decisions or predictions with minimal human intervention. The discipline draws on statistics, linear algebra, and computational theory, and has become one of the most influential areas of computer science since the late twentieth century.

History and development

The conceptual roots of machine learning trace back to the mid-twentieth century. Alan Turing proposed in his 1950 paper Computing Machinery and Intelligence that machines could in principle learn from experience, a question framed by the now-famous Turing Test. Frank Rosenblatt's development of the perceptron in 1958 at Cornell Aeronautical Laboratory represented an early practical step toward learning systems.

The field went through periods of intense optimism and subsequent funding contractions — sometimes called AI winters — in the 1970s and 1980s. The rise of backpropagation algorithms in the mid-1980s, associated with Geoffrey Hinton, David Rumelhart, and Ronald Williams, revived interest in neural networks. The late 1990s and 2000s saw the ascendancy of support vector machines and ensemble methods such as random forests.

The modern era of machine learning is commonly dated to around 2012, when a deep convolutional neural network (AlexNet) dramatically outperformed competing methods on the ImageNet image classification benchmark. Since then, advances in graphics processing units (GPUs), the availability of large datasets, and algorithmic improvements have produced rapid progress.

Core concepts and terminology

Data and features

Machine learning systems are trained on datasets — collections of examples, each described by features (measurable properties) and, in supervised settings, a label or target value. The quality, quantity, and representativeness of training data are central determinants of model performance. Poorly curated or biased data can lead to models that perform poorly or perpetuate existing societal biases.

Types of learning

Machine learning methods are conventionally grouped into three broad paradigms:

• Supervised learning: The algorithm is trained on labelled examples, learning a mapping from inputs to outputs. Common tasks include classification and regression.
• Unsupervised learning: The algorithm finds structure in unlabelled data. Prominent techniques include clustering (e.g., k-means) and dimensionality reduction (e.g., principal component analysis).
• Reinforcement learning: An agent learns by interacting with an environment and receiving reward or penalty signals, optimising a long-term cumulative reward. This paradigm underlies systems such as game-playing agents and robotics controllers.

Additional paradigms include semi-supervised learning, self-supervised learning, and transfer learning, in which knowledge gained from one domain is applied to another.

Generalisation and overfitting

A central challenge in machine learning is generalisation — the ability of a trained model to perform well on unseen data rather than merely memorising the training set. When a model fits training data too closely, capturing noise rather than underlying patterns, it is said to overfit. Techniques to mitigate overfitting include regularisation, dropout (in neural networks), cross-validation, and the use of held-out validation and test sets.

Major algorithm families

Linear models

Linear regression and logistic regression are foundational supervised learning models that assume a linear relationship between inputs and outputs. Despite their simplicity, they remain widely used in settings where interpretability is important, such as finance and healthcare.

Decision trees and ensemble methods

Decision trees partition the input space through a series of binary splits. Ensemble methods such as bagging, boosting, and random forests combine multiple trees to achieve higher accuracy and robustness. Gradient boosting frameworks such as XGBoost have achieved state-of-the-art results on many tabular data tasks.

Neural networks and deep learning

Deep learning refers to machine learning using neural networks with many layers (depth). Deep architectures — including convolutional neural networks (CNNs) for image data, recurrent neural networks (RNNs) for sequential data, and transformer models for language — have achieved remarkable performance across domains. The transformer architecture, introduced by Vaswani et al. in 2017, became the foundation for large language models such as those in the GPT series and BERT.

Probabilistic and Bayesian methods

Bayesian approaches incorporate prior knowledge and quantify uncertainty in model parameters and predictions. Gaussian processes, Bayesian networks, and variational inference are examples of probabilistic machine learning methods valued in scientific and safety-critical applications.

Applications

Machine learning is applied across virtually every industry:

• Natural language processing: Machine translation, sentiment analysis, chatbots, and large language models.
• Computer vision: Object detection, facial recognition, medical image analysis, and autonomous driving perception.
• Healthcare: Disease diagnosis from imaging, drug discovery, genomic analysis, and patient outcome prediction.
• Finance: Credit scoring, fraud detection, algorithmic trading, and risk modelling.
• Recommendation systems: Personalised content on streaming platforms and e-commerce sites.
• Scientific research: Protein structure prediction (e.g., AlphaFold), climate modelling, and particle physics event classification.

Ethics, bias, and governance

The deployment of machine learning systems raises significant ethical and societal concerns. Models trained on historically biased data can produce discriminatory outcomes in areas such as hiring, lending, and criminal justice. Transparency and explainability — the degree to which a model's decisions can be understood by humans — are subjects of active research under the heading of explainable AI.

Regulatory bodies in multiple jurisdictions have begun to address machine learning governance. The European Union's Artificial Intelligence Act, provisionally agreed in 2023 and expected to enter force in the mid-2020s, classifies AI applications by risk level and imposes corresponding obligations. The degree to which existing frameworks adequately govern rapidly evolving ML capabilities remains a subject of ongoing debate.

Privacy is also a concern, since machine learning models can sometimes be made to reveal sensitive information from their training data through so-called membership inference or model inversion attacks.

Frequently asked questions

Is machine learning the same as artificial intelligence?

Machine learning is a subfield of artificial intelligence, not a synonym for it. AI is the broader discipline concerned with creating systems that exhibit intelligent behaviour; machine learning is one approach to achieving that goal by enabling systems to learn from data.

Is machine learning the same as deep learning?

Deep learning is a subset of machine learning that uses multi-layered neural networks. All deep learning is machine learning, but machine learning also encompasses many methods that do not use deep neural networks, such as decision trees, support vector machines, and linear models.

Is machine learning supervised or unsupervised?

Machine learning encompasses both supervised and unsupervised learning, as well as reinforcement learning and other paradigms. The appropriate approach depends on the availability of labelled data and the nature of the task.

Is machine learning a recent invention?

The theoretical foundations of machine learning date to the 1950s, with practical methods developing steadily from the 1960s onward. The current period of rapid advancement is driven largely by increases in computational power and data availability since roughly 2010, making it feel recent in its most prominent modern form.

Is machine learning reliable enough for high-stakes decisions?

Reliability in high-stakes applications depends heavily on the quality of training data, model design, validation rigour, and ongoing monitoring. Machine learning systems have demonstrated strong performance in several medical and scientific domains, but concerns about bias, interpretability, and failure modes mean that human oversight and robust testing remain essential in critical settings.