Not every Transformer is a chatbot. Since the architecture appeared in the 2017 paper "Attention Is All You Need," it has split into three distinct architectural patterns—encoder-only, decoder-only, and encoder-decoder—and each pattern maps cleanly onto a different family of tasks. Encoder-only models such as BERT excel at understanding input, like judging whether a sentence is positive or negative. Decoder-only models such as GPT build output one token at a time and dominate generation. Encoder-decoder models combine both halves for tasks that transform one sequence into another, like translation. The same core ideas also work outside language entirely: Vision Transformers process images as sequences of patches, and the architecture has been applied to protein folding and biological sequence analysis.
Three Pillars of Transformer Architecture
The Transformer rests on two structural pillars: the encoder and the decoder. The encoder reads and processes the input, transforming it into a format the model can work with—absorbing a sentence and reducing it to its essential representation. The decoder takes that understanding (or its own partial output) and produces results sequentially.
What makes this architecture different from its predecessors is self-attention: rather than processing tokens strictly in order, the model weighs relationships between all parts of the input at once. That single mechanism is why the same blueprint can absorb a sentence, an image, or a protein sequence.
The three architectural categories emerge from how designers combine these pillars. A teaching series on Transformers for social scientists by Lorcan McLaren identifies "three main categories of transformer-based model, each suited to different tasks": encoder-only, decoder-only, and encoder-decoder. The choice among them is not stylistic—it determines what the model can fundamentally do.
