GPT

GPT stands for Generative Pre-trained Transformer. It is a family of LLMs developed by OpenAI that popularized the modern AI boom, starting with GPT-1 in 2018 and evolving through GPT-4o and beyond.

Definition

GPT (Generative Pre-trained Transformer) is a series of large language models created by OpenAI that established the modern paradigm of pre-training a Transformer on web-scale text, then fine-tuning for specific tasks. The acronym breaks down as: Generative (produces text), Pre-trained (trained on unlabeled data before task-specific tuning), Transformer (the neural network architecture based on self-attention). GPT-3 (2020) was a watershed moment that demonstrated scaling laws: bigger models with more data and compute predictably outperformed smaller ones. GPT-3.5 powered the original ChatGPT (2022), triggering the mainstream AI explosion. GPT-4 added multimodal capabilities and improved reasoning. GPT-4o (2024) unified text, vision, and audio processing. The GPT lineage has been the most influential model family in modern AI, though competitors (Claude, Gemini, Llama) have closed the gap significantly.

The GPT Lineage

GPT-1 (2018): 117M parameters. Proved the viability of generative pre-training for NLP tasks.

GPT-2 (2019): 1.5B parameters. OpenAI initially withheld release due to safety concerns. Demonstrated that bigger models produce qualitatively better text.

GPT-3 (2020): 175B parameters. Showed scaling laws in action. Zero-shot and few-shot learning emerged as capabilities not explicitly trained for. The API product launched.

GPT-3.5 / InstructGPT (2022): Fine-tuned with RLHF. Powered ChatGPT. Made LLMs useful for everyday tasks.

GPT-4 (2023): Multimodal (text + image input). Improved reasoning, longer context. Passed the bar exam in the 90th percentile.

GPT-4o (2024): Native multimodality (text, vision, audio). Faster, cheaper, with voice conversation capabilities.

Architecture Details

GPT models use a decoder-only Transformer architecture. Unlike encoder-decoder models (like the original Transformer for translation), GPT has only the decoder stack. It uses causal (masked) self-attention, meaning each token can only attend to tokens before it, not after. This makes it autoregressive: it generates one token at a time, left to right. The architecture choices massive scale, decoder-only, autoregressive generation have been widely adopted beyond OpenAI, including by Meta (Llama), Mistral, and others.

GPT as a Platform

GPT is accessible via OpenAI's API (chat completions, embeddings, assistants) and through ChatGPT (consumer product). The API follows a pay-per-token model with different pricing tiers for different models. Enterprise features include fine-tuning, custom models, data privacy guarantees, and dedicated capacity. The ecosystem includes plugins, GPTs (custom versions of ChatGPT), and a growing library of third-party integrations.

In code

import openai

response = openai.ChatCompletion.create(
    model="gpt-4o",
    messages=[
        {"role": "system", "content": "You are a technical writer."},
        {"role": "user", "content": "Explain what GPT stands for."}
    ],
    temperature=0.7,
    max_tokens=200
)
print(response.choices[0].message.content)

Common questions

Q: Is GPT free to use?

A: ChatGPT has a free tier with limited features and rate limits. The API is pay-per-use. GPT-4o access requires a ChatGPT Plus subscription ($20/month) or API credits. There is no fully free high-performance GPT.

Q: What is the difference between GPT and ChatGPT?

A: GPT is the underlying model. ChatGPT is the consumer-facing product that uses GPT models under the hood. Think of GPT as the engine and ChatGPT as the car.

Q: Are there open-source alternatives to GPT?

A: Yes. Meta's Llama series, Mistral, Qwen, and DeepSeek are competitive open-weight models. They may not match GPT-4o on every benchmark but offer self-hosting, lower cost, and data privacy advantages.

Key takeaways:

• GPT = Generative Pre-trained Transformer. The most influential LLM family.

• Decoder-only Transformer architecture with causal attention.

• Scaling laws drove progress: more parameters, data, and compute predictably improved performance.

• GPT-4o is the current flagship with native multimodal capabilities.