---
title: "Concepts Overview"
description: "Deep dive into the fundamental concepts powering Angelo and modern LLMs."
canonical_url: "https://angeloai.vercel.app/docs/concepts"
markdown_url: "https://angeloai.vercel.app/docs/concepts.md"
last_updated: "2018-10-20"
---

# Concepts Overview
URL: /docs/concepts
LLM index: /llms.txt
Description: Deep dive into the fundamental concepts powering Angelo and modern LLMs.

# Concepts Overview

Welcome to the **Concepts** section. This directory contains a deep dive into the foundational architecture and theory behind Large Language Models, with a specific focus on the decoder-only architecture used by Angelo.

Use the left sidebar to navigate through the detailed subtopics. Each subtopic explores a specific piece of the Transformer puzzle in plain English.

## What's Inside?

- **[Transformer Types](/docs/concepts/01-transformer-types):** The difference between Encoder, Decoder, and Encoder-Decoder models.
- **[Tokenization & Vocabulary](/docs/concepts/02-tokenization):** How text is chopped up into subwords via BPE.
- **[Embeddings & Position](/docs/concepts/03-embeddings):** Translating discrete tokens into continuous semantic vector space with RoPE.
- **[Attention Mechanism](/docs/concepts/04-attention):** The core engine of the transformer-how tokens "look" at each other.
- **[Feed-Forward & Activations](/docs/concepts/05-feed-forward):** Processing individual tokens with SwiGLU gating.
- **[Normalization & Residuals](/docs/concepts/06-normalization):** Keeping deep networks mathematically stable with RMSNorm.
- **[LM Head & Generation](/docs/concepts/07-lm-head):** Converting vectors back into words and autoregressive sampling.
- **[Predictive Coding](/docs/concepts/08-predictive-coding):** A look at biologically inspired alternatives to backpropagation.

Explore the sidebar on the left to begin your journey through the architecture.

## Sitemap

See the full [sitemap](/sitemap.md) for all pages.
Docs-scoped sitemap: [/docs/sitemap.md](/docs/sitemap.md).
Well-known sitemap: [/.well-known/sitemap.md](/.well-known/sitemap.md).