Generative Networks

These notes grew out of one of the courses I teach at the University of Oviedo. Over the years I kept extending the material, and it ended up being useful to students from other degrees too. So I decided to publish it in this format, with the idea of making it a free, accessible resource for learning about generative models.

The course code is available in this GitHub repository.

Chapter 1
Introduction
Nahuel Costa
Who is this course for? An introduction to generative models and their applications.
Chapter 2
Probabilistic Machine Learning
Nahuel Costa
A high-level view of generative models. A refresher on probability and Bayesian inference.
Chapter 3
Code
GAN Networks
Nahuel Costa
Architecture, applications, and where they're still useful today.
Chapter 4
Code
Variational Autoencoders
Nahuel Costa
From the classic autoencoder to the VAE: ELBO, posterior collapse, variants, and compression.
Chapter 5
Code
Flow-based Models
Nahuel Costa
Normalizing flows and flow matching: how to transform distributions.
Chapter 6
Code
Diffusion Models
Nahuel Costa
Forward and reverse diffusion, training, Stable Diffusion, and the relationship with Flow Matching.
Chapter 7
Code
Transformers
Nahuel Costa
From words to vectors, attention mechanisms, architecture, positional encodings, and Vision Transformers.
Chapter 8
Code
Foundation Models
Nahuel Costa
From supervised learning to self-supervised pretraining: encoder/decoder architectures, masked modeling, contrastive learning, CPC, and JEPA.
Chapter 9
Code
Efficiency and Cost Reduction
Nahuel Costa
Knowledge distillation, Low-Rank Adaptation (LoRA), and quantization: how to reduce training and inference cost for large generative models.

Generative Networks

Introduction

Probabilistic Machine Learning

GAN Networks

Variational Autoencoders

Flow-based Models

Diffusion Models

Transformers

Foundation Models

Efficiency and Cost Reduction