generative models through the lens of probability

Assume we have an input vector \( \vec{x} \) and an output \( y \).

Based on temperature and precipitation (\( \vec{x} \)), we wish to determine if it will snow (\( y \)).

\( p(y|\vec{x}) \)	Given a known input \( \vec{x} \), probability of getting output \( y \) The temperature is 30 \( ^{\circ} \) F and precipitation is 12%. What is the probability it snows?	\( \vec{x} \) known, \( y \) unknown	supervised learning
\( p(\vec{x}) \)	Probability of observing input \( \vec{x} \), regardless of the output What is the probability today's temperature is 30 \( ^{\circ} \) F and precipitation is 12%?	\( \vec{x} \) unknown, \( y \) n/a	unsupervised learning
\( p(\vec{x}, y) \)	Probability of observing both \( \vec{x} \) and \( y \) together What is the probability the temperature is 30 \( ^{\circ} \) F, precipitation is 12%, and it snows?	\( \vec{x} \) unknown, \( y \) unknown	generative models

Intuition:

Supervised learning is significantly more intuitive than the other two. A supervised model functions in this format: given some information, we'd like to know some outcome. Given today's temperature and chance of precipitation, will it snow? (Given \( \vec{x} \), find \( y \).)

Unsupervised learning tries to find patterns in the given data. If \( \vec{x} \) is the temperature and precipitation, unsupervised learning might learn that certain ranges of \( \vec{x} \) are more common than others, suggesting recurring weather patterns.

Generative models learn how the inputs (\( \vec{x} \)s) and outputs (\( y \)s) co-occur, which then enables them to create new instances of data that resemble the original dataset. For example, after analyzing past examples of previously-occurring instances of temperature, precipitation (\( \vec{x} \)), and the occurrence of snow (\( y \)), a generative model could generate new instances of weather data that have a high likelihood of resulting in snow, effectively creating realistic weather scenarios.

Note that:

\( p(y | \vec{x}) = \frac{p(\vec{x}, y)}{p(\vec{x})} \)