We examined cognitive domains and brain regions associated with olfactory performance in cognitively unimpaired older adults (CU-OAs) and individuals with or at risk for Alzheimer's Disease (AD). We compared CU-OAs (N = 55), individuals with subjective cognitive decline (SCD, N = 55), mild cognitive impairment (MCI, N = 101), and AD (N = 45) on measures of olfactory function (Brief Smell Identification Test), cognition (episodic and semantic memory), and medial temporal lobe thickness and volume. Analyses controlled for age, sex, education, and total intracranial volume. Olfactory function decreased from SCD to MCI to AD. CU-OAs outperformed all groups except SCDs on tests of cognition and olfaction. Although these measures did not differ between the CU-OAs and SCDs, olfactory function correlated with episodic memory tests and with entorhinal cortex atrophy only in the SCD group. Olfactory function also correlated with hippocampal volume and right-hemisphere entorhinal cortex thickness in the MCI group. Olfactory dysfunction reflects medial temporal lobe integrity and memory performance in a group at risk for AD with normal cognition and olfaction.