Objectives: We examined bidirectional, time-ordered associations between age-related changes in depressive symptoms and memory. Method: Data came from 107,599 community-dwelling adults, aged 49–90 years, who participated in the Survey of Health, Ageing, and Retirement in Europe (SHARE). Depressive symptoms were measured with the EURO-D inventory, and memory was evaluated as delayed recall of a 10-word list. Participants were assessed up to five times at 2-year intervals. Dynamic structural equation models were used to estimate longitudinal and time-ordered (lead-lag) relations between depressive symptoms and memory performance. Results: Depressive symptoms increased and memory scores decreased across the observed age range, with worsening mostly evident after age 62 years. These long-term changes were moderately negatively correlated (r = −.53, p < .001). A time-ordered effect emerged such that age-specific memory deficits preceded shorter-term increases in depression symptoms. This effect can be translated such that each 1-point decrement on a 10-point memory scale at a given age predicted a 14.5% increased risk for depression two years later. Statistical adjustment for covariates (sex, education, re-test, smoking, and body mass index) had little influence on these associations. Conclusion: In later adulthood, lower memory performance at a given age predicts subsequent 2-year increases in depressive symptoms.