Memory for objects and their location is a cornerstone of adequate cognitive functioning across the lifespan. Considering that human visual perception depends on the position of stimuli within the visual field, we posit that the position of objects in the environment may be a determinant aspect of mnemonic performance. In this study, a population of 25 young and 20 older adults completed a source-monitoring task with objects presented in the upper or lower visual field. Using standard Pr and multinomial processing tree analyses, we revealed that although familiarity-based item memory remained intact in older age, spatial memory was impaired for objects presented in the upper visual field. Spatial memory in aging is conditioned by the vertical position of information. These findings raise questions about the view that age-related spatial mnemonic deficits are attributable to associative dysfunctions and suggest that they could also originate from the altered encoding of object attributes.