Advances in technology for digital image capture have given rise to information overload problems in the geosciences and fields that rely on geospatial image retrieval. To help address such imagery information overload, our research is developing methods to extract and apply contextual knowledge relating user task goals to the images being used. As users analyze imagery retrieved to support specific tasks, multiple relevant images and salient image content can be captured together with task annotations to provide a basis for contextual knowledge management support. This paper describes how our environment for image interaction leverages captured task-based knowledge to support adaptive presentation and navigation in the space of available imagery.