Large repositories of geo-spatial images are employed to support tasks such as intelligence operations, recreational and professional mapping, urban planning and touristic systems. As imagery is retrieved to support a specific task, the interactions, analyses, and conclusions - based on relevant imagery - may be captured together with the images as an encapsulated experience. We are developing annotation-based image retrieval techniques and knowledge-management support for large geo-spatial image repositories incorporating sketch-based querying for image retrieval and manipulation. Leveraging interactive task knowledge to support current user goals requires a smart system that can perform knowledge capture. This paper describes our initial work in intelligent annotation-based image retrieval for geo-spatial imagery systems and presents the task-based knowledge management environment we have developed to support such retrieval.