One of the most important features of human vision is that it is selective. It flexibly samples the environment on the basis of what is relevant to our current tasks – tasks such as driving, finding a product in the supermarket, or collecting a child from school. This means that the brain maintains some representation of what we are currently looking for, be it a traffic sign, a coffee brand, or a kid’s face. This “picture in your head”, or “template” as it is often referred to, remains a huge mystery. Current models of visual exploration assume it to be there, but without making explicit what its properties and mechanisms are. The proposed program will change this. Based on a three-state model of human memory, I have identified several lines of research that collectively lead to a thorough understanding of one of the most fundamental concepts of perceptual theory. These lines systematically investigate what distinguishes the template from other types of memory, how many templates can be active at a time, how we set up, change, and abandon the template with changing task demands, and how training changes the nature, dynamics and capacity of the template. Each of these lines builds on my extensive track record in investigating memory and attention, and combines modern psychological, brain imaging, and modeling methods with innovative experimental paradigms. The result will be a comprehensive understanding of visual selection, of what makes human perception so adaptive. As such it will carry important implications for the fields of psychology and neuroscience, as well as clinical fields. In addition, the results are of direct relevance for how humans interact with ever more intelligent information systems, thus promoting safety and efficiency of everyday life.