We investigated whether top-down search templates for target-defining features can be established via reinforcement learning of the search rule alone, that is, without instruction to search for a specific target feature. In a spatial cueing experiment, participants reported the orientation of a T inside the target stimulus without knowing which exact feature defines the target. Via trial and error, they learned that a specific color defines the target. After the learning block, they were explicitly instructed to search for the same color to find the target. In both blocks, cues matching the searched-for target features elicited cueing effects (faster reaction time when the cue appeared at the same position as the target, compared to a different position), whereas non-matching cues (without a searched-for feature) did not. In the learning block, the cueing effect built up over the first 150 trials, before remaining stable for the rest of the block.