Do the firm’s people ask good questions and then listen? Do they get you thinking about approaches you haven’t considered before? Are they curious and passionate about doing great work and getting results? If they disagree with you, do they present a thoughtful explanation why? A great agency will give you things you didn’t think to ask for.
This is a great article on how to pick a marketing firm.