After growing up in Oxford and working with foreign visitors a lot in my late teens, as well as travelling abroad myself, I learnt quickly not to make assumptions when it comes to Americans and Canadians. I choose, instead, to ask them where they are from. Or, often, Canadians will intentionally carry a bag bearing a maple leaf to let people know.
And you are so right that there are few tourists from across the pond who make it to our beautiful part of the country. Again, working with tourists down here, I jump when I hear the sound of a North American accent, they are so unusual. They are seriously missing out!