Just remember, it's not our world; we're just watching it.
Does it matter when film and TV get it wrong or resort to cliches in filming? Should they (let alone could they) get around this? New York and London are probably two places that have most often suffered the indignities of poor location shooting substitutes and hackneyed backdrop items (Hackney black cabs doesn't get close to the regular red bus and phone boxes that appear); but surely other locations suffer this to - we just don't have the same knowledge and reference points to correct it.
It comes back to a key Freudian concept really; the sense of the uncanny - the unhomely, the recognisable-but-yet-not-home sense of place and self. New York is SO well filmed, and yet so wrong in our mind's eyes because editing of location makes characters turn different corners to the ones that exist, into different locations to the ones that are there. No wonder my dreams have so often featured places that are familiar but bigger, smaller, different inside: its merely the translation of what I adapt to every time I watch film and TV. I'm constantly adjusting things against what I know (or believe) to be the real world. And likewise when I visit places, I apply the same technique - isn't there a restaurant around this corner?
Does it destroy our viewing pleasure? Or our visiting pleasure? It depends how much you expect. But it definitely heightens that 'uncanny' feeling.