Why Mercantilism bothered Adam Smith

...one reason which I think is particularly interesting is that the rulers only cared about the types of trades that brought gold / silver money into Britain. Trade within Britain did not meet this criterion, though it was this trade that impacted people's lives the most.

In Wealth of Nations (Book IV, Chapter 1), he laments this fact.
Either its rulers did not understand the major sources of wealth for the regular person or they didn't care. Either way, it is unfortunate, but at least it motivated Smith to write one of the most influential books in history.