A/B tests might be useful in finding a color that will get people to click a link more often, but it can’t produce a product that feels like a pleasing and integrated whole.

Ken Kocienda

I have the same issue with A/B Testing as I do surveying for Net Promoter Score which is it doesn’t help you understand why people feel the way they do, just that they do.