Suggested Needle Size

Who comes up with the suggested needle size for each yarn?

I have been finding with some yarns, even when I use the needle size suggested on the ball band my stitches are wonky and irregular. Sure I get the gauge also given by the ball band, but I don’t like the look of the stitches when they are not neat. This doesn’t happen with all yarns so I am sure it has something to do with the fiber content more than anything else. But, this is causing a real problem when I want to use the same yarn called for by the designer, or when I buy yarn with the same suggested gauge, thinking it would work for the pattern, only to find out I don’t like the look of the yarn in that gauge or with that size need, which leads me to having to rework the entire pattern.

Does anyone else have this problem, or is it just me? Am I missing some unknown magically secret that makes the stitches neater?