Does removing milk from your diet cause tiredness?

I am casein intolerant and am not supposed to eat many milk products. I've had this condition for a couple years now. I've also found that I'm always tired. Could there be a possible connection between the two?