Have you ever heard the phrase 'burning the candle at both ends'?

I think you would do well to block it if you can and sleep regularly. Maybe do so anyway just to test the theory? If you are drawing from one of them and you deactivate your magic the effect should stop, right?