This point sucks. Because you're cruising along feeling good, feeling pumped, feeling like you could do more, but you CAN'T. Not yet. You have to wait for your body to fully adjust before you up the ante.
Of course there also comes a point in every exercise routine where you sprain an ankle or pull a muscle or twist a finger, and that sucks more. But still.