The mid-workout energy dip is real for me too, so I switched to shorter sessions with more rest between sets and that got me through without feeling wrecked after.

Mine crept up slowly too, nothing dramatic until maybe week five or six. Week four felt like nothing was happening at all and then it just quietly turned around.