Audio Lag when streaming over Wowza Server

When I switch between two Video File layers while streaming, the audio on the second video is lagging by the amount of time it takes to switch between the layers (i.e… if I turn off layer one and wait 2 seconds to take layer 2 live then the sound on layer 2 is delayed by 2 seconds).

I have found that if I have an audio layer live (but not actually sending any audio) at the bottom of the stack then there is no delay in the second video file.