Hey rascal.
The idea of “stealing” the speaker connection sounds useful.
I think this could be realized as advanced setting on speaker side.
Something like “Allow connection takeover by players”.