Thank you for testing and all the details!
I’m still not sure if this is an error in my code or if the problem is located in the Pi/Linux/JavaAudio/USB combination.
There is no difference on Java side, whether you stream to the default audio device or any other device/mixer.
Its good to know that you can use (most of) your USB-Cards for now, but I’ll keep investigating, how the playback can be improved.