We do this to him constantly – but he knew we were both sex addicts when he moved in