“This attack on me is, first and foremost, personal,” says Cadman