Not if you mark him as avoid. You can also report him for stalking.
Now YOU have to join HIS modded lobby for him to derank you.
Mark him as avoid, block all communications, change your status to offline for a month and NEVER contact him. In 30 days he will most likely be gone.
But as I said if he send you a bad PM you can report him to XBL. If you file good reports they see them within 1 business day or two depending on when you report.