Permanently Delete Messages. You can avoid the multiple-step process of deleting messages, then emptying the Deleting Items folder by simply holding down the Shift key when you hit Delete for any message. You’ll be asked if you want to permanently delete the message. If you’ve selected lots of messages for deletion, it works the same way. Just hold down the Shift key as you click Delete.