Cold sores and my boyfriend?

So my boyfriend and I noticed a cold sore on my lip. Then he went to kiss me on the cheek and I accidentally kissed him on the lips. He is now taking the standpoint that if I've transfered it to him that he could never forgive me and breaking up with me would be an appropriate action. Is he right? What do I do?