In DH, he was insecure and anxious over his feelings for Hermione and though she liked Harry and didn’t know how to handle it properly. It was war-time, so tensions were already high. Moreover he was wearing the Horcrux locket. I think it was perfectly understandable that he left. I think people just use this as a bad excuse to justify H/Hr