Meeting was at 5pm and I was 15 minutes early. At 5:30 I called the guy and he said he was still at work and would be there for another hour and a half, at least. I simply said forget it and hung up.
Heck, you should have said "no problem, I'll be waiting"---and gone home.
If he called and gave you any grief--tell him to Google "Golden Rule".