Didn't try the "Let's go on a family outing for our anniversary, darling." Approach?
From a guy with 22 years in... Next time that happens, respond with "No, that's Thursday"... Then let the calendar prove you wrong and you can feign the dummy who was one day off....
2 things with this.
You didn't forget.
It being a day earlier, somehow makes you less of an arse than it being a day later.
Don't over use it. It's a one or two time use idea.
Best bet is to never forget again, but we men can get absent minded and need some help. Good luck.