I don't think you're there yet. Maybe another 2 to 3 years to build up your portfolio a bit so you can have all of your planned fun in retirement.
Also, think about the possibility of your wife not being able to work anymore. Disability, layoff, ageism does happen when we least expect it.
Or...