You should discuss this with your son...he's the kid...you're the dad.
Put it on him to make sure you guys get to do this when the timing is better. You know...the guilt approach
What if...you go pick up the bike, and bring it to your house.
Then, later in the summer, he makes the trip down, and you go play?
Buzzkill moment...do you want to leave your wife alone w/the neighbor next door?