I read an article many years ago about altitude sickness. Your level of fitness has no bearing whatsoever, and it was pointed out that many that succumb are top athletes thinking they will be immune. It takes a body going from sea level to 5000' 24 hours to generate enough hemoglobin for normal oxygenation, and going from s.l. to 10,000 takes 3 days!
So, yeah, take the first day easy, stay hydrated, and then go have fun.