Guy I used to go out with? Am I being mental here?

i used to go out with a guy a couple of years back. we got on really really well and he seemed like really into me. i really liked him too, but at the time though i was at college and had nothing to offer him so i didn't feel like i could take it any further with him. soon after i told him that, he packed up and moved to the other side of the world on his own to go backpacking. we've kept in touch all this time and i've always felt like i really missed out by not giving us a chance together. last week, he casually asked me over there, and i've suddenly said yes. he said he'll look after me there and help me get on my feet and stuff. i'm like right in the middle of getting visa's (to australia, and i live in london) and booking flights and handing my notice in at my job. Am I making the wrong decision here? I don't want to get there and think...I have made a HUGE mistake..and I don't want him thinking the same thing!!