He has two brothers, one older, one younger, who also knew how to have a good time.
"I think each of us at different times in our lives have thought we were he black sheep. But the truth is there's no black sheep in our family. We've all had our own sets of problems down through our lives, and we still have problems."
"Our parents have just had to be there for us. They're strong people, very spiritual people. They understand the human side of us."
Being called to be a preacher is a worthy calling, but it can also be hell on the kids.
"Being a preacher's kid, I didn't want to be looked at as someone who was too goodie-two-shoes. One that couldn't have fun and enjoy myself and have a normal childhood like other kids. So, I guess I felt like had something to prove. 'I'm going to show these people I can party just as hard,' which is kind of sad in a way. You're different, but you want to be accepted and be part of the crowd."
Some of his behavior can be traced back to his upbringing, but he also believes a lot of it has to do with the way he came out of the box at birth.
"Who I am is a combination of my mom and dad and the people who came before them. I'm just a fun loving person. I enjoy being out."
"Years ago, I did a lot of things I probably shouldn't have done," he clarifies. "I try not to do those things any more. But it's tough if you grew up in the church and you tried to uphold those values. It makes it hard on a kid."
Worley's partying seemingly had no adverse affects on his school studies as he went on to earn a degree in biology from the University of Northern Alabama and even teach high school for a year.
"My classes were what they called the 'lower level learners.' These kids had previously just had someone there to baby sit them for several years, and they were used to not really having to do anything."
"I told them at the beginning that we may not do it the conventional way, but we're going to learn some things. Nobody's going to pass the class without earning it."
Worley looks back on this short stint in front of a blackboard as a personal triumph.
"I was successful in getting some of the kids out of those classes and into some higher level learning classes because that's where they belonged. Kids are smart. They figured out they could get in there and not do anything. I picked those types out pretty quick. We had some smart kids who moved on to do better and also go on to college."
Songwriters may never talk to all the listeners their songs have touched, but professor Worley still runs into his former pupils now and again.
"I still see those kids - who are grown men now - here in my hometown. They always come up and say, 'That's the best class I ever had.'"
"A lot of good came out of it. I just wouldn't want to do it for a living."
Instead, writing and performing music has always been his number one career goal.
Such an ambition keeps him ever moving forward and away from his various day jobs, which always end up feeling a little too much like work to him.
He addresses his unquenchable wanderlust in the song "Feels Like Work," which also succinctly summarizes what he's all about.
"We can run off somewhere every now and then/when it feels like work again."