As one of American Idol's most polarizing figures, fans either love or hate Sanjaya Malakar – but he says he's always been popular with women.

"I've always gotten along with girls better because I was raised by women," Malakar tells People magazine in its new issue.

Malakar, whose parents divorced when he was three, says his ease with women comes from being close to his mom and his sister, Shyamali Malakar, who were his biggest influence growing up in Seattle. But getting along with girls didn't always work to his benefit.

"I got teased in school because people figured I must be gay because I understand women," the popular American Idol castoff says. "I think that's why guys didn't like me – because I got along with girls so well. When I went up to girls they would give me a hug and a kiss on the cheek like I was their gay friend. But I was the straight guy that understood them."

As for his dating past, Malakar, who is now 17, says: "I had a girlfriend but she became clingy, and I didn't want to get into a really serious relationship because I was 16. Before his arrival on the hit FOX show, "I just wanted to date and have fun," he says.

Matt Richenthal is the Editor in Chief of TV Fanatic. Follow him on Twitter and on Google+.