One of the hardest things for me about writing on controversial topics is the possibility of alienating my own friends and colleagues.

Last week I wrote a piece, Why White People Don't See Racism, that raised a few objections from a couple of my white friends. And last year I wrote a piece about my friend and colleague Roland Martin and his Super Bowl tweets that got me blocked from his Twitter feed, even though I still respect and admire him.

With that history in mind, however, I certainly don't want my words today to alienate my friend Don Lemon, so I should start by saying that I really, really like Don Lemon.

Don was the very first person I followed when I signed up on Twitter. I've known him for years and I was a fan even before I did. In some ways he's been a hero, and I praised him when he became the first national Black TV news anchor to come out of the closet in 2011.

That's why I was so troubled when I turned on the TV Saturday and saw Lemon complaining about young Black men wearing sagging jeans. Echoing recent words from Fox News host Bill O’Reilly, Lemon said the controversial right-wing talk show host "doesn’t go far enough" in his critique of Black culture.

Lemon explained that he recently moved to Harlem and now sees lots of troubling inner-city social conditions. I understand that. I moved to Harlem 12 years ago myself and I saw behavior that troubled me too.

Over the years, I've seen the young kids with the sagging pants and the teens dropping their litter on the sidewalks. I wrote about these things for years on my old blog, and I even confronted some of the people who littered or didn't clean up after their dogs. Perhaps the most annoying issue for me was the way some people viewed my car. I used to own a classic Jaguar and it bothered me to find kids sitting on the hood of the car like a park bench when I walked outside the door.

I learned something valuable from those experiences. Although I had moved to the community to become part of the exciting "new Harlem" I had heard about, in reality I was still an outsider to a lot of the people who'd grown up and spent their lives there.

Over the past few months, I've seen Don Lemon in Harlem quite a few times. He's always friendly and always says hello, even when I don't recognize him in his street clothes. And although I understand his frustration with some things in our community, I fundamentally disagree with the views he expressed this weekend.

You see, as a relatively privileged well-educated Black man of a certain age, I have to agree with Lemon that sagging pants and littering teens bothers me. Nor do I like to hear the N-word every time I walk to the corner store. That was not my life experience as a product of the suburbs. But I also know these community issues are not the main problem facing young Black kids today.

Sagging pants and littering neighbors aren't stopping young Black men from getting jobs. It's racial, social and class inequality that's stopping them. It's the lack of educational and economic opportunities available to them. It's the disproportionate incarceration of young black men and the 700,000 stop-and-frisks on New York City streets. Unfortunately, what Lemon's analysis does is confuse cause and effect. That's because it's a lot easier to focus on the effects – the street issues – than to deal with the cause – entrenched systemic and institutional barriers that restrict opportunities for African-Americans.

In a country where white unemployment has never reached 10 percent since the Great Depression but Black unemployment has only rarely dipped below 10 percent since records have been kept, our problems are not sartorial but structural. In fact, if white unemployment remained at the level it has for Blacks over the past 40 years, we'd launch a new "New Deal" program to get people back to work. We'd invest in job training and education and encourage home ownership, just as the nation did with the GI bill after World War II. But that's not happening.

The truth is that jobs won't miraculously come to Black kids if they pull up their pants. Justin Bieber gets to prance around with sagging pants as often as he wants. Mark Zuckerberg can wear a hoodie without ever being accused of being "suspicious." White kids on college campuses can listen to the hardest rap music without being called "thugs." White kids get to be kids. They get to go through "phases," to listen to bad music, to wear stupid clothing and to make mistakes. But then they get to grow up and become successful adults. Black kids, on the other hand, don't often get the benefit of the doubt, the second chances and the opportunities that come along with it.

To be clear, I'm not making excuses for Black kids or assuming they're all the same. I'm just not blaming them for forces beyond their control. Like most African-American parents, I want my kids to be productive members of society. I don't want them to use racism as an excuse for failing to try. But I also don't want them to think that the burden to fix our community is theirs alone. If we really want to practice tough love in America, as Don Lemon argues, then we should start by examining the priorities of the adults in our larger society, not by knocking our kids.

Keith Boykin is a New York Times best-selling author and former White House aide to President Clinton. He attended Harvard Law School with President Barack Obama and currently serves as a TV political commentator. He writes commentary for BET.com each week.

The opinions expressed here do not necessarily reflect those of BET Networks.

BET National News - Keep up to date with breaking news stories from around the nation, including headlines from the hip hop and entertainment world. Click here to subscribe to our newsletter.