Challenging civil rights organizations and teachers' unions that have criticized his education policies, President Barack Obama said Thursday that minority students have the most to gain from overhauling the nation's schools.

"We have an obligation to lift up every child in every school in this country, especially those who are starting out furthest behind," Obama told the centennial convention of the National Urban League.

The Urban League has been a vocal critic of Obama's education policies, most notably the $4.35 billion "Race to the Top" program that awards grants to states based on their plans for innovative education reforms. A report released earlier this week by eight civil rights groups, including the Urban League, says federal data shows that just 3% of the nation's black students and less than 1% of Latino students are affected by the first round of the administration's "Race to the Top" competition.

Obama pushed back Thursday, arguing that minority students are the ones who have been hurt the most by the status quo.

Obama's reforms have also drawn criticism from education advocates, including prominent teachers' unions like the American Federation of Teachers, who have argued that the reforms set unfair standards for teacher performance.

Obama said the goal isn't to fire or admonish teachers, but to create a culture of accountability. He pinned some of the criticism on a resistance to change.

"We get comfortable with the status quo even when the status quo isn't good," he said. "When you try to shake things up, sometimes people aren't happy."