Mr. Carter made his comments nearly a week after U.S. Rep. Joe Wilson (R-S.C.) shouted "You lie!" at Mr. Obama during a joint session of Congress.
"I think it's based on racism," Carter said Sept. 15, responding during a Q-and-A session at his presidential center in Atlanta. "There is an inherent feeling among many in this country that an African-American should not be president."
Mr. Clinton gave his perspective as a fellow white Southerner.
"I would say it in a different way," Mr. Clinton told Russ Mitchell, news anchor for CBS' "The Early Show."
Mr. Obama is scheduled to speak Tuesday at the annual meeting of Mr. Clinton's non-profit group, the Clinton Global Initiative.
"I'm very sympathetic to what motivated President Carter," Mr. Clinton said. "If you're a white southerner and you spent a lot your life fighting against racial discrimination and seeing the ugliness that it causes, you're hypersensitive to it, and you don't like it."
Mr. Obama responded to Mr. Carter's comments in his own way Monday night, telling David Letterman "I was black before the election" on CBS' "The Late Show."
The president's comments stuck with the White House's strategy to avoid turning political debates into a race debate, a strategy with which Mr. Clinton agreed.
"I get why he said it," Mr. Clinton said about Mr. Carter. "But I think President Obama's response was more appropriate for the president. That is, I believe it is true that some of the president's more extreme critics are also racially prejudiced. But I believe if he were white, those same people would still be against him on health care."