A Review of A House Built by Slaves
By Nancy Spannaus
Sept. 25, 2023—This 2022 book by Jonathan W. White is an invaluable contribution to righting the misperceptions of President Abraham Lincoln’s attitude toward slavery and African Americans. By using largely first-hand recollections to reconstruct the many visits of African Americans to the White House to visit the President, White effectively destroys the idea that Lincoln only emancipated the slaves “from necessity, not conviction.”
Up until now, I have focused my attention primarily on the three visits which leading abolitionist Frederick Douglass made to see the President, all of which underscored Lincoln deep respect and lack of prejudice for the former slave. Those who condemn Lincoln for racism concentrate on the infamous meeting Lincoln held with five leading members of the African American community in August 1862, where the President pressed them to agree to a colonization scheme.

White expands the canvass to let us judge Lincoln in the context of the entire range of his meetings. From that perspective, the August 1862 meeting takes on the character of “the exception that proves the rule,” to quote White. Countless other encounters show the President to be supportive, compassionate, and just plain down-to-earth. Above all, they show how committed he was to using every means he felt to be Constitutional, to achieve his wish “mankind everywhere to be free.”[1]
I urge you to read this book.
In the remainder of this post, I will recount six of the meetings African Americans had with Lincoln in the White House, three of which were with Douglass. In this review, I believe Lincoln’s sincere dedication to human dignity and freedom for all Americans, regardless of color, comes through.
The First Visit
The first African American to visit the Lincoln White House was Bishop Daniel Alexander Payne of the African Methodist Episcopal Church. Payne was born free but had been through a considerable ordeal to maintain his freedom. He was located in Washington, D.C.

What impelled Bishop Payne to seek out the President was Congress’ passage of the D.C. Emancipation bill in early April 1862. The bill had gone to Lincoln for his signature, but the President had not yet acted. So, on April 14, Bishop Payne decided to pay the President a visit.[2] According to first-hand reports, the President came to the door of the room where he was meeting with Senator Elihu Washburn of Illinois, and greeted the bishop, taking him by the hand, and seating him between Washburn and himself. When Payne declared that his mission was to find out whether Lincoln would sign the emancipation bill, Lincoln said that a delegation of men was there to try to convince him not to sign it.
Bishop Payne stayed for about 45 minutes of conversation, after which he left. President Lincoln had still not declared what he intended to do, but on April 16, he signed the bill for D.C. emancipation.
Meeting with “Runaway” Robert Smalls
The next significant meeting I want to mention was Lincoln encounter with the “runaway slave” Robert Smalls. Smalls led several other slaves to join him in a mutiny, and take over the Confederate ship Planter in May of 1862. The men then piloted the ship into Union waters and received a hero’s welcome.

In August, Smalls visited Washington, D.C. Many years later, when serving as a Congressman, Smalls reported that “I was honored with several meetings with President Lincoln and Stanton,” and got a letter from them “authorizing the formation and mustering in of several regiments of colored soldiers.” Thus, it appears that the courage of Smalls and his band in their fight for freedom helped encourage Lincoln to decide to enlist blacks in the military.[3]
The First Meeting with Douglass
President Lincoln’s first meeting with Frederick Douglass came in August of 1863. It occurred upon Douglass’s initiative, at a point where Douglass, the prominent abolition leader and chief recruiter of black troops for the Union, was getting increasingly angry about discriminatory treatment of blacks in the army, as well as the atrocities being carried out by the Confederacy against black Union soldiers.

According to Douglass, he was escorted in to meet the President two minutes after he presented his card to the staff. Lincoln stood up and shook his hand, and then asked Douglass to say what was on his mind. The President then addressed his complaints by indicating how he had to tread a careful path in order to still the opposition to enlisting blacks at all; that blacks would ultimately get equal pay; and that, while he was reluctant to take innocent lives in retaliation for the massacres, he believed action had been taken so that such horrors would not be taken. (The President had already authorized retaliation, but none was ever carried out.)
Douglass came out of that meeting with a totally different view of Lincoln. He agreed to continue recruiting, and he began to make it a point to inform his audiences of how the President had treated him. He “received me … just as you have seen one gentleman receive another,” Douglass reported in a speech on December 4th of that year, and again numerous times. He said he was impressed with the President’s “tender heart” and “humane spirit.”
Douglass’ Second Meeting with Lincoln
Proceeding chronologically, White documents an acceleration in the number of visits by African Americans to the White House in 1864. These included the New Year’s Day reception, a North Carolina delegation seeking the right to vote, black picnics on the White House lawn in the summer, and a delegation of Baltimore ministers delivering a specially inscribed Bible to the President. (The Bible is currently at Fisk University.)
These visits all occurred during the Presidential election campaign, and a deepening war-weariness throughout the nation. The war was not going well for the Union, and President Lincoln was convinced that he was likely to lose the Presidential contest. He was strongly considering a new peace offer to Confederate President Jefferson Davis, one phrased in such a way that it did not appear conditioned on the abandonment of slavery, as his others had been.
Undecided what to do, the President sent a message asking Douglass to visit the White House. Douglass was again, at this juncture, very critical of the President, believing him not strong enough in his anti-slavery policies.

The content of this meeting was extraordinary. First, Lincoln asked Douglass’s advice on sending the peace offer to Davis; Douglass said no (and Lincoln didn’t). Then the President told Douglass about how he was convinced he was going to lose the election and was very concerned about what that would mean for the enslaved people. Not enough slaves from the South were fleeing to freedom, Lincoln told Douglass. There are great efforts by the Southern plantation owners to keep news of the Emancipation Proclamation from them, Douglass replied.
Then we have to get the word out more broadly, the President said. He wanted a plan devised by which as many slaves as possible escaped to freedom before March of 1865, when he expected that his presidency would end. “I hate slavery as much as you do and I want to see it abolished altogether,” he told Douglass. We have to get people to flee more rapidly.
At the conclusion of the meeting, Douglass agreed to organize a band of scouts “whose business should be somewhat after the original plan of John Brown, to go into the rebel States, beyond the lines of our armies, and carry the news of emancipation, and urge the slaves to come within our boundaries.”[4]
Douglass was very impressed with the President’s attitude, “a deeper moral conviction against slavery than I had ever seen before in anything spoken or written by him.” He was also stunned by the fact that the President talked with him for a full hour after having been alerted that Governor William Buckingham of Connecticut was outside waiting to see the President.
Meeting with Sojourner Truth
On October 29, 1864, President Lincoln had a consequential meeting with another African American leader, Sojourner Truth. Sojourner had travelled East from Michigan specifically to meet with the President. According to her account, the meeting was brief, with an exchange on the emancipation, and Sojourner presenting Lincoln with a photograph of herself. At the end, Lincoln wrote a note in her “little book,” addressing her familiarly as “aunty,” and asked her to come visit again. “I felt that I was in the presence of a friend,” Truth said of her short encounter with the President.
There are conflicting reports about whether the two did meet again, and also a lot of controversy about what actually happened at the meeting. White describes the conflicting reports in detail. He concludes by relying on Truth’s own report of her meeting the very next month. “I am proud to say that I never treated with more kindness and cordiality than I was by that great and good man, Abraham Lincoln.”
Douglass’ Last Meeting
As is well known, Frederick Douglass met Lincoln one last time. The encounter occurred at the White House reception after Lincoln’s second inauguration. After being blocked from entering by a couple of policemen, Douglass succeeded in getting a message to the President that he was being detained by officers at the door. Soon after, Douglass and his companion, a Mrs. Dorsey, were allowed to enter. What happened then is as follows (White’s description):
“Lincoln stood there greeting visitors, head-and-shoulders above most of the crowd. When he saw Douglass, he bellowed, “Here comes my friend Douglass.” Lincoln then took Douglass by the hand and said, “I am glad to see you. I saw you in the crowd to-day, listening to my inaugural address; how did you like it?
Douglass was taken aback by this question and replied: “Mr. Lincoln, I must not detain you with my poor opinion, when there are thousands waiting to shake hands with you.”
“No, no,” rejoined Lincoln, “you must stop a little, Douglass; there is not man in the country whose opinion I value more than yours. I want to know what you think of it?”
At this, Douglass said, “Mr. Lincoln, that was a sacred effort.”
“I am glad you like it!” said Lincoln. Douglass then walked on…
A “Reasoned Evaluation”
White concludes his book with an account of the erection and dedication of the Emancipation Statue in Washington, D.C – one of the many statues of Lincoln which has become an object of political controversy today. The statue was financed entirely by former slaves, led by a woman named Charlotte Scott. The dedication was given by Frederick Douglass on April 14, 1876.

Douglass’ optimism of earlier years had faded, and he could not keep from expressing words of criticism for President Lincoln. But toward his conclusion, he sought to put himself in Lincoln’s shoes, recognizing the President’s dual responsibilities to save the Union as well as abolish the evil of slavery. White reprints this crucial paragraph, which he calls a “reasoned evaluation:”
…I have said that President Lincoln was a white man, and shared the prejudices common to his countrymen towards the colored race. Looking back to his times and to the condition of his country, we are compelled to admit that this unfriendly feeling on his part may be safely set down as one element of his wonderful success in organizing the loyal American people for the tremendous conflict before them, and bringing them safely through that conflict.
His great mission was to accomplish two things: first, to save his country from dismemberment and ruin; and, second, to free his country from the great crime of slavery. To do one or the other, or both, he must have the earnest sympathy and the powerful cooperation of his loyal fellow-countrymen. Without this primary and essential condition to success his efforts must have been vain and utterly fruitless. Had he put the abolition of slavery before the salvation of the Union, he would have inevitably driven from him a powerful class of the American people and rendered resistance to rebellion impossible. Viewed from the genuine abolition ground, Mr. Lincoln seemed tardy, cold, dull, and indifferent; but measuring him by the sentiment of his country, a sentiment he was bound as a statesman to consult, he was swift, zealous, radical, and determined.[5]
Douglass’ main concern in the monument dedication was not so much to praise Lincoln, which he had done many times before, but the fate of the people of his race, who were facing a rising tide of violent reaction. Yet, as White stresses, Douglass was clear that whatever Lincoln’s faults, his hatred of slavery was deep and genuine. His is an assessment today’s critics would do well to take seriously, for the sake of truth and our nation.
[1] White, Johnathan W. A House Built by Slaves, African American Visitors to the Lincoln White House, Rowman & Littlefield, Inc., Lanham, Maryland, 2022, p. 138.
[2] For more information on the visit, see White, ibid, p. 29.
[3] For more detail, see idem, pp. 57-58.
[4] For a fuller description, see White, ibid, p. 135.
[5] See the whole speech and my previous coverage of the event here.
Tags: Abraham Lincoln, emancipation, Frederick Douglass, Jonathan W. White, Nancy Spannaus, slavery, Sojourner Truth, White House