To get around the problem of having to return self-emancipated (i.e. escaped) slaves to their Confederate owners, Union Gen. Benjamin Butler dubbed them “contraband of war,” i.e. captured property that didn’t have to be returned. Later, Congress made the term official.
The novel’s Pathenia Leila Ellis disliked hearing her servant/slave Brutus referred to that way when he was “volunteered” by Union engineer Captain Orlando Poe to work on the town’s fortifications. She thought Brutus should be called by his name.
Much of the Union army treated the contrabands not as freed slaves but as newly-arrived servants to be assigned to do camp work for Union soldiers, such as this pair of former slaves, photographed near Culpeper, Virginia, sometime in 1863.
Via Old Pictures