Young Phillip Ainsworth, (Bobby Breen), an orphan of the US Civil War, has been lovingly raised by Toinette, (Louise Beavers) a former slave. Toinette has big plans for the boy. She has saved her money to send him to a private school. But when the local priest, Father Josef, (Henry O'Neill) finds Phillip's family living in New York, the boy is sent north to live with people who refuse to accept him as their own. His only friend is the butler, Barrett, (Charles Butterworth). But his curmodgeon of a grandmother, (May Robson) is finally broken down by the boy's charm and good manners, and all ends happily. Along the way Breen has ample opportunity to show off his voice by singing several songs, including the title song three times.

In 1873, Philip, whose Northern-born father was a casualty of the Civil War and whose Southern mother was a victim of fire, is reared and sheltered in New Orleans by former slave Toinette. Touched by Toinette's devotion to Philip, a gifted singer and banjo player, kindhearted Father Josef traces the boy's surviving family to New York and determines that Mrs. Harriet Ainsworth, a wealthy widow, is his grandmother. Josef writes to Mrs. Ainsworth explaining the situation, but because she hates Southerners and still resents her son for marrying one, Mrs. Ainsworth views Josef's claims with great skepticism. Mrs. Ainsworth's niece, the fortune-hunting Julia Layton, also views Josef's letter with loathing, fearing that with a grandson, Mrs. Ainsworth will cut her and her daughter Lucille out of her will. Despite her reservations, however, Mrs. Ainsworth sends Julia's henpecked husband Ralph to New Orleans to investigate Philip's parentage. Although no documentation of Philip's birth survived the war, Ralph takes a liking to the talented boy and agrees to take him back to New York. Urged by Father Josef to unite Philip with his natural family, Toinette reluctantly parts with her charge, comforted only by the thought that he will have better opportunities in the North. Once in New York, Philip is ignored by Mrs. Ainsworth, who detests Southerners and still has her doubts about Philip's parentage, while Julia and the bratty Lucille plot to have him adopted into another family. Comforted only by his dancing pet mice and by Barrett, Mrs. Ainsworth's gentle butler, Philip leads a lonely existence at his grandmother's and yearns for the warmth of Toinette. Gradually, however, Philip's natural charm and cheeriness warm the heart of his stern grandmother, and a genuine friendship is formed. Philip's budding relationship with Mrs. Ainsworth does not curtail the Laytons' cruelty, however. At her birthday party, Lucille becomes jealous of the attention that Philip's singing and banjo playing attracts and orders another boy to release his mice into a crowd of children. After the ensuing ruckus, Lucille informs Philip that they are sending him to an orphanage the next day and have drowned his mice. Alerted by Barrett, Mrs. Ainsworth finally sees through Julia and Ralph's scheming and stops Philip from running away to New Orleans. Mrs. Ainsworth then takes Philip back to Toinette, who has been pining for her lost child and has become physically debilitated with grief. Moved by Toinette and Philip's intense reunion, Mrs. Ainsworth finally embraces Philip as own her flesh and blood.