The Lavender Hill Mob centers around Henry Holland (Guinness), a mild-mannered London bank clerk who has spent the last 20 years overseeing the delivery of gold bullion. Everyone thinks he is dedicated to his work, but he is secretly planning to steal enough bullion to retire. His plan is perfect but for one little detail - he can't think of a foolproof way of getting the gold out of England.

The answer comes when Alfred Pendlebury (Stanley Holloway), moves into his boardinghouse. Pendlebury owns a company that makes souvenirs. Holland realizes that if they worked together, they can melt the gold down into little statues of the Eiffel Tower and ship them to France, disguised as cheap trinkets sold in Paris.

Before the plan can be set into action, Holland learns that the bank is going to transfer him away from the gold. He and Pendlebury have to work fast, so they hire two crooks, Lackery Wood (Sid James) and Shorty Fisher (Alfie Bass) to help. All goes as planned and the towers are shipped to Paris, but once there, disaster strikes when the gold souvenirs are accidentally sold to a group of British schoolgirls. Now, they have to get back to England, find the girls and get the gold back before the police catch them.