The postcard Oxford college, film template for Hogwarts and home to the Oxford cathedral
The generic British word for dessert is "pudding."
In the 19th century, the "g" was sometimes pronounced as a harder "k." Sometimes, the "n" got dropped. Sometimes that was shortened by slicing off the "pud."
In other words, small, incremental changes resulted in pudding->puddink->puddik->dick.
It's not meant to be dirty; it's just a Victorian synonym for "dessert."
Pepper a cake with currants or raisins, and you get "spots" in your pudding, hence: spotted dick.