Dunn was convicted of shoplifting a sweatshirt, two ties and two armbands valued at £15.13 from the Army & Navy Stores in Victoria and a map valued at 60p from a stationers' shop in Artillery Row on 27 July 1979. At the time he had taking anti-depressant drugs. He had been followed out of the Army & Navy Stores by two store detectives who then saw him stealing the map. When they caught him he requested a quiet chat, but was turned over to the police.

His trial took place on 12 May 1980, and Dunn pleaded not guilty. Although he admitted taking the goods he used as his defence his state of mind at the time of the incident. Dunn claimed that he had panicked when he saw the store detectives as he thought they were people he had dealt with in Northern Ireland. He was in poor health at the time, having had a heart attack in 1978, and had found his time as a minister, which involved regular trips to and from Northern Ireland, had left him exhausted.

Additionally the evening before the offence he had attended a late-night debate in Parliament and consumed five half-pints of beer in the House of Commons bar. After this he had taken a sleeping tablet, but had only had "a fitful night's sleep". He said he was unable to remember exactly his actions the following day. Although he was found guilty, the judge accepted that his "appalling state of health" had caused him to act out of character and gave him a conditional discharge though he had to pay £100 costs.

In 1981, Dunn was among the Labour MPs who defected to the new Social Democratic Party. In 1983, he left the House of Commons when his seat was abolished by boundary changes. He died at the age of 59 in 1985.