Anderson's legal career was ended when he was fined £50 for accosting two 14-year-old girls, and asking them to walk on him and beat him up. Anderson had been in Troon on the 18th December 1972, presiding over the first major public inquiry of his new post. The prosecution claimed that Anderson, finding himself out of his home town, had approached the girls and asked them to go to a quiet place with him.
The case had been controversial as the girls failed to identify Anderson and he was given an alibi by one of the staff members of the hotel where he was staying. Anderson's wife also gave evidence that the coat he was supposedly wearing on the night was being cleaned at the time. Anderson, who claimed the KGB had framed him in an act of revenge by using a lookalike to impersonate him and get in trouble, appealed the conviction but lost. He was dismissed from his posts in 1974.
Several high-profile, unsuccessful attempts were made to clear Anderson's name, including debates in the Lords and Commons and an investigation by the Scottish Criminal Cases Review Commission. He continued to press his innocence and in 1980 the playwright John Hale wrote The Case of David Anderson QC which was sympathetic to his position. The play was put on in Manchester, Edinburgh and at the Lyric Theatre in Hammersmith. Anderson had not succeeded in clearing his name by the time of his death. In September 2002 it was announced that the Scottish Criminal Cases Review Commission was looking into the case, but it concluded in February 2005 that the conviction should stand.
In her 2010 autobiography Lady Judy Steel claimed that a man had made an almost identical indecent proposal to her when she was a teenage student at Edinburgh University. In the light of subsequent events, she concluded that it was Anderson. Her husband is Liberal Democrat life peer and former MP David Steel.