Jon Sheffield was a Youth Team product who came to Norwich's initial attention via Kit Carson's PGL scheme in 1985. Jon opted for Norwich despite interest from Wolves, Ipswich, Cambridge, Watford and Arsenal. However he was unable to displace Bryan Gunn in the Norwich goal. His only appearance was a 5-0 thrashing at home to Aston Villa in April 1989. Loan spells at Aldershot (twice), Ipswich and Cambridge were followed by a permanent move to Cambridge United. Since then, he has spent time at Colchester, Swindon and Hereford on loan before joining Peterborough United in July 1995 for £150,000. In two seasons at Peterborough, Jon made 62 league appearances but lost his first team place during the 1996/97 season to Bart Griemink.

He was one of three former Canaries to be released by Posh manager Barry Fry in the summer of 1997. Jon joined Plymouth for £100,000 whilst Louie Donowa moved to Walsall and Tony Spearing joined King's Lynn. An ever-present in the Pilgrim's goal, Jon could not prevent them from relegation to Division Three. Jon was released by Plymouth on 18th April 2001 but joined local non-league rivals Yeovil on 11th July 2001

Yeovil won the Conference Championship on April 12th 2003 when rivals Chester could only draw with Woking. Jon had made just 9 appearances prior to the championship winning weekend. The club felt that he was not of the appropriate standard for Division Three and released him at the beginning of May 2003 along with Jimmy Aggrey, Neil Mustoe and Kim Grant. The 13th July 2003 edition of Sport First reported that Jon had joined Saltash United of the Carlsberg South Western League.