And the victory was the perfect way for temporary manager Adrian Heath to throw his hat into the ring to take over from Peter Reid as the permanent boss at Highfield Road.
But Coventry had to survive a testing opening 20 minutes as the visitors - without the Norwich-bound Dean Ashton - were on top early on.
Lee Bell blasted a shot wide after 13 minutes before Steve Jones saw his screaming effort from the edge of the box blocked by Adie Williams.
Coventry got themselves into trouble in the 17th minute when Clayton Ince's long kick was allowed to bounce around in the box, before keeper Luke Steele failed to claim but Williams was there again to clear the danger.
Mark Rivers then saw a low shot turned away by Steele and Crewe were left ruing their missed chances as Coventry took the lead on 24 minutes.
Neil Wood latched on to a fine throughball from Steve Staunton and pulled it back for Stern John. The striker had the awareness to pick out Andy Morrell whose shot was cleared off the line by Stephen Foster, but McSheffrey was on hand to crack home from six yards.
The Sky Blues had plenty of chances to double their lead before the break as Wood and Staunton both came close.
But they did eventually make it two as Stephen Hughes picked out Morrell who took the ball in his stride before crossing for John to turn the ball home from close range.
Crewe offered very little in the second period and were lacking firepower without Ashton and it seemed only a matter of time before City scored a third.
McSheffrey grabbed it in the 69th minute when Hughes and Morrell combined before the former crossed with the outside of his left foot for the striker to stab home first time.