I have found that the panel switch on my fridge actually did not control the main power to the fridge unit… I never found what it did switch, but my assumption is that the panel switch was actually inline with the thermostat (caused a right headache when I came to install a digital thermostat!!)
Anyway, it sounds like the controller (you can try the LED diagnostics, where you jump an LED across the Fans + and D on the controller).
When you say there is a whirring sound, is the fan running or is it the compressor? Its likely to just be the fan.
I have gone through 2 controllers in the last 5 years and my guess is its likely to be that… They are like hens teeth though, and crazy expensive at the moment, though…. If you have trouble, get in touch as I have a spare.
Jason