Build a budget 3090 deep learning workstation Part Three: undervolt an RTX GPU on Ubuntu

1 minute read

Other parts in the “Build a budget ML workstation”

How to undervolt the GPU on Ubuntu

On Windows there is the famous MSI Afterburner to dial down the voltage of the GPU, so that the overall heat will be significantly reduced, yet the performance takes only minor hit because the clock of the GPU will be higher.

On Ubuntu there is no native Afterburner, but we can config things to achieve something similar. First edit or create /etc/rc.local, and add the following to the file:

sudo nvidia-smi -i 0 -pl 280
exit 0

Note #!/bin/bash part is necessary and -pl 280 means power limit being 280W (the stock power is 350W). Next step is to set a frequency offset to 105Mhz, so that during active computing, the GPU core will be kicking off and running at a frequency of 105Mhz higher than designed. Again, adding the following script as a startup application will do the magic:

nvidia-settings -a '[gpu:0]/GPUGraphicsClockOffset[4]=105' -a '[gpu:0]/GPUGraphicsClockOffset[3]=105' -a '[gpu:0]/GPUGraphicsClockOffset[2]=105'

The result is promising, using the default tensorflow CNN benchmark default setting:

python3 --model resnet50 --batch_size 64

The 280W/105Mhz offset can do 447 images/sec, while the stock 350W setting is at 471 images/sec. The performance hit is about 5%, yet the peak power is down 20%. The GPU can now sustain running at a relatively low temperature (55-62 degrees Celsius when being at full load) for an extended period of time (1 or 2 days non-stop). The sacrifice is totally worth it.