If a GPU overheats (possibly due to poor airflow of a bad fan), the processor speed will be slowed down to prevent the GPU from damage.  This will often manifest itself as CUDA cores showing much higher load than expected.


Use following command to view GPU Temperature:


nvidia-smi

Most Nvidia GPUs will start to throttle if the temperature reaches 80° C.


HWiNFO is a free tool that will provide more detailed information about your hardware.  You can download it from the following URL:


https://www.hwinfo.com/download/


Following shows a GPU that is overheated (running at 91.4° C) and is running at 1/6 its normal clock speed (300MHz).