You can use the NVidia System Management Interface (nvidia-smi) utility to monitor GPU utilization and memory. nvidia-smi ships with NVidia GPU display drivers on Linux, 64-bit Windows Server 2008 R2, and Windows 7. nvidia-smi can report query information such as XML or human-readable plain text to either standard output or a file. 

To output GPU utilization, memory utilization, memory in use, and free memory with timestamp every 5 seconds, run the following command:

nvidia-smi --query-gpu=timestamp,name,utilization.gpu,utilization.memory,memory.used,memory.free --format=csv -l 5

Here's sample output:

timestamp, name, utilization.gpu [%], utilization.memory [%], memory.used [MiB], memory.free [MiB]
2020/11/20 00:18:00.669, GeForce MX150, 0 %, 0 %, 2182 MiB, 1914 MiB
2020/11/20 00:18:05.684, GeForce MX150, 0 %, 0 %, 2182 MiB, 1914 MiB

By default, output is written to stdout, but you can use the -f filename argument to write the output to a file instead.

To learn all the query-gpu parameters and definitions, run the following command:

nvidia-smi --help-query-gpu

For more details about the nvidia-sml utility, please refer to the nvidia-smi documentation.