TensorRT Installation Guide

Overview

TensorRT Installation Guide

Ubuntu 22.04
Nvidia Driver 538.78
CUDA 12.2
cuDNN 8.9.7
TensorRT 10.7

Remove existing NVIDIA drivers, CUDA, cuDNN, and TensorRT to avoid conflicts.

 1sudo apt remove --purge '^nvidia-.*' '^cuda.*' '^libcuda.*' '^libnvidia.*' '^tensorrt.*' '^cudnn.*' 'libnvinfer*' 'nv-tensorrt*' 'python3-libnvinfer*' 'libcudnn*' 'libnvidia*'
 2sudo apt autoremove -y 
 3sudo apt clean
 4# Remove CUDA Toolkit and Related Files
 5sudo rm -rf /usr/local/cuda*
 6sudo rm -rf /usr/lib/cuda*
 7# Remove NVIDIA Repository (If Installed)
 8sudo rm -f /etc/apt/sources.list.d/cuda*
 9sudo rm -f /etc/apt/sources.list.d/nvidia-*
10# update your package lis
11sudo apt update
...
bash

Verify uninstallation:

1dpkg -l | grep -E 'cuda|nvidia|tensorrt|cudnn'
bash

After removing the drivers and CUDA-related packages, reboot the system:

1sudo reboot
bash

Verify uninstallation:

1# Check Nvidia driver
2nvidia-smi
3# check CUDA
4which nvcc
bash

Alt text
Alt text

https://www.nvidia.com/en-us/drivers/

1# install the latest supported driver
2sudo ubuntu-drivers autoinstall
3# Or install the specific version
4sudo apt install nvidia-driver-xxx
5# Verification
6nvidia-smi
bash

Select the model from the website and get instruction: https://developer.nvidia.com/cuda-12-2-0-download-archive

Or use below for Ubuntu 22.04 x86_64 architecture.

1wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64/cuda-keyring_1.1-1_all.deb
2sudo dpkg -i cuda-keyring_1.1-1_all.deb
3sudo apt-get update
4sudo apt-get -y install cuda
bash
1# Update cuda path to .bashrc
2echo 'export PATH=/usr/local/cuda-12.8/bin:$PATH' >> ~/.bashrc
3echo 'export LD_LIBRARY_PATH=/usr/local/cuda-12.8/lib64:$LD_LIBRARY_PATH' >> ~/.bashrc
4source ~/.bashrc
5# verification
6nvcc --version
bash

Download the relevant .deb file from the website: https://developer.nvidia.com/rdp/cudnn-archive

1# install repository
2sudo dpkg -i cudnn-local-repo-ubuntu2004-8.9.7.29_1.0-1_amd64.deb
3# copy keyring [switch to your folder name after /var/]
4sudo cp /var/cudnn-local-repo-ubuntu2004-8.9.7.29/cudnn-*-keyring.gpg /usr/share/keyrings/
5# update and install
6sudo apt-get update
7sudo apt-get install libcudnn8
8# verify installation
9dpkg -l | grep cudnn
...
bash

Download .deb from website: https://developer.nvidia.com/tensorrt-getting-started

 1# install repository [check name]
 2sudo dpkg -i nv-tensorrt-local-repo-ubuntu2204-10.7.0-cuda-12.6_1.0-1_amd64.deb
 3# copy keyring [check name]
 4sudo cp /var/nv-tensorrt-local-repo-ubuntu2204-10.7.0-cuda-12.6/nv-tensorrt-local-F234AD55-keyring.gpg /usr/share/keyrings/
 5# update package list
 6sudo apt-get update
 7# install tensorrt
 8sudo apt-get install tensorrt
 9# Add path
10echo 'export PATH=$PATH:/usr/src/tensorrt/bin' >> ~/.bashrc && source ~/.bashrc
11source ~/.bashrc
12# verification
13dpkg -l | grep nvinfer
14sudo apt show tensorrt
15trtexec --version
...
bash