References » Using GPU for Yadle AI

There are 3 different ways for doing AI analysis with Yadle:
  1. Using on-premise server CPU
  2. Using on-premise server GPU
  3. Using GPUs on Yadle’s hosted AI server.

This guide will provide instruction for using #2, on-premise GPUs for AI analysis.

Pre-requisites

Determine if current Yadle setup is using CPU, GPU, or Yadle’s AI Server for analysis.

Step 1: Run the following command on the Yadle server node. 
$ sudo docker ps -a --filter "name=aitags" --format {{.Names}}
Scenario A:

If nothing is returned, the Yadle server microservices are not running. Start them and then rerun command above.

Scenario B:

If the following is returned, the Yadle setup is using local CPU for AI analysis:

<org>_server_changes_mode_aitags-colossus_1
<org>_server_changes_mode_aitags-krell_1
Scenario C:

If the following is returned instead, the Yadle setup is already using local GPU OR Yadle’s hosted AI server:

<org>_server_changes_mode_aitags-tfserving-colossus_1
<org>_server_changes_mode_aitags-tfserving-krell_1

To determine if current setup is using local GPU or Yadle’s hosted AI server, inspect the contents of the tf_serving_config.json file with the following:

$ cat /opt/yadle/servers/config/tf_serving_config.json
Scenario A:

If the following is returned, Yadle’s hosted AI server is being used for analysis.

{
    "tf_serving_host": "ms1.yadle.com",
    "tf_serving_port": "8500"
}
Scenario B:

If the following is returned instead, the local GPU is already being used for analysis.

{
    "tf_serving_host": "0.0.0.0",
    "tf_serving_port": "8500"
}

Install NVIDIA Container Toolkit

The NVIDIA Container Toolkit allows users to build and run GPU accelerated Docker containers.

Step 1: Setup the NVIDIA Container Toolkit repo.

CentOS:

$ distribution=$(. /etc/os-release;echo $ID$VERSION_ID) \
   && curl -s -L https://nvidia.github.io/nvidia-docker/$distribution/nvidia-docker.repo | sudo tee /etc/yum.repos.d/nvidia-docker.repo
Ubuntu: 
$ distribution=$(. /etc/os-release;echo $ID$VERSION_ID) \
   && curl -s -L https://nvidia.github.io/nvidia-docker/gpgkey | sudo apt-key add - \
   && curl -s -L https://nvidia.github.io/nvidia-docker/$distribution/nvidia-docker.list | sudo tee /etc/apt/sources.list.d/nvidia-docker.list

Step 2: Install nvidia-docker2 package (and dependencies) after updating the package listing.

CentOS:

$ sudo yum clean expire-cache
$ sudo yum install nvidia-docker2

Ubuntu:

$ sudo apt-get update
$ sudo apt-get install nvidia-docker2

Step 3: Stop all Yadle components.


Step 4: Restart Docker Engine.

$ sudo systemctl restart docker

Step 5: Start all Yadle components.

Install Yadle AI Server

The Yadle AI Server will use the local CUDA enabled GPU for analysis.

Step 1: Create a directory to house Yadle AI server components. If installing on same system as the Yadle server, it is recommend to place in same location.

$ sudo mkdir -p /opt/yadle/ai_server

Step 2: Download Yadle AI Server installer and allow executable permissions.

$ cd /opt/yadle/ai_server
$ sudo curl -O https://download.yadle.com/ai_server/install_ai_server
$ sudo chmod +x install_ai_server

Step 3: Run Yadle AI Server Installer. Script must run as root or sudoer. The script will download the AI server docker image & generate start/stop scripts.

$ sudo ./install_ai_server

Verify Yadle AI Server is running.

Step 1: Verify setup by running nvidia-smi.

$ nvidia-smi

Which should return:

+-----------------------------------------------------------------------------+
| NVIDIA-SMI 510.47.03    Driver Version: 510.47.03    CUDA Version: 11.6     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  Tesla K80           On   | 00000000:00:04.0 Off |                    0 |
| N/A   56C    P0    58W / 149W |  10877MiB / 11441MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
                                                                               
+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|    0   N/A  N/A     13769      C   tensorflow_model_server         10872MiB |
+-----------------------------------------------------------------------------+

Update Server Microservices to Use Containers Specifically for GPU analysis.

Step 1: Go to the location where Yadle server microservices are installed. Typically this will be /opt/yadle/microservices/server.

$ cd /opt/yadle/microservices/server

Step 2: Run the following command.

$ sudo sed -i '$s/$/\\/' update_yadle_microservices \
   && echo '--tfserving' | sudo tee -a update_yadle_microservices > /dev/null

Step 3: Run Yadle server microservices update script.

$ sudo ./update_yadle_microservices