I will guide you through the process of deploying Red Hat OpenShift AI on the NVIDIA DGX H100 system and run the Mistral AI model. This blog post details the process of deploying and managing a fully automated MLOps solution for a large language model (LLM) presented in three main parts:
TL;DR
The bare metal installation of OCP is only this installer command:
$ openshift-baremetal-install --dir ~/clusterconfigs create cluster but I’ll take time in this post to explain how to prepare your platform and how to follow the installation.
The NVIDIA GPU Operator has been available as a Beta since 2020, Jan 27, it’s a technical preview release: https://github.com/NVIDIA/gpu-operator/release
The GPU Operator manages NVIDIA GPU resources in an OpenShift cluster and automates tasks related to bootstrapping GPU nodes.
OPAE is the Open Programmable Acceleration Engine, a software framework for managing and accessing programmable accelerators (FPGAs): https://01.org/opae
The OPAE SDK is open source and available in this git: https://github.
We will enable GPU with Podman on a RHEL 8.1 system with a NVIDIA Tesla T4: Take a RHEL 8.1 system:
[egallen@lab0 ~]$ cat /etc/redhat-release Red Hat Enterprise Linux release 8.
Red Hat OpenShift Container Platform 4.2 introduces the general availability of full-stack automated deployments on OpenStack. With OpenShift 4.2, containers can be managed across multiple public and private clouds, including OpenStack.
We will describe the steps to try and download NVIDIA GRID software:
Create a NVIDIA account Redeem your Product Activation Key (PAK) Download packages Prepare the VM and operating system of the license server based on RHEL 7.
Red Hat OpenStack Platform 14 is now generally available \o/
NVIDIA GRID capabilities are available as a technology preview to support NVIDIA Virtual GPU (vGPU). Multiple OpenStack instances virtual machines can have simultaneous, direct access to a single physical GPU.
Red Hat OpenStack Platform provides two ways to use NVIDIA Tesla GPU accelerators with virtual instances:
GPU PCI passthrough (only one physical GPU per instance) vGPU GRID (one physical GPU can be shared to multiple instances, Tech Preview OSP14) This blog post is intended to show how to setup GPU PCI passthrough.
Some OpenStack users users would like to attach USB devices to OpenStack instances for security or legacy applications.
For example, a security application which run inside an OpenStack instance could require access to a Java card from an USB Gemalto eToken:
