vSphere Supervisor Services

Private AI Services

Private AI Services (PAIS) supervisor service was introduced at version 2.0 as part of VMware Private AI Foundation with NVIDIA 9.0. More information is available in this blog post and this blog series. (The 1.0 version of PAIS did not provide a Supervisor Service component.) The following versions of PAIS Supervisor Services are available:

Supported VCF/Supervisor versions	PAIS
9.0.*	2.0
9.0.*, 9.1.*	2.1

PAIS CRDs

PAIS is operated via two Kubernetes Custom Resource Definitions (CRDs) available in your Supervisor cluster:

• PAISConfiguration is a singleton per-namespace, where you specify a database, certificates, authentication, and other configuration for a PAIS "instance" within that namespace, including a PAIS API server and web UI.
• ModelEndpoints configure inference servers for a given AI model, using VMs within your namespace, with routing via the namespace-level PAIS API server.

After you've installed the PAIS Supervisor Service, you can use kubectl explain to see the full documentation for each of these resources. For example:

kubectl explain paisconfiguration.spec

or

kubectl explain modelendpoint.spec.engine

Example resources

Here we're publishing example YAML files to help you get started with Private AI Services on VCF 9.

You must first install the Private AI Services (PAIS) Supervisor Service.

Then you can customize and apply:

• An example PAISConfiguration to configure Private AI Services within the tenant namespace
• Example ModelEndpoint resources for running language models in a tenant namespace

PAIS features

Observability: Metrics Configuration

PAIS 2.1 supports collecting metrics from various PAIS components into a Prometheus server with TimeseriesDB storage and provides access to those metrics through Prometheus server query interface (ref: Prometheus HTTP API).

PAISConfiguration CRD supports an optional field spec.observability.prometheusRuntime, specifying following fields enables metrics collection. If prometheusRuntime is unspecified (default behavior), it implies that metrics collection is not enabled.

• metricsRetention: Unit is number of days, min 1d, max 90d, and also default if unspecified.
• storageClass: Used for placing the PVC for Prometheus TSDB storage, default is value of top level spec.defaultStorageClassName), useful to override if default storage class is unsuitable for metrics storage (e.g. space constraints, IO perf constraints etc.).

Please see PAISConfiguration for example values.

This site is open source. Improve this page.