Skip to content

Model Storage

To store and version information about trained models, such as architecture and parameters, there are different options. See the storage overview documentation for more information about access methods and the general advantages and disadvantages. The destination should be chosen based on the level of reliability and the need for deployment using the KubeFlow inference service.

EOS

EOS can be used to store models for reliable long term storage. Though, models can not be deployed directly from EOS using the KubeFlow inference service.

Object Storage

Object storage can be used for serving models via the KubeFlow inference service. CERN provides a managed object storage under s3.cern.ch but external object storage providers or the cluster internal object storage can be used as well. Models are then organized as objects that are part of a bucket. For example, there could be a bucket models in which model files are uploaded via a unique identifier, e.g. model-v1.pkl and model-v2.pkl.

Shared Filesystem Storage

Shared filesystem storage can be provisioned via the KubeFlow dashboard. As it is a Kubernetes resource, it can be used to share the model between Kubernetes workloads by referencing the volume in the respective manifest. For example, the KubeFlow inference service can fetch models using the volume claim identifier and the path of the model within the volume.

CERN Registry

Experimental, fully OCI compliant registry, is available at registry.cern.ch.
The official Harbor documentation can be found here.
To interact with CERN registry to store ML models, the ormb tool is recommended.
ormb can be used to create, version, share and publish models.

Examples

Follow the notebook below to try out the different backends for model storage.

  • Go to https://ml.cern.ch/_/jupyter/, create and start a new notebook
  • Clone git repository with examples 
    git clone https://gitlab.cern.ch/mlops/platform/kubeflow/kubeflow-examples.git
  • Open notebook examples/storage/model-storage.ipynb
    • This notebook containes examples on how to store models via EOS, minio, s3.cern.ch and CERN registry.
  • Follow the instructions and run all the cells