Enabling Multi-user Machine Learning Workflows for Kubeflow Pipelines

August 18, 2020
KubeCon Amsterdam 2020 - Enabling Multi-User Workflows for Kubeflow Pipelines

August 18, 2020 @ KubeCon + CloudNativeCon Europe Amsterdam 2020

Kubeflow is an open source machine learning platform built on Kubernetes. Every service in Kubeflow is implemented either as a Custom Resource Definition (CRD) (e.g., TensorFlow Job) or as a standalone service (e.g., Kubeflow Pipelines).

As enterprises start to adopt Kubeflow, the need for access control, authentication, and authorization is emerging. Kubernetes CRDs come with their own auth story, but what about Services with their own API and database, like Kubeflow Pipelines? In this talk, we explore how we enabled multi-user workflows for Kubeflow Pipelines, in a Kubernetes-native way.

We present how we combined open-source, cloud-native technologies to design and implement a flexible, Kubernetes-native solution for services with their own API and database.

The talk will include a live demo.

Speakers

Yannis Zarkadas

Yannis Zarkadas

Software Engineer

Yannis is a software engineer at Arrikto, working with Kubeflow and the Kubernetes sig-storage group. He loves contributing to open source projects and has authored the Cassandra Operator in Rook and the official Scylla Operator, which he is currently maintaining.

Yuan Gong

Yuan Gong

Google Cloud - Software Engineer

I’m a software engineer at Google Cloud working on Kubeflow Pipelines project.

Want the presentations?