Last week we hosted our second FREE course in the “ Introduction to Kubeflow” training and certification series. This course focused on Kubeflow Fundamentals. In this blog post we’ll recap some highlights from the class, plus give a summary of the Q&A. Ok, let’s dig in.
Congratulations to Panagiotis Simakis!
The first person to earn the “Fundamentals” certificate at the conclusion of the course was Panagiotis Simakis, Sr. Data Engineer at Netdata. Well done!
What topics were covered in the course?
This initial course aimed to get data scientists and DevOps engineers with little or no experience familiar with the fundamentals of how Kubeflow works.
- Kubeflow architecture
- Overview of machine learning workflows
- Kubeflow components
- Tools and add-ons (Kale, Rok, Istio, etc)
- Distributions
- Installing Kubeflow on AWS
- Community overview
What did I miss?
Here’s a short teaser from the 90 minute training. In this video we demonstrate there things in regards to Katib (which is the component that provides AutoML, hyperparameter tuning and other features for Kubeflow):
- How to view an experiment
- How to set up experiments
- How to identify the best run
Missed the Dec 16 Kubeflow Fundamentals training?
If you were unable to join us last week, but would still like to attend a future training, the next “Kubeflow Fundamentals” training is happening on Jan 27, 2022. You can register directly on Zoom here.
Additional FREE upcoming trainings
Registration for the next course in the series, “Jupyter Notebook Fundamentals” is live. You can register directly on Zoom here.
We’ll be covering the following topics in this course:
- Kubeflow Fundamentals Review
- Notebook Basics
- Getting Started with Kubeflow Notebooks
- Working with Jupyter Lab Notebooks
- Jupyter Example: Titanic Disaster
- Jupyter Example: Dog Breed Identification
- Course Review
- Next Steps
Arrikto Academy
If you are ready to put what you’ve learned in the course into practice, check out Arrikto Academy! On this site you’ll find a variety of FREE skills-building tutorials:
- Kale 101: Transform Jupyter Notebooks into Kubeflow Pipelines
- Katib 101: Automated Hyperparameter Tuning for Models in Kubeflow Pipelines
- Rok 101: Manage and Restore Kubeflow Pipeline Snapshots
Q&A from the training
Below is a summary of some of the questions that popped into the Q&A box during the course. [Edited for readability and brevity.]
What are the prerequisites for this course?
- Familiarity with cloud computing environments like AWS, GCP or Azure
- Basic understanding of cloud-native architectures and Kubernetes concepts like pods, controllers, nodes, container images, volumes, etc.
- Familiarity with ML concepts like algorithms, model training and parameter tuning
Where can I find a list of companies using and/or contributing Kubeflow?
You can find a list of some of the companies using Kubeflow on this “Success Stories” page and you can find the companies contributing to the Kubeflow project here. These companies include Google, Arrikto, IBM, Bloomberg, Canonical, Cisco and more.
What is the difference between Kubeflow and MLflow?
Kubeflow is an end-to-end MLOps platform with container orchestration built-in. Meanwhile, MLflow is a Python program for tracking experiments and versioning models as well as a model’s parameters and metrics. MLflow is an artifact tracking solution at its core.
What is the difference between Google’s Vertex.ai and Kubeflow?
Vertex is Google’s managed service that actually uses the Kubeflow Pipelines component of Kubeflow. The rest of the services on Vertex AI are closed source. Kubeflow is open source and provides the same experience on every cloud or on-prem environment.
How do you create pipelines in Kubelfow?
Kubeflow Pipelines is a platform for building and deploying portable, scalable ML workflows. It consists of a UI for managing training experiments, jobs, and runs. It also includes an engine (based on Argo Workflows) for scheduling multi-step ML workflows.
In Kubeflow, how can pipelines be designed in a way so that they can horizontally scale?
In general you’ll want to apply parallelism to every step possible by leveraging Pipelines’ ability to make use of public cloud infrastructure, GPUs and also run dozens of TensorFlow or similar instances of an ML framework in parallel. Kubeflow Pipelines allows you to create scalable Directed Acyclic Graphs (DAGs), which may have multiple steps that run in parallel.
Are Jupyter Notebooks similar to Ansible/Chef?
The short answer is, “No.” Ansible and Chef are configuration management tools that help sysadmins and DevOps folks manage servers. JupyterLab Notebooks are open source, web-based environments for creating Jupyter notebook documents. These are useful in interactive data science and scientific computing across a variety of programming languages. These JSON documents follow a versioned schema, contain an ordered list of input/output cells, can contain code, text (Markdown), math (LaTex), plots and rich media.
What are the advantages of Jupyter Notebooks?
In Kubeflow, Jupyter Notebooks integrate with enterprise authentication and access control mechanisms, you can create notebook pods/servers directly in the Kubeflow cluster using images provided by admins, and you can submit single node or distributed training jobs. You can also add the Kale extension which turbo-boosts the productivity of your model development, experimentation and deployment.
Does Arrikto provide support in regards to patching Kubeflow for example with new features or CVE issues?
The short answer is, “Yes”. Arrikto provides Arrikto Enterprise Kubeflow (EKF), which is an enterprise-grade Kubeflow distribution, and supports patching of Kubeflow for new features, enhancements, bug fixes, as well as CVE issues.
Is MiniKF planned to be available for Azure soon?
We are planning to release MiniKF in Azure in the future but there is no specific release date yet.