Back to Blog

Set up Azure ML workspaces with CLI

Complete tutorial about az ml workspace create in Azure CLI. Learn workspace creation, identity, storage linkage, compute targets.

Set up Azure ML workspaces with CLI

Set Up Azure ML Workspaces with CLI

Introduction

Creating an Azure Machine Learning (ML) workspace is a fundamental step for anyone looking to leverage Azure's capabilities for machine learning projects. An Azure ML workspace serves as a centralized hub where you can manage and collaborate on machine learning resources including datasets, experiments, models, and compute resources. By utilizing Azure CLI, you can automate the workspace creation process, making it easier to manage your ML resources efficiently and effectively.

The az ml workspace create command plays a crucial role in establishing a workspace and its associated resources, such as Azure Storage and Azure Key Vault. This tutorial will guide you through the process of setting up an Azure ML workspace using Azure CLI, discussing its significance in AI & ML projects, and providing practical examples to ensure you can apply your newfound knowledge effectively.

Prerequisites

Before proceeding, ensure you have the following:

  • Azure CLI installed on your machine. You can download it from here.
  • An active Azure subscription.
  • Proper permissions to create resources within your Azure subscription (e.g., Contributor or Owner role).
  • Authentication configured with Azure CLI. You can authenticate by running az login.

Fundamental Concepts

Key Terminology

  • Workspace: A centralized resource for managing machine learning activities, including datasets, experiments, and models.
  • Resource Group: A container that holds related Azure resources for an Azure solution.
  • Compute Target: A virtual machine or cluster used to run your ML experiments.

When to Use

Use the az ml workspace create command whenever you need to:

  • Set up a new workspace for a machine learning project.
  • Automate the creation of associated resources like storage, container registries, and application insights.
  • Integrate with CI/CD pipelines for MLOps.

Command Syntax

The basic syntax for creating a workspace is as follows:

az ml workspace create --name <workspace-name> --resource-group <resource-group-name> [--location <location>] [--tags <tags>] [--file <configuration-file>]

Parameters Table

Parameter Short Form Description
--name -n The name of the Azure ML workspace.
--resource-group -g The name of the resource group.
--location -l The location of the new workspace.
--tags -t Space-separated key-value pairs for tagging resources.
--file -f Local path to the YAML file containing workspace settings.

Practical Examples

1. Basic Workspace Creation

Create a new workspace with default settings.

az ml workspace create --name myWorkspace --resource-group myResourceGroup --location eastus

2. Workspace with Tags

Create a workspace and tag it for organization.

az ml workspace create --name myWorkspace --resource-group myResourceGroup --location eastus --tags project=AI-Project

3. Workspace Using Existing Resources

Use a YAML configuration file to create a workspace with existing resources.

az ml workspace create --resource-group myResourceGroup --file workspace-config.yml

4. Enable Data Isolation

Create a workspace with data isolation enabled.

az ml workspace create --name myWorkspace --resource-group myResourceGroup --location eastus --enable-data-isolation true

5. Create Workspace with Application Insights

Link an existing Application Insights resource.

az ml workspace create --name myWorkspace --resource-group myResourceGroup --application-insights <app-insights-id>

6. Create Workspace with Key Vault

Link an existing Azure Key Vault.

az ml workspace create --name myWorkspace --resource-group myResourceGroup --key-vault <key-vault-id>

7. Create Workspace with a Container Registry

Link an existing Azure Container Registry.

az ml workspace create --name myWorkspace --resource-group myResourceGroup --container-registry <container-registry-id>

8. Create Workspace with User-Assigned Managed Identity

Assign a user-managed identity during workspace creation.

az ml workspace create --name myWorkspace --resource-group myResourceGroup --primary-user-assigned-identity <identity-id>

Real-World Use Cases

Scenario 1: Collaborative ML Project

A team of data scientists needs to work on a machine learning project. They create a workspace to manage datasets, experiments, and models. By using Azure CLI, they can automate the setup, allowing them to focus on development.

Scenario 2: Automated CI/CD for ML Models

In a continuous integration and deployment (CI/CD) pipeline, automated scripts create and manage Azure ML workspaces. This ensures that each deployment has a clean environment, improving efficiency and reducing errors.

Scenario 3: Resource Management for Multiple Projects

A company runs several machine learning projects simultaneously. By creating separate workspaces for each project, they can manage resources, costs, and access controls effectively, using tags to track expenses per project.

Best Practices

  1. Naming Conventions: Use clear and descriptive names for workspaces and resources to avoid confusion.
  2. Resource Group Management: Organize workspaces in resource groups based on projects or departments.
  3. Tagging: Implement a consistent tagging strategy for better cost management and resource identification.
  4. Security: Use managed identities and key vaults for secure management of sensitive information.
  5. Automation: Automate workspace creation with scripts to streamline the setup process for new projects.

Common Errors

  1. Error: Workspace Already Exists

    • Cause: Trying to create a workspace with a name that is already taken.
    • Solution: Use a different workspace name or specify --exist-ok to allow the command to succeed even if the workspace exists.
  2. Error: Invalid Resource Group

    • Cause: Specified resource group does not exist.
    • Solution: Create the resource group first using az group create.
  3. Error: Missing Required Parameters

    • Cause: Not providing necessary parameters like workspace name or resource group.
    • Solution: Ensure all required parameters are included in the command.
  4. Error: Insufficient Permissions

    • Cause: Lack of appropriate permissions to create resources.
    • Solution: Contact your Azure administrator to assign the necessary roles.

Related Commands

Command Description
az ml workspace show Show details of a specific workspace.
az ml workspace delete Delete a workspace.
az ml workspace update Update properties of a workspace.
az ml compute create Create a compute target for workloads.

Automation Script

Here is a simple Bash script to automate the creation of an Azure ML workspace:

#!/bin/bash

# Variables
WORKSPACE_NAME="myWorkspace"
RESOURCE_GROUP="myResourceGroup"
LOCATION="eastus"

# Create resource group
az group create --name $RESOURCE_GROUP --location $LOCATION

# Create Azure ML workspace
az ml workspace create --name $WORKSPACE_NAME --resource-group $RESOURCE_GROUP --location $LOCATION

echo "Azure ML workspace '$WORKSPACE_NAME' created successfully in resource group '$RESOURCE_GROUP'."

Conclusion

Setting up an Azure ML workspace using Azure CLI is a straightforward process that can significantly enhance your machine learning workflows. By automating the creation and management of workspaces, you can streamline your projects and focus on building innovative ML solutions. Explore further by integrating your workspace with compute targets and data stores to fully utilize Azure's capabilities in AI and ML.

References