Local Deployment

This guide covers deploying ML pipelines to local machines using Ansible.

Overview

Local deployments are available for:

Directory Structure

deploy/local/{machine}/
├── 01-infrastructure/     # VM/hardware setup
│   └── Vagrantfile
├── 02-configuration/      # Ansible configuration
│   ├── inventory.ini
│   └── playbook.yml
├── 03-application/        # Application config
│   ├── config.json
│   ├── requirements.txt
│   └── run.sh
├── 04-datasources/        # Data sources
└── 05-identities/         # Credentials

Deploying to Local Machine (asus)

The asus deployment runs mlflow-tf pipeline locally.

Quick Start

cd deploy/local/asus/03-application
chmod +x run.sh
./run.sh all

Using Ansible

cd deploy/local/asus/02-configuration
ansible-playbook -i inventory.ini playbook.yml

Configuration

Edit 03-application/config.json:

{
    "mlflow": {
        "tracking_uri": "http://localhost:5000",
        "experiment_name": "tensorflow_training_asus"
    },
    "training": {
        "epochs": 10,
        "batch_size": 32
    }
}

Deploying to Remote Machine (thinkpad)

The thinkpad deployment runs mlflow-sklearn pipeline on a remote machine.

Prerequisites

1. SSH key-based authentication
2. Ansible installed locally

Setup SSH Keys

# Generate SSH key
ssh-keygen -t rsa -b 4096

# Copy to thinkpad
ssh-copy-id thom@192.168.1.100

Update Inventory

Edit 02-configuration/inventory.ini:

[thinkpad]
thinkpad ansible_host=192.168.1.100 ansible_user=thom ansible_python_interpreter=/usr/bin/python3

Deploy

cd deploy/local/thinkpad/02-configuration

# Test connectivity
ansible thinkpad -i inventory.ini -m ping

# Run deployment
ansible-playbook -i inventory.ini playbook.yml

Run Pipeline on Thinkpad

ssh thom@192.168.1.100
cd ~/mlops-with-mlflow/deploy/local/thinkpad/03-application
./run.sh all

Ansible Playbook Details

The playbook performs these steps:

1. Install system packages

   - name: Install required packages
     apt:
       name: [python3, python3-pip, python3-venv, git]
   

2. Clone repository

   - name: Clone project
     git:
       repo: "{{ repo_url }}"
       dest: "{{ project_root }}"
   

3. Create virtual environment

   - name: Create venv
     command: python3 -m venv {{ venv_dir }}
   

4. Install dependencies

   - name: Install requirements
     pip:
       requirements: "{{ deploy_dir }}/requirements.txt"
       virtualenv: "{{ venv_dir }}"
   

5. Install local libraries

   - name: Install local libraries
     pip:
       name: "{{ project_root }}/src/{{ item }}"
       editable: yes
     loop:
       - doe-library
       - feature-library
       - io-library
   

6. Configure MLflow server (as systemd service)

MLflow Server Management

Start Server

# Manual start
mlflow server \
    --backend-store-uri sqlite:///mlflow.db \
    --default-artifact-root ./mlruns \
    --host 0.0.0.0 \
    --port 5000

Systemd Service

# Check status
sudo systemctl status mlflow-server

# Restart
sudo systemctl restart mlflow-server

# View logs
sudo journalctl -u mlflow-server -f

Troubleshooting

Connection Issues

# Test SSH
ssh -v thom@192.168.1.100

# Test Ansible
ansible thinkpad -i inventory.ini -m ping -vvv

Permission Issues

# Fix script permissions
chmod +x run.sh

# Fix ownership
sudo chown -R $USER:$USER ~/mlops-with-mlflow

MLflow Issues

# Check if server is running
curl http://localhost:5000/health

# Check logs
tail -f /var/log/mlflow-server.log