Journal of Dizi's Life

Zero-to-One MLOps (Part 2): Hands-on Guide to k3s, MinIO Helm, and Traefik Ingress

5 min read

In Part 1, we covered the architectural rationale and the 5-phase blueprint.

This post is the hands-on implementation guide for Phase 0: provisioning a lightweight k3s cluster on an AMD EPYC VM, deploying MinIO S3 object storage via Helm and decoupled Terraform loaders, and securing the web console with Cloudflare Universal SSL.

┌────────────────────────────────────────────────────────────────────────┐
│ 1. Cloud Perimeter (Oracle Cloud Infrastructure - OCI)                 │
│    • OCI Network Security Group (NSG): Ports 22, 6443, 80, 443 Open    │
│      (Admin IP whitelisted for internal dashboard ports)               │
│    • Compute Node: AMD EPYC Genoa (4 OCPU / 32GB RAM / 200GB NVMe)     │
└───────────────────────────────────┬────────────────────────────────────┘
                                    │
                                    ▼
┌────────────────────────────────────────────────────────────────────────┐
│ 2. Kubernetes Control Plane (k3s v1.31)                                │
│    • Embedded SQLite & Containerd                                      │
│    • Traefik Ingress Controller (:80 / :443)                           │
│    ┌─────────────────┬─────────────────┬───────────────┬──────────────┐│
│    │ mlops-system    │ kubeflow        │ mlflow        │ katib        ││
│    │ (MinIO & Secret)│ (KFP v2 Engine) │ (Tracking Svr)│ (AutoML)     ││
│    └────────┬────────┴─────────────────┴───────────────┴──────────────┘│
└─────────────┼──────────────────────────────────────────────────────────┘
              │
              ▼
┌────────────────────────────────────────────────────────────────────────┐
│ 3. S3 Storage & Public Routing Layer                                   │
│    • MinIO S3 Server (Helm Chart 5.4.0) with 50Gi NVMe PVC             │
│    • Auto-Initialized Buckets:                                         │
│      ├── xidx-market-data (Point-in-time EOD Parquets)                 │
│      ├── feature-store (Calculated ATR & transformed features)         │
│      ├── model-registry (Blessed AutoGluon artifacts)                  │
│      ├── mlflow-artifacts (Experiment runs & metric traces)            │
│      └── idx-filings (Corporate disclosure text & PDF attachments)     │
│    • Traefik Ingress: https://minio-sentinel.diziasp.tech (Cloudflare) │
└────────────────────────────────────────────────────────────────────────┘

1. Directory Structure & Layout

All infrastructure code is organized under deployment/kubernetes/:

deployment/kubernetes/
├── backend.tf                # Local state backend
├── versions.tf               # Terraform, Kubernetes (~> 2.36), Helm (~> 2.17)
├── provider.tf               # Providers linked to ~/.kube/config-mlops-sentinel
├── variables.tf              # Config inputs (storage size, base domain, admin credentials)
├── terraform.tfvars          # Root passwords & domain overrides (gitignored)
├── outputs.tf                # HTTPS console URL and internal S3 DNS endpoints
├── main.tf                   # Single orchestrator (Namespaces, Helm, Secret, Dynamic Ingress)
├── helm-values/
│   └── minio.yaml            # Declarative Helm values for official MinIO chart
└── ingress/
    └── minio.yaml            # Declarative Ingress manifest for minio-sentinel.<domain>

2. Step-by-Step Implementation Runbook

Step 1: Provision Cloud Perimeter (OCI Network Security Group)

Lock down the perimeter firewall to allow SSH, Kubernetes API, and standard web traffic, while whitelisting sensitive dashboard ports exclusively to your admin IP:

cd infra/terraform/oci
terraform init && terraform apply

Ports configured:

  • 22 (SSH) & 6443 (Kubernetes API server)
  • 80 (HTTP) & 443 (HTTPS Ingress)
  • 9000, 9001, 5000, 8080 restricted strictly to var.admin_cidr (<YOUR_IP>/32).

Step 2: Bootstrap k3s via Ansible

Run the automation playbooks to install k3s (v1.31 LTS channel) with embedded SQLite:

cd infra/ansible
ansible-playbook -i inventory.ini 00-bootstrap-node.yaml
ansible-playbook -i inventory.ini 01-install-k3s.yaml

Sync the generated kubeconfig to your local machine:

scp ubuntu@<VM_PUBLIC_IP>:/etc/rancher/k3s/k3s.yaml ~/.kube/config-mlops-sentinel
sed -i '' 's/127.0.0.1/<VM_PUBLIC_IP>/g' ~/.kube/config-mlops-sentinel
export KUBECONFIG=~/.kube/config-mlops-sentinel
kubectl get nodes -o wide

Step 3: Declarative MinIO & Ingress Configuration

A. MinIO Helm Values (helm-values/minio.yaml)

We configure standalone MinIO with a 50Gi PersistentVolumeClaim on the local-path storage provisioner and declare five auto-initialized buckets:

rootUser: "${root_user}"
rootPassword: "${root_password}"
 
mode: standalone
replicas: 1
 
persistence:
  enabled: true
  size: "${storage_size}"
 
service:
  type: ClusterIP
  port: 9000
 
consoleService:
  type: ClusterIP
  port: 9001
 
buckets:
  - name: xidx-market-data
    policy: none
    purge: false
  - name: feature-store
    policy: none
    purge: false
  - name: model-registry
    policy: none
    purge: false
  - name: mlflow-artifacts
    policy: none
    purge: false
  - name: idx-filings
    policy: none
    purge: false
 
resources:
  requests:
    cpu: 250m
    memory: 512Mi
  limits:
    cpu: 2000m
    memory: 4Gi

B. Dynamic Kubernetes Orchestrator (main.tf)

Instead of embedding raw Ingress metadata in Terraform, main.tf acts as a lightweight orchestrator with a dynamic fileset manifest loader:

locals {
  namespaces = ["mlops-system", "kubeflow", "mlflow", "katib"]
}
 
resource "kubernetes_namespace" "namespaces" {
  for_each = toset(local.namespaces)
  metadata {
    name = each.value
    labels = {
      "app.kubernetes.io/part-of" = "mlops-sentinel"
    }
  }
}
 
resource "helm_release" "minio" {
  name       = "minio"
  repository = "https://charts.min.io/"
  chart      = "minio"
  version    = "5.4.0"
  namespace  = kubernetes_namespace.namespaces["mlops-system"].metadata[0].name
 
  values = [
    templatefile("${path.module}/helm-values/minio.yaml", {
      root_user     = var.minio_root_user
      root_password = var.minio_root_password
      storage_size  = var.minio_storage_size
    })
  ]
}
 
resource "kubernetes_secret" "s3_credentials" {
  metadata {
    name      = "s3-credentials"
    namespace = kubernetes_namespace.namespaces["mlops-system"].metadata[0].name
  }
 
  data = {
    AWS_ACCESS_KEY_ID     = var.minio_root_user
    AWS_SECRET_ACCESS_KEY = var.minio_root_password
    AWS_DEFAULT_REGION    = "us-east-1"
    S3_ENDPOINT           = "http://minio.mlops-system.svc.cluster.local:9000"
  }
 
  type = "Opaque"
}
 
# Dynamic Ingress Loader: deploys any manifest dropped in ./ingress/
resource "kubernetes_manifest" "ingress" {
  depends_on = [helm_release.minio]
  for_each   = fileset("${path.module}/ingress", "*.yaml")
 
  manifest = yamldecode(templatefile("${path.module}/ingress/${each.value}", {
    base_domain = var.base_domain
  }))
}

C. Traefik Ingress Manifest (ingress/minio.yaml)

Expose only the MinIO Web Console (:9001) to public HTTPS. The S3 data plane (:9000) remains strictly internal:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: minio-ingress
  namespace: mlops-system
  labels:
    app.kubernetes.io/part-of: mlops-sentinel
  annotations:
    kubernetes.io/ingress.class: "traefik"
    traefik.ingress.kubernetes.io/router.entrypoints: "web,websecure"
spec:
  ingressClassName: traefik
  rules:
    - host: "minio-sentinel.${base_domain}"
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: minio-console
                port:
                  number: 9001

Deploy the cluster layer:

cd deployment/kubernetes
terraform init && terraform apply

Step 4: Configure Cloudflare DNS & SSL

In Cloudflare Dashboard:

  1. Add DNS Record:
    • Type: A
    • Name: minio-sentinel
    • Target: <VM_PUBLIC_IP>
    • Proxy Status: Proxied (Orange Cloud ☁️)
  2. SSL/TLS Settings:
    • Set Encryption Mode to Full.
    • Enable Always Use HTTPS under Edge Certificates.

3. Verification & Smoke Testing

1. Web Console Check

Navigate to https://minio-sentinel.<your-domain>.tech:

  • Verify TLS 1.3 padlock.
  • Log in and confirm the 5 initialized buckets (xidx-market-data, feature-store, model-registry, mlflow-artifacts, idx-filings).

2. S3 API Boto3 Verification

Forward the internal S3 API port locally and verify bucket access:

kubectl --kubeconfig ~/.kube/config-mlops-sentinel port-forward -n mlops-system deployment/minio 9000:9000 &
import boto3
 
s3 = boto3.client(
    "s3",
    endpoint_url="http://localhost:9000",
    aws_access_key_id="sentinel_admin",
    aws_secret_access_key="<YOUR_PASSWORD>"
)
 
buckets = [b["Name"] for b in s3.list_buckets()["Buckets"]]
print("Available Buckets:", buckets)
assert "xidx-market-data" in buckets
print("✅ S3 Storage Layer Verified!")

4. Summary

With Phase 0 complete:

  • The k3s control plane is active.
  • S3 storage is persistent on 50GB NVMe with standard credentials.
  • The web console is securely routed via Cloudflare Universal SSL and Traefik Ingress.

In Part 3, we implement Phase 1: Point-in-Time Market Data Lineage, building deterministic ingestion with calendar alignment and RFC 8785 cryptographic Parquet manifests.

Graph View

Backlinks