Architecture

This document explains the design and architecture of the dotfiles management system.

Design Philosophy

The system follows a hybrid architecture combining:

Go CLI for orchestration, dependency resolution, and state management
Shell scripts for actual system operations and tool installation

This separation provides:

Type safety and structure from Go
Flexibility and portability from shell scripts
Cross-platform compatibility without requiring platform-specific Go code

For a deeper discussion of why Go was chosen over pure shell or other languages, and why the system builds from source rather than distributing pre-built binaries, see Design Rationale.

System Overview

Core Components

1. CLI Layer (Cobra Framework)

Location: cmd/dotfiles/

The command-line interface built with Cobra.

Commands:

install - Install modules with dependency resolution
uninstall - Uninstall modules and rollback changes
status - Show installed modules and their state
list - Show available modules
new - Generate new module skeleton
get-secret - Retrieve secrets (internal, called by shell scripts)
render-template - Render Go templates (internal, called by shell scripts)

Entry Point: main.go → cmd/dotfiles/root.go

2. Config Package

Location: internal/config/

Handles configuration loading and management.

Responsibilities:

Parse config.yml (main configuration file)
Parse profiles/*.yml (profile definitions)
Apply environment variable overrides
Provide module-specific settings lookup

Key Types:

type Config struct {
    Profile string
    Secrets SecretsConfig
    User    UserConfig
    Modules map[string]map[string]interface{}
}

3. Sysinfo Package

Location: internal/sysinfo/

Detects system information.

Capabilities:

Operating system detection (parses /etc/os-release for Linux)
Architecture detection (amd64, arm64)
Package manager detection (brew, apt, pacman)
Sudo availability check (non-blocking, 2s timeout)
Interactive terminal detection
Dotfiles directory resolution

Key Type:

type SystemInfo struct {
    OS            string
    Arch          string
    PackageManager string
    HasSudo       bool
    IsInteractive bool
    Home          string
    DotfilesDir   string
}

4. Module Package

Location: internal/module/

Core module system implementation.

Components:

Discovery (`discovery.go`)

Scans the engine’s modules/ directory — and, when DOTFILES_CONTENT_DIR is set, the overlay’s modules/ as a later root
Parses module.yml files
Validates module schema
Sorts by priority, then name
Content-wins layering: modules are merged across roots keyed on name (not directory). A module defined only in the engine is tagged built-in; a later (content) root that shadows a same-name engine module is override (whole-module replacement); a module that exists only in a content root is custom. dotfiles list/status surface these tags in a Source column when an overlay contributes.

Config overlay (`config.Load`)

Loads the base config.yml, then deep-merges the overlay’s config.yml ($DOTFILES_CONTENT_DIR/config.yml) over it — only the keys the overlay sets are changed, and YAML types are preserved.
Profiles resolve overlay-first: a bare --profile name prefers the overlay’s profiles/name.yml over the engine’s, so a content profile overrides a same-name built-in.
With no content dir set, every layer collapses to the engine alone — behavior is identical to having no overlay.

Schema (`schema.go`)

Defines module structure
YAML unmarshaling
Validation rules

type Module struct {
    Name         string
    Description  string
    Version      string
    Priority     int
    Dependencies []string
    OS           []string
    Requires     []string
    Files        []FileEntry
    Prompts      []Prompt
    Tags         []string
    Dir          string
}

Resolver (`resolver.go`)

Dependency resolution using Kahn’s algorithm (topological sort)
Transitive dependency expansion (BFS)
OS compatibility filtering
Cycle detection with descriptive errors
Deterministic ordering (priority → name)

Algorithm:

Build dependency graph
Expand transitive dependencies
Filter incompatible modules (OS mismatch)
Topological sort with priority ordering
Detect and report cycles

Runner (`runner.go`)

Module execution orchestration
Operation recording for rollback capability
8-phase lifecycle per module:
1. Prompts (if interactive)
2. Environment variable setup
3. Template context preparation
4. OS-specific script execution
5. Install script execution (with operation tracking)
6. File deployment (symlink/copy/template, with operation tracking)
7. Verification script execution
8. State recording (including operation history)
Automatic rollback on failure with interactive prompt

Interface Pattern:

type RunnerUI interface {
    StartSpinner(msg string) any
    StopSpinner(spinner any, success bool, msg string)
    // ... other UI methods
}

This interface decouples the module package from the UI package, preventing import cycles.

5. Secrets Package

Location: internal/secrets/

Pluggable secrets provider system.

Interface:

type Provider interface {
    Name() string
    Available() bool
    Authenticate() error
    IsAuthenticated() bool
    GetSecret(ref string) (string, error)
}

Implementations:

1Password (onepassword.go) - Integrates with op CLI
Noop (noop.go) - No-op provider for systems without secrets. This is the shipped default (secrets.provider: noop in the committed config.yml), so installs need no external tooling; 1Password is opt-in via the overlay or DOTFILES_SECRETS_PROVIDER.

1Password Provider:

Checks op CLI availability
Authenticates via op vault list
Retrieves secrets: op read op://vault/item/field
30-second timeout per operation
Caches authentication status

6. Template Package

Location: internal/template/

Go template rendering with custom functions.

Context:

type Context struct {
    User          map[string]string   // lowercase keys: name, email, github_user
    OS            string              // Operating system
    Arch          string              // Architecture
    Home          string              // Home directory
    DotfilesDir   string              // Repository path
    XDGConfigHome string              // Resolved XDG_CONFIG_HOME (env or ~/.config)
    Module        map[string]any      // config.yml modules.<name>.* settings only (NOT prompt answers)
    Secrets       map[string]string   // always an empty map here; secrets reach scripts via get_secret
    Env           map[string]string   // env overrides, incl. DOTFILES_PROMPT_* (prompt answers)
}

Access .User with lowercase keys — {{ .User.name }}, {{ .User.email }}, {{ .User.github_user }} (.User.Name renders empty). .Module carries only the module’s config.yml settings (including any content-overlay values, type-preserving); prompt answers are not in .Module — they arrive via .Env as DOTFILES_PROMPT_* (e.g. index .Env "DOTFILES_PROMPT_SSH_KEY_TYPE"). Since phase 4C the shell-invoked render-template subcommand builds this same context (with overlay) as the in-process runner.

Custom Functions:

env - Get environment variable
default - First non-empty value
upper, lower - Case conversion
contains - Substring check
join - Join slice with separator
trimSpace - Whitespace trimming

7. State Package

Location: internal/state/

Persistent module state tracking with operation history for rollback.

Storage:

Location: ~/.dotfiles/.state/
Format: JSON per module
Filename: <module-name>.json

ModuleState:

type ModuleState struct {
    Name        string
    Version     string
    Status      string      // "installed", "failed", "removed"
    InstalledAt time.Time
    UpdatedAt   time.Time
    OS          string
    Error       string
    Checksum    string
    Operations  []Operation // Operation history for rollback
}

Operation:

type Operation struct {
    Type      string            // "file_deploy", "dir_create", "script_run", "package_install"
    Action    string            // "created", "modified", "backed_up", "symlinked", "executed"
    Path      string            // File path or package name
    Timestamp time.Time
    Metadata  map[string]string // Additional context (backup_path, source, type, etc.)
}

Rollback Capability:

CanRollback() - Check if operations can be reversed
RollbackInstructions() - Generate human-readable rollback plan
RecordOperation() - Add operation to history with timestamp

Operation Types:

file_deploy: File or symlink creation/modification
dir_create: Directory creation
script_run: Shell script execution (informational, not rolled back)
package_install: Package manager installation (informational, not rolled back)

8. UI Package

Location: internal/ui/

Terminal user interface with colors and spinners.

Features:

ANSI color output (Catppuccin Mocha palette)
Animated spinners (braille characters)
Interactive prompts (input, confirm, choice)
Execution plan visualization
TTY detection with graceful fallback

Colors:

const (
    ColorRed    = "\033[38;5;210m"
    ColorGreen  = "\033[38;5;166m"
    ColorYellow = "\033[38;5;229m"
    ColorBlue   = "\033[38;5;147m"
    ColorGray   = "\033[38;5;245m"
    ColorReset  = "\033[0m"
)

9. Logging Package

Location: internal/logging/

Structured logging built on Go’s log/slog (Go 1.21+).

Logger Interface:

type Logger interface {
    Info(msg string, args ...any)
    Warn(msg string, args ...any)
    Error(msg string, args ...any)
    Debug(msg string, args ...any)
    Success(msg string, args ...any)
    With(args ...any) Logger
    WithGroup(name string) Logger
}

Handlers:

PrettyHandler: Colorized human-readable output for terminals
- [INFO], [WARN], [ERROR], [DEBUG], [OK] level prefixes
- ANSI color coding (cyan, yellow, red, magenta, green)
- Compact key=value attribute formatting
JSONHandler: Machine-readable structured output
- Standard log/slog JSON format
- Useful for log aggregation and analysis

Configuration:

type Config struct {
    Level     string    // "debug", "info", "warn", "error"
    Format    string    // "pretty", "json"
    Output    io.Writer // Defaults to os.Stderr
    AddSource bool      // Include source file/line
}

Usage:

# Enable JSON logging
dotfiles install git --log-json

# Enable debug logging
dotfiles install git --verbose

Data Flow

Installation Flow

The installation flow proceeds through system detection, module discovery, dependency resolution, and sequential module execution (prompts → environment setup → OS script → install script → file deployment → verification → state recording).

Module Execution Flow

Each module runs through 8 phases: prompts, environment variable injection, template context preparation, OS-specific script, install script, file deployment, verification, and state recording.

Shell Script Execution

The Go runner wraps each shell script in a generated wrapper that sources lib/helpers.sh first, then executes the module script. All DOTFILES_* environment variables are available to the script.

Dependency Resolution

Algorithm: Kahn’s Topological Sort

Example

# Module dependencies
ssh: []
git: [ssh]
zsh: [git]
neovim: [git]

Execution order:

ssh (priority 20, no deps)
git (priority 30, after ssh)
zsh (priority 40, after git)
neovim (priority 50, after git)

Note: zsh and neovim could run in parallel since they only depend on git, but the system runs sequentially for simplicity.

Module Lifecycle

Module Directory Structure:
modules/example/
├── module.yml      # Metadata
├── install.sh      # Main logic (required)
├── verify.sh       # Verification (optional)
├── os/             # OS-specific (optional)
│   ├── macos.sh
│   ├── ubuntu.sh
│   └── arch.sh
└── files/          # Config files (optional)
    └── config.conf

Execution Phases

1. Prompts

Only in interactive mode
Answers stored as DOTFILES_PROMPT_<KEY>

2. Environment Variables

System info: OS, arch, package manager
Paths: home, dotfiles dir, module dir
User config: name, email, github user
Execution context: dry-run, verbose flags
Prompt answers

3. Template Context

Combines all data sources
Available to template rendering

4. OS Script

Runs only os/${DOTFILES_OS}.sh if it exists
Platform-specific setup
Has access to helpers

5. Install Script

Runs install.sh (required)
Main installation logic
Has access to helpers

6. File Deployment

Processes all files from module.yml
Creates backups of existing files
Symlinks, copies, or renders templates

7. Verification

Runs verify.sh if it exists
Post-install checks
Can fail the module

8. State Recording

Writes JSON to .state/ directory
Records success/failure, timestamps, errors

Communication Patterns

Go → Shell

Environment Variables:

export DOTFILES_OS="ubuntu"
export DOTFILES_ARCH="amd64"
export DOTFILES_VERBOSE="true"
# ... many more

Wrapper Script:

#!/usr/bin/env bash
set -euo pipefail
source lib/helpers.sh
source modules/example/install.sh

Shell → Go

Subprocess Calls:

# In module script:
SECRET=$(get_secret "op://vault/item/field")

# Helper function calls Go CLI:
get_secret() {
    "$DOTFILES_BIN" get-secret --ref "$1"
}

# In module script:
render_template "config.tmpl" ~/.config/app/config

# Helper function calls Go CLI:
render_template() {
    "$DOTFILES_BIN" render-template --src "$1" --dest "$2"
}

Design Patterns

1. Interface Decoupling

Problem: Module package needed UI functionality but importing UI caused cycles.

Solution: RunnerUI interface in module package:

// In module package
type RunnerUI interface {
    StartSpinner(msg string) any
    StopSpinner(spinner any, success bool, msg string)
}

// UI package implements it
func (u *UI) StartSpinner(msg string) any {
    // Implementation
}

2. Pluggable Providers

Secrets Provider:

type Provider interface {
    Name() string
    Available() bool
    Authenticate() error
    IsAuthenticated() bool
    GetSecret(ref string) (string, error)
}

Easy to add new providers (AWS Secrets Manager, HashiCorp Vault, etc.)

3. Phased Execution

Module execution is broken into discrete phases:

Clear separation of concerns
Easy to add new phases
Consistent error handling per phase

4. Composition over Inheritance

Modules are composed of:

Metadata (module.yml)
Scripts (install.sh, os/*.sh, verify.sh)
Files (files/*)

No base class or inheritance hierarchy.

Performance Considerations

Parallel Execution

Currently sequential, but architecture supports parallel execution:

Dependency graph enables parallelism
Modules at same dependency level could run concurrently
State tracking handles concurrent writes

Caching

Module Discovery: Cached during execution
System Info: Detected once at startup
Authentication: 1Password auth cached for session
Docker Layer Caching: CI uses GitHub Actions cache

Minimal Dependencies

Small Go binary (~5-10MB)
No runtime dependencies except bash
Modules pull in their own dependencies

Security Model

Trusted Boundaries

Trusted:

Environment variables
CLI flags
config.yml (user’s own file)
module.yml files (repo content)

Untrusted:

Network downloads (modules should verify checksums)
External secrets (validated by provider)

Secrets Handling

Never logged or displayed
Passed via environment or temp files
Provider authentication required
30-second timeouts prevent hanging

File Operations

Automatic backups before overwrite
Permissions preserved on copy
Symlinks created safely
No arbitrary file writes (destination in module.yml)

Testing Strategy

Unit Tests

Located in internal/*/ with *_test.go
Test individual packages in isolation
Run with go test ./...

Integration Tests

Docker-based for Ubuntu and Arch
Full installation in clean container
Verify all modules work end-to-end
Run with make test-integration

CI Pipeline

Unit Tests (always)
    ↓
Integration Tests (matrix: ubuntu, arch)
    ↓
All green → Ready to merge

Extensibility Points

Add New Command

Create cmd/dotfiles/newcommand.go
Implement Cobra command
Add to root command

Add New Module

Create modules/newmodule/
Add module.yml
Add install.sh
Optionally add OS scripts, files

Add New Secrets Provider

Implement secrets.Provider interface
Add provider factory in config
Update config.yml schema

Add Template Function

Add function to template/render.go
Register in template.FuncMap

Future Enhancements

Potential improvements:

Parallel Module Execution - Run independent modules concurrently
Module Marketplace - Discover and install community modules
Rollback Support - Undo failed installations
Diff Preview - Show file changes before applying
Remote Profiles - Load profiles from URLs
Plugin System - Extend CLI with plugins
GUI Interface - Web or desktop UI for management

Architecture

Architecture

Design Philosophy

System Overview

Core Components

1. CLI Layer (Cobra Framework)

2. Config Package

3. Sysinfo Package

4. Module Package

Discovery (discovery.go)

Config overlay (config.Load)

Schema (schema.go)

Resolver (resolver.go)

Runner (runner.go)

5. Secrets Package

6. Template Package

7. State Package

8. UI Package

9. Logging Package

Data Flow

Installation Flow

Module Execution Flow

Shell Script Execution

Dependency Resolution

Algorithm: Kahn’s Topological Sort

Example

Module Lifecycle

Execution Phases

Communication Patterns

Go → Shell

Shell → Go

Design Patterns

1. Interface Decoupling

2. Pluggable Providers

3. Phased Execution

4. Composition over Inheritance

Performance Considerations

Parallel Execution

Caching

Minimal Dependencies

Security Model

Trusted Boundaries

Secrets Handling

File Operations

Testing Strategy

Unit Tests

Integration Tests

CI Pipeline

Extensibility Points

Add New Command

Add New Module

Add New Secrets Provider

Add Template Function

Future Enhancements

References

Discovery (`discovery.go`)

Config overlay (`config.Load`)

Schema (`schema.go`)

Resolver (`resolver.go`)

Runner (`runner.go`)