← Back to Blog

Building Zero-Cloud Desktop AI: Swift Concurrency & Apple Intelligence on macOS

Native macOS utility applications have one non-negotiable architectural rule: Never block the main thread. If a clipboard app micro-stutters when a user hits a hotkey or copies text, it breaks the operating system's fluid user experience.

When integrating machine learning features into desktop applications, this rule becomes an engineering challenge. AI classification, text summarization, and OCR image analysis are computationally intensive tasks. In L2Cache, we built a zero-cloud architecture using Swift Concurrency, FoundationModels, and native Apple Silicon NPU hardware.

1. Structured On-Device AI with FoundationModels

In macOS 26+, Apple introduced the FoundationModels framework, enabling developers to prompt system-level generative AI models directly in Swift. To ensure deterministic outputs suitable for indexing, we use type-safe structured output annotations (@Generable and @Guide):

#if canImport(FoundationModels)
import FoundationModels

@available(macOS 26.0, *)
@Generable
struct ClipClassificationOutput {
    @Guide(description: "Specific title describing the content, max 80 chars.")
    var title: String

    @Guide(description: "1-4 lowercase tags. For CODE: first tag MUST be programming language.")
    var tags: [String]

    @Guide(description: "Best matching type: code, url, email, text, image, color, apiKey, unknown")
    var contentType: String

    @Guide(description: "Concise natural language summary under 30 words.")
    var aiSummary: String
}
#endif

By defining typed structs with field-level guidance annotations, the on-device language model generates structured JSON matching our domain models directly, avoiding fragile regex parsing of free-form text responses.

2. Actor Isolation & Sub-100ms Queue Architecture

Clipboard polling in L2Cache occurs every 0.5 seconds on a background actor. If a new clipboard item is captured, we execute AI categorization asynchronously off the main thread using Swift Concurrency (Task and actor state isolation):

actor AIProcessingQueue {
    private let provider: AIService
    
    init(provider: AIService) {
        self.provider = provider
    }
    
    func enqueue(clip: ClipItem) async {
        Task.detached(priority: .utility) {
            do {
                let result = try await self.provider.categorise(clip: clip)
                await self.persist(result, for: clip.id)
            } catch {
                // Graceful fallback to sub-millisecond regex rules
                await self.fallbackToRuleBased(clip: clip)
            }
        }
    }
}

Because the AI classification runs inside a background actor context at .utility task priority, the main UI thread remains completely unblocked. The floating panel opens in under 100ms regardless of whether an AI classification is actively processing in the background.

3. Multi-Pass Vision OCR off the Main Thread

To support extracting text from terminal and IDE screenshots without UI lag, we built a background VisionOCRService actor that runs multi-pass image analysis using Apple's Vision framework:

• Pass 1 (Barcodes & QR Codes): VNDetectBarcodesRequest scans for QR payloads. If circular dot-matrix modules are detected, a high-contrast grayscale pass using CIColorControls + CIUnsharpMask binarizes module edges for high recognition accuracy.
• Pass 2 (Text Recognition): VNRecognizeTextRequest configured with recognitionLevel = .accurate and custom developer vocabulary tokens (["SQL", "API", "CLI", "JSON", "iOS"]).
• Pass 3 (Privacy & Scene Labeling): VNClassifyImageRequest and VNDetectFaceRectanglesRequest identify faces or sensitive image content.

4. Diagnostic Pipeline & Graceful Fallback Cascading

On-device AI systems must handle runtime edge cases gracefully (e.g. older macOS versions, system language mismatches, or disabled system settings). L2Cache uses a multi-tier fallback cascade:

1. Tier 1 (Sub-millisecond Heuristics): Fast regex pattern matching for JWT tokens, SQL queries, hex colors, and API keys.
2. Tier 2 (On-Device Apple Intelligence): FoundationModels inference on Apple Silicon NPU.
3. Tier 3 (Local Ollama / Self-Hosted LLMs): User-configurable local model endpoint.

If Apple Intelligence is unavailable or returns an incomplete payload, the cascade automatically degrades to instant rule-based categorization without surfacing errors or breaking the user's flow.