screenmates

ScreenMates AI is a macOS menubar utility that enhances your interaction with screen content using local Vision Language Models (VLMs) powered by MLX Swift. Core Functionality: - On-Device VLM Processing: Leverages advanced Vision Language Models directly on your Mac for privacy and offline capability (after initial model download). - Advanced OCR: Extracts text from any image or content on your screen. - Custom Prompts for Image Understanding: Utilize the "Custom Prompt" panel to instruct the VLM on how to analyze or process the captured screenshot (e.g., describe the image, answer questions about its content, summarize visual data). - Selectable VLM Models: Choose from a predefined list of supported VLM models (downloaded on demand from Hugging Face Hub via the swift-hub library). The app will indicate download progress. - Menubar Integration: Operates as an accessory app from the menubar for quick access without cluttering the Dock. - Launch at Login: Optional setting to start ScreenMate AI automatically. Key Features: - Local AI: All VLM inference happens on your device, keeping your data private. - Versatile Image Interaction: Go beyond simple OCR; ask questions, get descriptions, or guide the VLM with your own text prompts. - Efficient Workflow: Integrated into the menubar for easy access while you work. - Model Choice: Select different VLMs based on your needs (list available in Settings).