UNDER THE HOOD

Models on disk. Inference on your CPU.

Each model is a pre-quantized ONNX file (50–200 MB). PhotoSphere downloads them on-demand the first time you use a feature. Everything from there is local.

📥

1. One-time download

Models fetch from our CDN. SHA-256 verified. Resumable.

2. ONNX Runtime

Microsoft's runtime, optimized for CPU + GPU. Same engine as Office.

🧠

3. Run on your hardware

8 GB RAM works. GPU optional. M-series Macs and modern Intel/AMD fly.

🚫

4. Zero outbound

No telemetry, no API call, no "anonymized features." Verify with Wireshark.

MODEL 01 · OPENAI CLIP

Search by meaning, not by tags.

CLIP (Contrastive Language–Image Pretraining) maps photos and text into the same 512-dimensional space. Photos with similar meaning have similar embeddings. We compute one embedding per photo, store it in an HNSW index, and at query time embed your text query and find the nearest neighbors.

  • Indexing: ~80ms per photo on CPU, ~15ms with GPU.
  • Search: <200ms across 50,000-photo libraries.
  • Storage: 2 KB per photo (512 floats × 4 bytes).
  • Examples: "blue car at night", "kid blowing out candles", "my dog playing in snow", "receipt with coffee", "concert with red lights".
red car at night
142 results · 187ms · CPU only
DETECTED · 4 FACES
Mom Dad Sarah + unknown
MODELS 02 + 03 · RETINAFACE + ARCFACE

Faces clustered. Names you give.

Two-stage pipeline. RetinaFace detects every face in a photo with sub-pixel landmark precision. We crop each face to 112×112 and feed it into ArcFace, which produces a 512-dimensional embedding. Cosine similarity between embeddings clusters the same person across thousands of photos.

  • RetinaFace: face detection + 5-point landmarks (eyes, nose, mouth corners).
  • ArcFace: 99.6% accuracy on LFW benchmark.
  • Pet recognition: same architecture, trained on cats/dogs.
  • Manual override: merge clusters when AI splits one person; split clusters when it merges two.
MODEL 04 · YOLOv8

Every object, every photo, automatically tagged.

YOLOv8-nano runs on every photo at indexing time. 80 COCO classes covers most of what you photograph: people, vehicles, animals, food, sports gear, kitchen items, electronics, furniture. Auto-tags become regular searchable tags — you can edit, merge, or delete them.

  • ~50ms per photo on CPU, ~8ms with GPU.
  • Confidence threshold tunable (default 0.35).
  • Per-class NMS so we don't double-count overlapping boxes.
  • Bounding boxes visible in the inspector — verify what was detected.
personcardog catbicyclefood cakelaptopchair + 71 more
person · 0.94
bicycle · 0.87
backpack · 0.71
Brush selection
LaMa output
MODEL 05 · LaMa INPAINTING

Magic Eraser. Without the cloud.

LaMa (Large Mask Inpainting) takes a 4-channel input — your photo plus a binary mask of what to remove — and synthesizes plausible content for the masked area using Fourier convolutions. Unlike older inpainting algorithms, it handles large masks beautifully.

  • Selection: brush, lasso, or rectangle.
  • Resolution: 512×512 native; output upscaled and pasted back into the original.
  • Speed: ~3 seconds on CPU, ~0.6s with GPU.
  • Non-destructive: the original is preserved; the heal lives as an edit layer.
MODEL 06 · U²NET

Background remover. Instant cutouts.

U²Net is a salient-object segmentation network. Feed it a photo, get a soft alpha matte that separates the subject from the background. Use it to delete the background, swap skies (combined with masking), or extract subjects for product shots.

  • Subject extraction: portrait, product, pet — anything visually salient.
  • Sky replacement: uses the same alpha to mask in a new sky.
  • PNG export: with transparent background, ready for design.
  • Refinement: brush in/out for fringe correction.
Original
Alpha matte (U²Net)
CAFÉ DELLA STRADA
Cappuccino$4.50
Croissant$3.25
Tax$0.62
TOTAL$8.37
2024-03-15 · ORDER #4827
⌕ Searchable: "café della strada", "croissant", "$8.37"
MODELS 07 + 08 · PADDLEOCR

Find any text. In any photo.

Two models in sequence. DBNet locates text regions with bounding polygons. CRNN with CTC decoding reads each region into Unicode text. Output is indexed alongside filename/camera in SQLite FTS5 so receipts, signs, screenshots, and notes become full-text searchable.

  • DBNet: Differentiable Binarization for text detection.
  • CRNN: Convolutional + RNN + CTC for recognition. 6,000-character dictionary.
  • Languages: English + 25 more (Chinese, Japanese, Korean, Arabic, Cyrillic, Hindi…).
  • Throughput: ~1 second per photo with text, <100ms when no text detected.
EXTRA MODELS

Plus four more, for the polish.

NIMA · Aesthetic scoring

Neural Image Assessment scores every photo from 1–10 on aesthetic quality. Combined with a heuristic Laplacian variance for sharpness. Powers "auto-pick the best from a burst" and a "gems" Memory.

DeblurGAN-v2 · Photo unblur

Recover detail from motion blur and slight defocus. Generative deblurring trained on synthetic blur kernels. Run on the photos you almost deleted.

Fast Style Transfer · Painting filters

One ONNX model per artistic style (Van Gogh, Hokusai, Monet, etc.). Drop a style on a photo. Each style is small (~5 MB) and downloadable separately.

Scene classifier

Beach, mountain, urban, forest, indoor, sport, food, document, screenshot. A lightweight classifier auto-labels every photo. Searchable as a tag, mappable to album rules.

SYSTEM REQUIREMENTS

What you need.

Minimum CPUx86-64 with AVX2 (any 2015+ Intel/AMD), Apple M1+, ARM64 with NEON
Recommended CPUApple M-series, Intel Core i5 12th gen+, AMD Ryzen 5000+
Minimum RAM4 GB (basic library), 8 GB (with AI features)
Recommended RAM16 GB for libraries over 25,000 photos
GPU accelerationOptional. CUDA, DirectML, CoreML, ROCm — auto-detected if available
Disk for app100 MB
Disk for AI models~1.2 GB if you install all nine. Selective download.
Disk for libraryBring your own. Index lives at ~50 MB per 10,000 photos.
NetworkOne-time model download (CDN). After that: zero.

AI without the audit log.

Apple, Google, and Adobe also offer most of these features. They also see every photo. PhotoSphere doesn't.