Mend
Engineer. F26 Origin Hackathon, finalist.
Swift, SwiftUI, Supabase, PostGIS, Deno Edge Functions, PyTorch, Core ML
I built a crowdsourced infrastructure-inspection app in 72 hours. A photo of damage gets classified on the phone, matched to the building the camera faces, and checked by a vision model.
- On-device damage classifier (EfficientNet-B0, Core ML)
- 8 MB
- mAP, against 0.954 for ConvNeXt-tiny with 5x the parameters
- 0.949
- Radius of OpenStreetMap footprints each photo is ray-cast against
- 50 m
A classifier that fits on the phone
I trained the damage classifier on EfficientNet-B0 and shipped it as an 8 MB Core ML model. ConvNeXt-tiny scored 0.954 mAP against 0.949, but it has 5x the parameters, so the smaller one won. Along the way I caught train/test leakage: neighbouring frames from the same drone video were landing on both sides of the split, so I re-split by frame block.
Which building is this
Each photo gets matched to the asset the camera is facing by ray-casting its compass heading against OpenStreetMap building footprints within 50 m. When the ray misses, it falls back to the nearest footprint.
The verdict
The server verdict comes from a vision LLM with schema-enforced JSON: damage type, severity from 1 to 5, and confidence. Reporter notes are fenced off as untrusted input, and anything under 0.6 confidence goes to a human instead.
Offline first
Inspectors lose signal. Reports wait in an offline outbox with idempotent retry keys, so a re-sent report never files or pays twice.
Next: EcoPrompt