Engineering notes, incident walkthroughs, and product thinking from the Vouqis team.
The PR looked fine. The prompt change was small. The model still passed manual review. Then it shipped, and a week later someone noticed the output quality had quietly degraded. This is the failure mode Vouqis exists to prevent — and why "eval on every PR" is not optional once you ship AI.
Coming soonA score of 0.74 tells you nothing about what to do. A verdict tells you exactly what to do. We spent three weeks debating whether to surface the raw eval number or collapse it to three outcomes. Here is why we chose three outcomes — and why the threshold is the most important config value in vouqis.yml.
Coming soonThe first question teams ask when they set up vouqis.yml is: which directories should go in ai_paths? Prompt files are obvious. Model config is obvious. But what about the Python code that constructs the prompt? The retrieval logic? The system message template? This is our current thinking.
Coming soonWe ran Vouqis Verify with five teams before writing a single line of marketing copy. Every one of them had merged AI changes without running evals that week. Two had production regressions they traced back to prompt changes. This is what we learned, and what we built differently because of it.
Coming soon