Prompt Engineering Adoption at Ivani

The problem

• As AI tools became available across Ivani, engineers were experimenting without a shared usage guide: prompting styles varied widely, and nobody had checked whether the differences actually changed the answers.
• Firmware-team queries involving the ESP-Zigbee SDK were a natural test case: technical enough to expose real prompting friction, and specific enough that a vague prompt's failure mode (context drift, a confidently invented API detail) would be easy to spot.

The approach

• Ran existing, as-written engineer prompts and deliberately restructured versions of the same queries side by side, locally, using Ollama running Qwen against sanitized firmware-team queries about the ESP-Zigbee SDK, at no external API cost.
• Local resource observations (CPU and memory on the test machine, via Task Manager) were kept separate from the qualitative comparison of the responses themselves: accuracy, whether the model stayed on the original constraint across a multi-turn exchange, and how much follow-up clarification each version needed. Task Manager showed local resource usage, not model reasoning, context usage, or output quality.
• The structured prompts more consistently held onto the original constraint across turns and asked for clarification instead of guessing when a query was ambiguous. The existing, as-written prompts more often produced a plausible-sounding but unverified API detail with no hedge attached.
• Turned the comparison into reusable material rather than a one-off writeup: a prompt template, a one-page cheat sheet, context-window guidance (how much surrounding code or spec to include before responses degraded), and a system-prompt template other engineers could adapt directly.

What broke (and how it got fixed)

• Existing, as-written prompts frequently let the model answer with unwarranted confidence: a wrong SDK function signature stated as fact, with no signal that the model was guessing. Prompts that explicitly asked the model to flag uncertainty reduced this, though not to zero.
• Longer, unstructured exchanges were where context drift showed up most: several turns in, the model would lose track of a constraint stated earlier, like a specific SDK version. Prompts that restated key constraints held up noticeably better across turns.
• There was no tooling to measure any of this precisely. This was a self-run, informal comparison, not an instrumented experiment: "which response was better" was a qualitative judgment call, not a scored metric.

Outcome

• Adoption spread by demonstrated usefulness rather than a formal training rollout: two engineers began using the templates first, and use spread across the roughly 10-person team over about two weeks.
• Observed estimates from this self-run comparison, not independently validated research, put the effect at a directional estimate of approximately 50% less engineer rework and a directional estimate of approximately 8 minutes saved per engineering query. There was no formal survey, no usage dashboard, no time-to-proficiency measurement, and no controlled adoption study behind either number.
• The reusable output (the prompt template, cheat sheet, context guidance, and system-prompt template) is what actually carried the finding into the team's daily use, not the comparison itself.

Stack

Honest scope

A retrospective written in September 2026 from personal working notes dated April–May 2026, not an internal Ivani document. Reflects one person's self-run, informal comparison at no external API cost, on a single local machine, using sanitized firmware-team queries, not a controlled study, and not independently validated. No confidential prompts, colleague identities, proprietary code, or internal screenshots are reproduced here.