Before You Build Anything in AI
Before You Build Anything in AI
Before You Build Anything in AI
Why most organizations fail to adopt AI effectively, and some guiding principles for thoughtful execution.
Share
In most organizations, the process of adopting AI usually starts with an email from a leader giving some directive to their teams to 'use AI'.
Typically, this instruction does not identify any particular use case, but is a general and vague order. What results is a flurry of activity around trying to meet the goal of ‘using AI’, rather than a pin-pointed approach of ‘using AI for a specific purpose’.
Once the flurry of activity has calmed, the question then becomes: ‘what should we apply AI to, and where do we even start?’
This is not a question to ask in passing. Rather, it is arguably the most consequential decisions about AI use in an organizational setting.
Much of the work of responsible AI is about dilemmas, which have trade-offs. And rather than finding solutions, you have to make trade-offs.
The more clearly a dilemma is articulated, the more coherently the trade-off can be made. This extends to governance, public policy, budgets, team structures, and organisational mobilisation.
Organisations attempting responsible innovation always encounter six challenges: not knowing where to start; not knowing how to structure the work; difficulty quantifying potential harms; low bandwidth; difficulty measuring impact; difficulty prioritising the work.
They often feel that AI is the answer to these questions, or some magical solution for them. But adopting AI in itself will not solve anything—as we will see from the following three case studies.
“The principle here is to start with the problem, rather than using AI for the sake of it.”
In most organizations, the process of adopting AI usually starts with an email from a leader giving some directive to their teams to 'use AI'.
Typically, this instruction does not identify any particular use case, but is a general and vague order. What results is a flurry of activity around trying to meet the goal of ‘using AI’, rather than a pin-pointed approach of ‘using AI for a specific purpose’.
Once the flurry of activity has calmed, the question then becomes: ‘what should we apply AI to, and where do we even start?’
This is not a question to ask in passing. Rather, it is arguably the most consequential decisions about AI use in an organizational setting.
Much of the work of responsible AI is about dilemmas, which have trade-offs. And rather than finding solutions, you have to make trade-offs.
The more clearly a dilemma is articulated, the more coherently the trade-off can be made. This extends to governance, public policy, budgets, team structures, and organisational mobilisation.
Organisations attempting responsible innovation always encounter six challenges: not knowing where to start; not knowing how to structure the work; difficulty quantifying potential harms; low bandwidth; difficulty measuring impact; difficulty prioritising the work.
They often feel that AI is the answer to these questions, or some magical solution for them. But adopting AI in itself will not solve anything—as we will see from the following three case studies.
“The principle here is to start with the problem, rather than using AI for the sake of it.”
In most organizations, the process of adopting AI usually starts with an email from a leader giving some directive to their teams to 'use AI'.
Typically, this instruction does not identify any particular use case, but is a general and vague order. What results is a flurry of activity around trying to meet the goal of ‘using AI’, rather than a pin-pointed approach of ‘using AI for a specific purpose’.
Once the flurry of activity has calmed, the question then becomes: ‘what should we apply AI to, and where do we even start?’
This is not a question to ask in passing. Rather, it is arguably the most consequential decisions about AI use in an organizational setting.
Much of the work of responsible AI is about dilemmas, which have trade-offs. And rather than finding solutions, you have to make trade-offs.
The more clearly a dilemma is articulated, the more coherently the trade-off can be made. This extends to governance, public policy, budgets, team structures, and organisational mobilisation.
Organisations attempting responsible innovation always encounter six challenges: not knowing where to start; not knowing how to structure the work; difficulty quantifying potential harms; low bandwidth; difficulty measuring impact; difficulty prioritising the work.
They often feel that AI is the answer to these questions, or some magical solution for them. But adopting AI in itself will not solve anything—as we will see from the following three case studies.
“The principle here is to start with the problem, rather than using AI for the sake of it.”
Case study one: risk and monitoring
A large social platform under regulatory pressure scaled its privacy operations to meet requirements.
The internal challenge was a structural one: the company operated vertically, with teams that did not regularly communicate, which made tackling the issue of privacy difficult, because it spans multiple functions. So, they build a central privacy infrastructure, with internal tooling, program management, a product review platform, incident management etc.
Within two years, the infrastructure was working, but it had also produced substantial bureaucracy. Every product launch passed through multiple review layers.
To solve this, leadership proposed introducing AI to accelerate review cycles. In theory, AI could automate parts of privacy review, translate legalese into product-readable summaries, and detect privacy vulnerabilities earlier. The risk profile, however, was extreme, and a failure in this system would cause regulatory breach. There was no question about whether AI would be capable for this job. The question was whether the amount of risk was acceptable.
They chose to approach the project with caution; with human-in-the-loop deployment and legal team involvement. For some of the programs, AI was deliberately not deployed at all.
Case study two: asking the right questions
The second case study is in perhaps more familiar territory: a leader proposed putting AI in customer support on the (incorrect) assumption that customers would prefer to interact with a chatbot.
A more productive approach to using AI in this context would be to look at the workflow, rather than fixate on the tool itself. Sitting with support agents, shadowing calls, and observing the day-to-day reveals where AI can actually help. It reveals that customer support personnel rarely find frustration in actually supporting customers, but rather in the busy work and administration surrounding this.
Understanding this, the questions then become: can AI reduce summarisation time? Can it surface relevant information faster — known outages affecting multiple customers, flagged accounts, similar past tickets? Can it reduce handle time without reducing quality?
The principle here is to start with the problem, rather than using AI for the sake of it.
Case study three: being realistic
The last example focuses on AI-assisted coding tools. When these became genuinely productive, many organisations responded with broad directives, such as: ‘everybody should be coding with AI.’
Advantages of AI-assisted coding include faster iteration, a shared language across functions, better cross-functional alignment, lower barriers to prototyping. But there are disadvantages too: token costs can escalate quickly, quality varies wildly between users, hallucinated requirements start to appear. And an overconfidence problem emerges: non-engineers start producing prototypes that they feel are polished, but that would not hold up if actually shipped.
AI-assisted coding for early ideation, drafting, and exploring alternatives is appropriate. AI-assisted coding for final specification, compliance-sensitive content, or anything requiring factual accuracy is not.
A broader principle we can draw out from these examples is that not every function should use and monitor AI in the same way. AI is simply a tool, not a replacement for expertise or processes, and output should still be audited.
Before attempting to building any AI system, answer these five questions:
What jobs are being attempted? Or what pain is being solved?
What are the success criteria? Think about accuracy, latency, cost, safety, adoption.
Who is accountable for approving AI output and maintains AI systems?
What is the risk? What is the result when the model goes wrong?
Is an LLM actually the right choice? A rules-based system, heuristics, or classical machine learning might be a better fit.
If there is one message to remember from all this, it is: think before you build. Deeply consider methods, systems, consequences, risk, before you start implementing anything, because it will be harder to retrospectively repair.
Ambitious use cases are worth pursuing — but not without prior planning and consideration.
This article is based on a talk given at Planhat Open 2026. Certain sections have been modified for editorial clarity.
Case study one: risk and monitoring
A large social platform under regulatory pressure scaled its privacy operations to meet requirements.
The internal challenge was a structural one: the company operated vertically, with teams that did not regularly communicate, which made tackling the issue of privacy difficult, because it spans multiple functions. So, they build a central privacy infrastructure, with internal tooling, program management, a product review platform, incident management etc.
Within two years, the infrastructure was working, but it had also produced substantial bureaucracy. Every product launch passed through multiple review layers.
To solve this, leadership proposed introducing AI to accelerate review cycles. In theory, AI could automate parts of privacy review, translate legalese into product-readable summaries, and detect privacy vulnerabilities earlier. The risk profile, however, was extreme, and a failure in this system would cause regulatory breach. There was no question about whether AI would be capable for this job. The question was whether the amount of risk was acceptable.
They chose to approach the project with caution; with human-in-the-loop deployment and legal team involvement. For some of the programs, AI was deliberately not deployed at all.
Case study two: asking the right questions
The second case study is in perhaps more familiar territory: a leader proposed putting AI in customer support on the (incorrect) assumption that customers would prefer to interact with a chatbot.
A more productive approach to using AI in this context would be to look at the workflow, rather than fixate on the tool itself. Sitting with support agents, shadowing calls, and observing the day-to-day reveals where AI can actually help. It reveals that customer support personnel rarely find frustration in actually supporting customers, but rather in the busy work and administration surrounding this.
Understanding this, the questions then become: can AI reduce summarisation time? Can it surface relevant information faster — known outages affecting multiple customers, flagged accounts, similar past tickets? Can it reduce handle time without reducing quality?
The principle here is to start with the problem, rather than using AI for the sake of it.
Case study three: being realistic
The last example focuses on AI-assisted coding tools. When these became genuinely productive, many organisations responded with broad directives, such as: ‘everybody should be coding with AI.’
Advantages of AI-assisted coding include faster iteration, a shared language across functions, better cross-functional alignment, lower barriers to prototyping. But there are disadvantages too: token costs can escalate quickly, quality varies wildly between users, hallucinated requirements start to appear. And an overconfidence problem emerges: non-engineers start producing prototypes that they feel are polished, but that would not hold up if actually shipped.
AI-assisted coding for early ideation, drafting, and exploring alternatives is appropriate. AI-assisted coding for final specification, compliance-sensitive content, or anything requiring factual accuracy is not.
A broader principle we can draw out from these examples is that not every function should use and monitor AI in the same way. AI is simply a tool, not a replacement for expertise or processes, and output should still be audited.
Before attempting to building any AI system, answer these five questions:
What jobs are being attempted? Or what pain is being solved?
What are the success criteria? Think about accuracy, latency, cost, safety, adoption.
Who is accountable for approving AI output and maintains AI systems?
What is the risk? What is the result when the model goes wrong?
Is an LLM actually the right choice? A rules-based system, heuristics, or classical machine learning might be a better fit.
If there is one message to remember from all this, it is: think before you build. Deeply consider methods, systems, consequences, risk, before you start implementing anything, because it will be harder to retrospectively repair.
Ambitious use cases are worth pursuing — but not without prior planning and consideration.
This article is based on a talk given at Planhat Open 2026. Certain sections have been modified for editorial clarity.
Alik Brundrett
Product Designer
Microsoft
Alik is a product leader at Microsoft AI, where he works on shaping next-generation AI experiences and bringing advanced capabilities to developers and enterprises. With over a decade of experience across product management and engineering, he has led initiatives at the intersection of AI, cloud, and developer platforms. His work focuses on translating cutting-edge research into scalable, user-facing products, balancing technical depth with a strong sense of product vision and usability.
