Feras AllaouLet’s talk (opens in a new tab)
← Selected work

Case studyA friend’s company (anonymised)

Hiring an AI engineer when no one agrees what the job is

A company hired two or three engineers for an “AI engineer” role, and each left within one or two weeks. I told the CTO the problem wasn’t the engineers: nobody, them included, had defined the job. We defined it, changed how we interviewed, and the next hire is still there months later.

Earlier hires lasted
1–2 weeks
New hire so far
3–4 months
Interview rounds
3
My role
Part-time advisor

The problem

A friend runs a company that needed its next engineer, someone with AI experience. They tried two or three candidates. Each one started, stayed a week or two, and left.

He asked me to join. I didn’t have time for a full-time role, so I offered to help find out what was going wrong instead.

What I told the CTO

I was direct with the CTO: the problem isn’t the engineers. The problem is that “AI engineer” or “forward deployed engineer” isn’t a defined job yet, not at his company and not in the industry.

Ask ten companies what the role means and you get ten answers. So before looking at another candidate, we had to answer what this person would actually do.

What does an AI engineer do here?

  • Do they use AI to build the product faster?
  • Do they set up AI tools and workflows for the rest of the team?
  • Do they build AI agents for specific jobs in the company?
  • Are those agents run on a managed platform, or built and hosted by us?

The CTO’s honest answer was “a mix of all of them”. That was fine. Saying it out loud gave us a real AI engineer job description, and something to interview for.

How we filtered

  1. 1

    Team interview

    The company’s own interview: is this a person the team wants to work with every day?

  2. 2

    AI conversation

    My part. Plain questions about how they really use AI, to separate daily users from people who only read about it.

  3. 3

    Hands-on task

    A real problem to solve with any tools they like: Claude Code, Codex, anything. We watch how they work, not just the result.

AI engineer interview questions

  1. How do you use AI in your daily life?
  2. Which AI tools do you use, and why those?
  3. Do you use plugins or skills to extend them?
  4. Have you used Codex or Claude Code? What’s your take on them?
  5. Have you built agents yourself, with something like LangChain, LangGraph or Google’s Agent Development Kit?

Simple on purpose. People who use these tools every day answer in detail without thinking. People who don’t, can’t.

What we watched in the task

  • How they go back and forth with the AI: do they steer it, or accept whatever comes back?
  • Do they start by planning, or jump straight into code?
  • Do they write instruction files that keep the AI on track, or leave it open-ended?

Which framework someone knows matters less now, because AI agents write most of the code. What matters is how well a person directs those agents.

Who we hired

  • Wrote the instructions firstStarted with an instructions file for the agent (like AGENTS.md): what it’s for, the scope, how to break the work down, and a roadmap so the agent knows where it is.
  • Added a reviewerSet up a second agent to review the work, with clear criteria for what counts as good.
  • Used what experts already madeBrought in ready-made skills and plugins from well-known companies, such as Vercel’s, instead of reinventing best practices.
  • Gave the AI the right sourcesFor each tool they chose, like Figma, Vercel, NestJS or Express, checked whether that company offers an official MCP server or AI instructions, and made them available to the agent.

Like hiring a full-stack engineer, nobody knows everything on day one. This candidate didn’t either, but showed they were willing to learn, and that mattered as much as what they already knew.

The result

The person we hired is still there after three to four months. The work runs in a normal, steady flow, with no more people leaving after two weeks.

What I’d tell any team

  • If new hires keep leaving fast, look at the job before you look at the people.
  • It’s fine if the role is “a mix of everything”. Say so plainly, and interview for that.
  • Ask how people really use AI, then watch them do it. A hands-on task shows more than any CV.
  • Hire the person who directs AI well and is willing to learn, not the one who knows the most frameworks.

The company and the people involved are not named.

Hiring an AI engineer? Define the job first.

Tell me who you’re trying to hire. In 15 minutes we’ll work out what the role really is, and how to interview for it.

Book a free 15-min call (opens in a new tab)