Jobs › Direct Supply

AI Engineer

Direct Supply · Milwaukee, WI · Full time

Posted Aug 11, 2026 · We last checked this listing on Oct 6, 2026

Apply at Direct Supply

Likely interview questions for this role

Written from this job description, not a generic list. Each one notes what the interviewer is really checking.

About your past

Tell me about an AI system you shipped to production. What did you own and what happened after launch?

What they are checking: A real track record of shipping and operating AI systems, which the posting stresses.

How do you partner with a product manager to sharpen their product vision rather than just take tickets?

What they are checking: Product ownership mindset, which the posting expects from engineers.

Tell me about a tool or practice you introduced that moved your whole team forward.

What they are checking: Whether they drive the organization forward on tooling, as the posting asks.

What would you do

A new model capability comes out. How do you decide whether it creates clear leverage for your product or is just a distraction?

What they are checking: Disciplined product judgment about adopting new AI capabilities.

You are working directly with a customer who wants a feature that you think will not solve their real problem. What do you do?

What they are checking: Customer-facing judgment and willingness to dig into the underlying need.

These systems support care for seniors. What safeguards would you build into an AI feature used by caregivers?

What they are checking: Awareness of risk, privacy and human oversight when AI touches care delivery.

About the work

Walk me through how you would turn an ambiguous customer problem into a prototype and a technical spec.

What they are checking: Ability to move from fuzzy needs to something concrete, which the role expects early in the lifecycle.

How do you evaluate an LLM-based feature before and after deployment? What would you measure?

What they are checking: Evaluation rigor, since the posting lists evaluation and monitoring as owned responsibilities.

Describe a system design trade-off you made between cost, latency and quality in an AI feature.

What they are checking: Architecture discipline and the judgment to own trade-offs rather than default to the biggest model.

How do you build codebases that are legible to both other developers and AI agents?

What they are checking: Thinking about structure, conventions and docs for agent-assisted development, which the posting names.

Your AI feature starts giving worse answers in production but nothing in the code changed. How do you investigate?

What they are checking: Operating discipline for AI systems, where inputs and models shift underneath stable code.

Practice this interview out loud.

Offer builds a real interview for this exact role at Direct Supply from your resume and this job description, asks the questions one at a time, and tells you what landed. The first one is free.

Practice this out loud

The full job description

As published by Direct Supply.

About the Role Direct Supply is building AI systems that change how care is delivered to millions of seniors and support the people who care for them. We're hiring engineers to design, ship, and operate those systems in production. This role spans the full product lifecycle: discovery, experimentation, application delivery, and deployment. You'll work directly with customers and product teams, own technical calls, and see your product used at scale . We expect technical rigor, architecture discipline , strong product judgment, and a track record of shipping. In return, you get autonomy and real customer impact. What You’ll Do • Design, build, and operate systems in customer-facing production environments • Translate ambiguous business and customer problems into prototypes & technical specs • Track AI capabilities and apply them where they create clear leverage • Own the full application lifecycle: product d iscovery , experimentation, evaluation, deployment, and monitoring • Own system design, tradeoffs, and long-term scaling and maint ainability • Build code bases that are legible to agents and other developers ; drive the organization forward on tooling . • Partner with product managers to enhance their product vision , including with AI-native solutions What We’re Looking For • Strong applied AI and software engineering fundamentals • Builders who can span tech , product, and design thinking with high autonomy • Bias for shipping, iterating, and following customer feedback over polish • High ownership & agency — measured by outcomes, not deliverables • Curiosity to improve systems, products, and your own craft What We Work In You don't need every one of these, but this is the stack you'll be productive in: • P ython, C#, TypeScript , PostgreSQL ; • Frontier model ecosystems & agent frameworks (e.g., Anthropic, OpenAI) • Docker, Terraform, and AWS Minimum Qualifications • 2+ years in software engineering or applied AI • Experience working with AI development tools in f ull-stack applications • Experience designing AI-based solutions to real workflows • Working knowledge of cloud and frontier AI platforms Nice to Have • Degree in Computer Science, Engineering, Data Science, or a related field • Experience operating AI systems in production , with attention to evaluation, cost, and performance tradeoffs • Background in high-ambiguity environments with proximity to customers ( e.g ., early-stage startups, forward-deployed engineering, internal product teams) • Experience in large-scale Python, C#, and TypeScript codebases • Experience integrating AI solutions into existing, established products. • Experience working with healthcare, regulated, or sensitive data Job to be performed in the location listed. Generous benefit package available. © 2013 to 2026 Direct Supply, Inc. All rights reserved.

Apply at Direct Supply

More ai engineer roles