Hiring an AI dev shop is closer to hiring a co-founder than buying a service. You're handing over a chunk of your product, your data, and weeks of runway before you know if it works out. Here's what separates a good hire from an expensive mistake.
Start with the portfolio, not the pitch
Any shop can put "AI Development" in their nav bar. Ask for two or three case studies close to your actual problem — same modality (text, vision, voice), similar data volume, similar accuracy or latency bar. Generic chatbot demos don't tell you much if you need a computer vision pipeline for a factory floor.
Then push past the case study copy: ask who on the team actually built it, whether that person is still there, and whether you can talk to the reference client directly — not a quote pulled from their site, an actual short call. A shop that hesitates to connect you with a past client is telling you something.
Test technical depth with a paid pilot, not a proposal
Proposals are marketing documents. The real signal comes from watching a team write code against your problem. Structure a small, scoped, paid pilot — two to four weeks, one well-defined slice of the real project against your actual data — before signing anything larger.
During the pilot, watch for:
- How they handle ambiguity. Do they ask sharp questions about edge cases and what "good enough" means, or just start building?
- Whether they benchmark before they build. A good team flags when a simpler approach — a tuned prompt, a smaller model, a rules-based fallback — beats a heavier custom model, instead of defaulting to the most impressive-sounding architecture.
- Code and documentation quality, not just a working demo. Ask for repo access during the pilot.
- How they talk about limitations. Teams upfront about where a model will fail are more trustworthy than ones that only talk about upside.
Nail down IP ownership before day one
This belongs in the contract before any code is written. Confirm in writing:
- You own all code, model weights, and fine-tuned artifacts produced for you — not a license to use them.
- Any pre-existing frameworks the shop reuses across clients are disclosed and licensed to you, not silently baked into your deliverable as lock-in.
- Your data, including anything used for fine-tuning, stays yours, with a clear deletion policy once the engagement ends.
- If they build on their own proprietary agent framework, understand what happens if you leave — can another team maintain it, or are you locked in?
Staffing and turnover risk
Ask directly: who is assigned to my project, and what happens if they leave mid-engagement? Boutique shops (under roughly 20 people) carry more key-person risk — your project may ride on two or three engineers. Not automatically bad, but know it going in.
For larger shops, ask the opposite question: will you get the senior team from the sales call, or will work get handed to a junior bench after signing? Get names and seniority levels written into the statement of work, not just "a dedicated team."
Communication cadence
Set this explicitly before work starts: how often you get a working build (weekly demos beat monthly status decks), expected response time on blocking questions, and who your single point of contact is. AI projects carry more inherent uncertainty than typical software work — models underperform, data is messier than expected, timelines shift. A shop that surfaces that early is easier to work with than one that goes quiet and delivers a surprise at the end.
Red flags in proposals
- Vague scoping with a big price tag. If a proposal can't define what "done" looks like — specific metrics, specific deliverables — it's a placeholder, not a plan.
- No mention of evaluation. A serious engagement defines how you'll measure whether the system works — accuracy, latency, cost per inference — before building starts.
- Guaranteed outcomes. Nobody can promise a specific accuracy number before seeing your data. Confident guarantees at the proposal stage are a sales tactic, not an engineering commitment.
- Unwillingness to do a paid pilot. A large upfront ask with no smaller proof step means you're underwriting their learning curve.
- No discussion of ongoing costs. Inference, hosting, and monitoring costs don't stop at launch — a shop that prices only the build and waves off run-cost questions leaves you to find out the hard way.
A practical checklist before you sign
- Two to three relevant case studies, verified with a real reference call.
- A scoped, paid pilot against your actual data before any larger commitment.
- IP ownership and data handling terms in writing.
- Named team members and what happens if they leave.
- An agreed evaluation method and definition of "done."
- A weekly demo cadence during active build.
- Clarity on post-launch ownership — maintenance, monitoring, and fixing model drift.
Shops vary widely in size and specialty — a small boutique like asaasin.ai and a several-hundred-person firm are both legitimate ways to get AI work built, but they carry different risk profiles. Match the shop to your project's scope, verify claims with a paid pilot instead of a slide deck, and get IP and staffing terms in writing before real work starts.