AI Engineer · Project 6

Your AI Startup, in 20 Days

Pick a real problem. Build the AI product. Get 5 real users. Ship.

Duration: ~20 daysDeliverable: A live, public AI product with ≥5 real users. Public GitHub. Public eval scorecard. Tech-blog post + case study + Loom on your portfolio site.

Your capstone is the AI startup you've been thinking about — built and shipped. A real product solving a real problem for ≥5 real users (not classmates, not friends doing you a favor). Integrates everything from Projects 1–5. Public GitHub. Live URL. A public eval scorecard. A tech-blog write-up of one architecture decision. The portfolio centerpiece every other artifact points back to — and the foundation that's already in production if you decide to keep going commercially.

The product in detail

The capstone is the AI product idea you've been carrying around for months — now built and shipped. A real product solving a real problem for real users who aren't doing you a favor. Public live URL with auth. Public eval scorecard. Public tech-blog write-up of one architecture decision. A 5-minute Loom that lands the impact in the first 30 seconds. A README that reads like senior-engineer documentation. This is the portfolio centerpiece that every other project artifact points back to.

The user persona is whoever your specific product idea targets. The constraint is that they must be real. Not classmates, not your mom. Real users you reached via cold outreach, Reddit, Hacker News, Twitter, a Discord community, or a tight-network ask. Onboarded with a working product. Provided feedback you actually acted on. Five of them, each using the product 3+ times. The north-star metric depends on your product — weekly active queries, task-completion rate, PR-author thumbs-up rate, or first-paid-customer for a micro-SaaS.

This is the project that converts a portfolio into job offers. Every Anthropic, Mistral, and Notion AI Engineer JD has the same line buried in it: "Production experience building LLM-powered applications." Many candidates have built side projects. Few have built side projects with real users, a public eval scorecard, and a published architecture write-up. Anthropic's own careers page says: "If you have done interesting independent research, written an insightful blog post, or made substantial contributions to open-source software, put that at the TOP of your resume."

You integrate every skill from Projects 1-5: RAG or advanced RAG (Project 1 or 4), an agent layer if relevant (Project 2), eval and observability instrumentation (Project 3), production hardening with guardrails, caching, and routing (Project 5). Cost-per-user-per-day must stay stable across a 7-day window. At least one feature ships because a user asked for it. At least one feature gets deprecated because users ignored it. This iteration loop — ship, measure, learn, ship — is the discipline that separates AI Engineers from AI experimenters.

What you ship: a live URL on your portfolio site. A 5-minute Loom that lands the impact in the first 30 seconds, then walks through the architecture. A one-page case study (problem, solution, metrics, lessons learned). A published tech-blog post about one architecture decision — with the data behind it. And a working product that continues to exist after the curriculum ends, that you can choose to keep operating, iterate on, or turn into a real business if the traction warrants it.

Pick what you'll build

This is one project with one skill graph — but you pick the product framing that fits your interests or career goals. Each option below is a viable portfolio piece. We'll narrow it together in the kickoff call.

  • Code review agent — deployed as a GitHub App reviewing real OSS PRs. North-star metric: PR-author thumbs-up rate.
  • Personal knowledge agent — your reading library + 4 friends' libraries, queryable via web + iMessage. North-star metric: weekly active queries.
  • Domain research assistant — RAG + agent over a specific public corpus (immigration paperwork for one country, legal precedent for a niche, school board minutes). North-star metric: real-user task-completion.
  • Internal-tool LLM — pitched at a real company in your network, solving a specific workflow. Free in exchange for usage data. North-star: usage retention.
  • Real micro-SaaS — narrow vertical, paid. "AI that drafts insurance-claim appeals," "AI that summarizes school board minutes," "AI that prepares immigration paperwork." North-star: first paying customer.
Why this project
  • The integration project.
  • The portfolio centerpiece.
  • AI Engineer is fundamentally about shipping production LLM products — pick any of the 46 posts: 'production experience building LLM-powered applications' (Anthropic), 'Productionize and scale' (Notion), 'Deploy LLMs into production' (MoneyLion).
  • Mentees who complete this have the artifacts (live URL, eval scorecard, blog post, Loom) that distinguish them in AI Engineer interviews.

Research backing this project

Every project on projeckt.ai is calibrated against real job posts and real interview questions, not opinion. Here's the data behind this one.

46
Job posts citing concepts in this project
77
Interview questions this project prepares you for
3
Tacit must-knows drilled — concepts interviewers test but JDs rarely list
Tacit must-knows you'll drill
  • Integration of all prior must-knows
  • Real-user product sense
  • Portfolio-grade write-ups

Themes covered

Each chip shows how many of the 46 analysed job posts cite this theme.

LLM & Prompt Foundations · 43 postsAgents & Tool Use · 31 postsRetrieval & Knowledge Systems · 27 postsEvals, Observability & Safety · 24 postsProduction AI Engineering · 9 posts

Justified by real job posts

Verbatim quotes from the postings that drove the design of this project.

Production experience building LLM-powered applications (prompting, context engineering, agents, evaluation frameworks, deployment)

Anthropic — Applied AI Engineer, London

What matters is what you ship, not the time you spend on it

Mistral — Applied AI Forward Deployed ML Engineer

If you have done interesting independent research, written an insightful blog post, or made substantial contributions to open-source software, put that at the TOP of your resume.

Anthropic — Careers page

Builder credibility with technical founders / engineering teams

Recurring requirement across 8+ posts

What you'll be able to do

  • 1Pitch and validate an AI product idea with a real user, not a classroom audience
  • 2Ship a vertical slice fast — first real user onboarded within 5 days
  • 3Onboard 5+ real users with a working live URL and an iteration channel
  • 4Run Project 3's eval harness on live traffic with a public scorecard page
  • 5Iterate based on user feedback: ship 1+ feature because users asked, deprecate 1+ because users ignored it
  • 6Integrate Project 5's production hardening (guardrails, caching, routing) into the live service
  • 7Keep cost-per-user-per-day stable across a 7-day window
  • 8Draft a tech-blog post about an architecture decision with the data backing the call
  • 9Produce a one-page case study (problem, solution, metrics, lessons) for the portfolio
  • 10Demo the product in a 5-minute Loom that lands the impact in 30 seconds

Tools you'll learn

Hands-on with these tools across the paid and open-source stacks. The Paid / OSS / Both label shows which stack each tool belongs to.

BR
Braintrust (production evals)
Paid stack
HE
Helicone (live LLM ops)
Paid stack
LangSmith Live
LangSmith Live
Paid stack
LL
LlamaParse (document parsing)
Paid stack
VO
Voyage AI (domain embeddings)
Paid stack
CA
+ carry from P1–P5
Both stacks

Week-by-week checkpoints

Each checkpoint is a Pull Request to your public GitHub repo. Rohan reviews on the PR; merge = checkpoint complete.

CP1: Pitch + MVP + first real user
Days 1–5
What ships

Product idea pitched, validated with Rohan. Minimal vertical slice deployed. First real external user onboarded with feedback received.

Acceptance criteria

1 real external user has used the product and given feedback (recorded).

New skills
  • Product discovery for AI products
  • User-acquisition for tiny apps
CP2: 5 users + live eval harness
Days 6–10
What ships

5+ real users onboarded. Eval harness from P3 running on live traffic. Public scorecard page.

Acceptance criteria

Scorecard updates daily; 5 real users each used the product 3+ times.

New skills
  • User onboarding flows for AI products
  • Live-traffic eval
CP3: User-driven iteration + production hardening
Days 11–16
What ships

At least 1 feature added because a user asked; at least 1 deprecated. P5's guardrails / caching / routing integrated. Cost-per-user-per-day stable for 7 days.

Acceptance criteria

Cost/user/day stable across a 7-day window; iteration documented in 1 blog post draft.

New skills
  • User-feedback synthesis
  • Cost discipline for AI products
  • Production iteration
CP4: Tech-blog + case-study + Loom demo
Days 17–20
What ships

Published tech-blog post (architecture decision write-up). One-page case study. 5-min walkthrough Loom. Live URL on portfolio site.

Acceptance criteria

All 4 artifacts public; capstone passes a peer-review by another mentor.

New skills
  • Technical writing for AI eng portfolio
  • Portfolio-grade demo videos
  • Case-study craft

Pick your stack

Same skills, different credentials. Start free with the OSS stack or use the paid stack to learn directly on production tools.

ToolPurposeLicenseHardwareMarket equivalent
Carry-over from P1–P5 OSS stacksYour existing servicesOSSYour VPS
Plausible Analytics (self-hosted) or UmamiPrivacy-respecting analyticsAGPL / MITDockerPostHog / Mixpanel

Estimated cost: $5–10/month (VPS only).

Interview questions you'll be ready for

Real questions from 13 sources, mapped to the skills this project builds.

Walk me through a project you led end-to-end.
Source: Universal interview opener
Prepares you for: Your capstone is exactly this story — pitch, build, ship, iterate.
What's the hardest architectural decision you made on this project? Why?
Source: Senior engineer staple
Prepares you for: Your blog post is the rehearsed answer — with the data behind it.
How did you measure success?
Source: Universal across AI Engineer interviews
Prepares you for: Your eval scorecard answers this.
What would you do differently if you started again?
Source: Senior interview staple
Prepares you for: Your iteration log informs an honest, specific answer.
How did users react to a feature? What did you change as a result?
Source: Product-engineering interview question
Prepares you for: CP3's documented iteration IS this answer.

Ready to start building?

Book a free 30-minute project planning session. We'll tailor this project to your stack, team, and goals.