themarceloribeiro.com

Marcelo Sousa Ribeiro.

AI Engineer / Forward Deployed Engineer · 15+ years full-stack — San Francisco, CA, US · Authorized to work in the US

Engineer with 15+ years turning ambiguous business problems into production systems: from running a 15-person client-services firm, to scaling platforms at Deliveroo and BarkBox, to deploying into hotel chains like Wyndham and IHG. For the past year I've been building my own agentic platform (FlooxAI) and open-source LLM tooling (Evalstone, Turnstile), deploying LLM agents into real business tools: CRMs, email, and third-party MCP servers. I gate LLM features on evals: golden sets, LLM judges calibrated against human labels, and CI regression gates. Python, TypeScript, Go · FastAPI, Next.js · AWS.

01Where I've worked

Experience

Senior AI Founding Engineer — FlooxAI

Oct 2025 — Present

San Francisco, CA

My own venture: an agentic AI platform that connects LLMs to the business tools people already use (HubSpot, Salesforce, Gmail) and to third-party MCP servers, so agents can carry out real multi-step work. I set the product direction and built the platform end to end.

  • Built the agent runtime connecting LLMs to 35+ business connectors and 100+ tools via LangChain, Composio, and third-party MCP servers.
  • Multi-model routing across Claude, GPT, Gemini, and Llama via OpenRouter, with RAG over pgvector and Qdrant.
  • Built and deployed Turnstile (open source) in front of the platform's agent traffic, cutting AI spend ~35% through per-session budgets and runaway-loop detection.
  • Designed credit-based agent workflows for multi-step tasks: analyzing analytics and graphical data, preparing events, and drafting OKRs.

Senior Software Engineer — Canary Technologies

Mar 2024 — Oct 2025

Remote

Deployed Canary's guest-journey products into major hotel chains' existing infrastructure — mobile room keys and the in-hotel digital compendium (upgrades, room service, amenities) — live across 4,000+ hotels.

  • Partnered with Wyndham and IHG technical teams to deploy Apple Wallet / Google Wallet mobile room keys into their existing lock infrastructure, integrating Apple PassKit with the Vostio and Dormakaba lock SDKs.
  • Owned the upgrades and room-service flows of the compendium across frontend and backend, including Stripe payments (Python, TypeScript).
  • Built the tipping and feedback service, integrated with Google and TripAdvisor.

Lead Full-Stack Engineer — TPXImpact

Jul 2022 — Feb 2024

Remote

Worked directly with UK local-government stakeholders to deliver tenant, benefits, and payment systems for council housing — a regulated environment built to GOV.UK standards and accessibility requirements (Ruby on Rails, JavaScript, AWS).

  • Led a team of 3; gathered requirements from council staff and turned them into council-management tools tenants and officers actually used.
  • Built systems for benefits and payment collection from council homes.
  • Brought UX and UI to GOV.UK accessibility standards, driving meaningful gains in tool adoption.
  • Planned and executed deployment strategies that streamlined production releases and cut downtime.

Senior Software Engineer — Deliveroo

Jun 2020 — Jul 2022

Remote

Worked across the platform for the UK's leading food-delivery service, focused on database performance and a large-scale, multi-market sharding effort (Ruby on Rails, PostgreSQL, AWS).

  • Drove the zero-downtime sharding of the monolithic PostgreSQL database — migrated to UUID keys and split it into 20+ shards across market areas (UK, EU, Emirates) to absorb major growth in traffic.
  • Cut the top-10 slowest queries from ~5s to ~350ms average through denormalization and index improvements.

Senior Software Engineer — BarkBox

Oct 2017 — Jun 2020

Remote

Led the payments team (4 engineers, 1 QA) and was a key developer in warehouse management and fulfillment for a subscription business that grew past 2 million subscribers (Ruby on Rails, JavaScript, TypeScript, Vue.js, AWS).

  • Owned payments end to end for 1.5M active subscriptions — ~700K recurring charges per month across recurring and gift subscriptions, gift cards, and multiple recurrence models, integrated with Recurly.
  • Built the assignment logic determining which products shipped for each subscription.
  • Built warehouse-management and fulfillment features, integrating Pitney Bowes.

Lead Software Engineer — CrystalCommerce

Oct 2015 — Oct 2017

Remote

Led a team of 6 on a high-traffic marketplace for the game and hobby industry — 1,200+ online stores, ~2M orders/month, 5M monthly active users.

  • Built an Elasticsearch cluster with geocode search over hundreds of millions of records, cutting unified search from 20+ seconds to ~1 second.
  • Led a datacenter migration of 40+ servers/instances — DNS, PostgreSQL, MySQL, EC2, and S3.

Founder & Lead Engineer — RubyThree Information Services

Feb 2010 — Oct 2015

Dublin, CA

Founded and ran a client-services software house, growing it to 15 engineers delivering Ruby on Rails APIs, JavaScript frontends (React, Angular, Ember.js), and native iOS and Android apps — owning each client engagement from the first conversation to launch.

  • Ran end-to-end client engagements across industries: discovery, scoping, estimates and proposals, then delivery and post-launch support.
  • Owned the client relationships — translating ambiguous business asks into roadmaps, negotiating scope changes, and being the technical point of contact for stakeholders.
02Public work

Open Source

Evalstone — open-source evals for LLM apps

github.com/marceloribeiro/evalstone

A PyPI library + CLI (MIT) that makes evals a CI gate, a self-hostable center (FastAPI, Next.js, PostgreSQL) that calibrates LLM judges against human labels, and Evals 101, a free course.

  • Deterministic, RAG, safety, pairwise and trajectory checks; CLI, pytest, GitHub Action, MCP.
  • Judges promoted only after calibration (Cohen's kappa); production monitors with alerts.

Turnstile — session-layer circuit breaker for LLM / agent traffic

github.com/marceloribeiro/turnstile

A transparent streaming proxy between an app and any LLM provider that meters every agent session in real time and trips a circuit breaker the instant one session goes rogue (a runaway agent loop or a blown budget) — without taking down the rest of the app. Drop-in (swap one base URL, no SDK changes), session-precise, fail-open. Running in production at FlooxAI (my own venture), where it cut AI spend ~35%.

  • Go data plane (stdlib only): streaming proxy, provider adapters for OpenAI, Anthropic, Gemini, and OpenRouter, session resolution, metering/pricing, and synchronous pre-execution enforcement.
  • ~6 µs median added latency, self-instrumented, with a CI latency gate that fails the build on regression.
  • FastAPI control plane and a live Next.js dashboard; raw prompts and API keys are never persisted.
03Patterns, each backed by evidence above

Deployment patterns I've shipped

/Agents in production

Agents acting on real business data

LLM tool-calling wired to real service operations with a context-injected user and DB session — the model acts on real records, never raw SQL or secrets (FlooxAI; the "Ask my resume" agent on this site).

Connector & MCP integration

35+ business connectors (HubSpot, Salesforce, Gmail, Slack) and 100+ tools via LangChain, Composio, and third-party MCP servers, enabled per deployment from a DB-backed registry.

LLM cost & safety controls

Per-session budgets, runaway-loop detection, and a fail-open circuit breaker in front of production agent traffic — Turnstile, open source, running at FlooxAI.

Observability

Per-session metering and live telemetry, Langfuse / LangSmith tracing, and a CI latency gate that fails the build on regression.

Multi-model routing & RAG

Claude, GPT, Gemini, and Llama behind one interface via OpenRouter, with retrieval over pgvector and Qdrant — chosen per workload for cost and latency.

Streaming agent UX

Replies and the agent's narrated thought process stream token-by-token over WebSocket / SSE — the "Ask AI" dock on this site is a live example.

/Evals & LLM quality

Golden sets & checks

Versioned golden datasets harvested from real traffic, scored by deterministic, RAG, safety, pairwise and agent-trajectory checks — the Evalstone library.

LLM-as-judge you can trust

Specific rubrics and small scales, verdicts cached by judge version, and every judge calibrated against human labels (agreement, Cohen's kappa) before it is promoted — never trusted by default.

Regression gates in CI

Suites that run on every PR and fail the build on a quality regression (pass rate, mean score, cost, error rate) — CLI, pytest plugin, GitHub Action.

Evals in production

Sampled production traces scored against a check catalog, with windowed alert policies and webhooks when quality drops.

Human labeling at speed

Keyboard-driven grade / compare / verify queues with assignment, feeding both golden sets and judge calibration.

Teaching the practice

Wrote Evals 101, a free 10-chapter course on doing evals well (golden sets, kinds of checks, judges, CI gates, production, pitfalls) at evalstone.com.

/Enterprise integration

Deploying into someone else's stack

Apple / Google Wallet mobile keys into Wyndham and IHG's existing lock infrastructure (Vostio, Dormakaba SDKs, Apple PassKit), live across 4,000+ hotels.

Payments & billing at scale

Recurly-backed recurring billing for 1.5M subscriptions (~700K charges/month) at BarkBox; Stripe checkout, saved cards, and subscriptions elsewhere.

Regulated, public-sector delivery

Tenant, benefits, and payment-collection systems for UK councils, built to GOV.UK and accessibility standards with council stakeholders.

Logistics & fulfillment

Warehouse-management and fulfillment flows integrated with Pitney Bowes; assignment logic deciding what ships for each subscription.

/Full-stack delivery

FastAPI services

Production APIs with JWT auth, SQLAlchemy 2 + Alembic migrations, PostgreSQL, REST + WebSocket, and Celery + Redis for background and async LLM work.

Next.js web apps, typed end to end

App Router + React 19, TypeScript, Tailwind, shadcn; domain types mirror the backend models so the whole stack stays honest.

React Native mobile

Cross-platform apps with HTTP + WebSocket + FCM push, deep links, and App Store / Play Store releases via fastlane.

Rapid prototyping

Discovery to working demo in days: this site (API, agent, PDF, deploy) is one example; client proofs-of-concept at RubyThree were the routine.

/Scale & operate

Database scaling

Zero-downtime sharding of a monolithic PostgreSQL database into 20+ shards at Deliveroo; query tuning from ~5s to ~350ms.

Search at scale

Elasticsearch with geocode search over hundreds of millions of records, cutting unified search from 20+ seconds to ~1 second.

Domain → production, end to end

Register a domain, deploy API + frontend to a server, wire DNS, and serve over HTTPS — automated. (This very site shipped that way.)

Operations & reliability

CI/CD, health checks, staging/rollback, backups, Sentry, security headers, rate limits, nginx + systemd, Let's Encrypt TLS, and Playwright end-to-end suites.

04Toolkit

Skills

AI / LLM

Agentic systemsLLM orchestration & tool-calling (LangChain, LangGraph, Composio, MCP)OpenAI APIAnthropic API (Claude)GeminiMulti-model routing (OpenRouter)RAG / embeddingspgvectorQdrantPrompt engineeringLLM evals & judge calibrationLLM observability (Langfuse, LangSmith)LLM cost & latency controls

Customer & delivery

Technical discovery & scopingStakeholder communicationEnterprise integrationRapid prototypingClient delivery & team leadershipPublic speaking & demos (YouTube)

Languages

PythonTypeScriptJavaScriptRubyGo

Frameworks

FastAPISQLAlchemyNext.jsReactReact NativeTailwind CSSRuby on RailsVue.js

Architecture & Data

PostgreSQLRedisElasticsearchCeleryWebSocketsMicroservicesAPI designScaling

Cloud & DevOps

AWSS3DockerDocker ComposeGitHub Actions / CI/CDnginxsystemdPlaywright
05Education

Education

University of São Paulo

BSc, Physics with Scientific Software Programming · São Paulo, Brazil

2000 — 2005
06Communication

vibeworkin

Where I explain and demo AI tooling to a public audience — my YouTube channel on AI tools for productivity. Latest videos below.

Watch on YouTube
07Get in touch

Let's talk