Working with Claude — Project Portfolio

A summary of work built using Claude (Cowork / Claude Code), compiled from 38 local sessions.

Overview

Over around six months, I worked across key engineering initiatives: I built DatasetRadar, a complete data-intelligence product from technical spec through to a running instance with a full review UI; I architected the LangGraph Playground monorepo for developing and deploying stateful multi-actor agent workflows; I engineered a production automation that watches for pipeline failures on Factly’s public-data repo and opens fix PRs unattended; and I authored a modular ETL skill suite and tooling investigations.

DatasetRadar — Dataset Intelligence Platform

I built DatasetRadar to keep an eye on news and journal websites and automatically spot which public datasets, surveys, or reports each new article mentions — no manual tagging required. The payoff: a live, searchable map of who’s citing which dataset, where, and how often.

Planning and architecture. Before writing any code, I did a full design walkthrough of the app end to end, then wrote a detailed technical architecture document covering the database design, the API, the crawling rules, and a short list of open questions I flagged for engineering sign-off.

The finished product. I shipped a complete, working app, not a demo: a React frontend talking to a FastAPI backend, background workers that scrape sites on a schedule, and a database with vector search built in for finding related datasets. I set up a self-hosted AI model that reads every article and figures out which datasets it’s talking about — even when the wording is different — so the same dataset never gets counted twice under two different names.

Human-in-the-loop by design. I designed it so not every match is treated the same: confident matches get accepted automatically, unsure ones wait in a review queue for a person to check (with the option to ask the AI for a second opinion), and weak guesses get dropped entirely. I built the app to cover a trending-datasets dashboard, an exportable usage report, per-site health monitoring, a simple screen for managing which sites get watched, and a master list of all known datasets.

Ask — retrieval-augmented Q&A. I added an Ask feature: type a plain-English question and get back a cited answer pulled straight from the scraped articles — no need to know a dataset’s exact name. Because I run the AI model locally, there’s no per-question cost and no data ever leaves the system.

Deployment. I shipped it ready to run either way: a quick dev setup for local testing, and a production-ready setup with load balancing, resource limits, and a locked-down internal database.

Stack: FastAPI, SQLAlchemy 2.0, Alembic, Pydantic v2, Celery + Celery Beat, Redis, PostgreSQL 16 + pgvector, React 18, TanStack Query, Tailwind CSS, Playwright, LangChain, Ollama (nomic-embed-text embeddings), Docker Compose, Nginx.

View the product demo → — screenshots of a running instance, walked through slide by slide.

LangGraph Playground Monorepo — Stateful Multi-Actor Agents

I architected and implemented LangGraph Playground, a modular Python monorepo for experimenting with, engineering, and deploying stateful, multi-actor AI agent workflows using LangGraph.

Monorepo architecture with editable shared packages. I structured the codebase cleanly around modular shared packages and decoupled applications:

Ready-to-run pattern examples. I developed four end-to-end reference agent workflows:

Applications & developer experience. I built deployable surfaces including an HTTP API server scaffold (apps/api) for service invocation and an interactive terminal REPL agent runner (apps/cli), backed by a comprehensive 13-test integration suite, Jupyter experimentation notebooks, and Makefile shortcuts (make run-tools, make run-sql, make test).

Stack: Python, LangGraph, LangChain, SQLite, Pydantic, Make, Unit testing.

View GitHub Repository →

Dataset Pipeline Skill Suite (Factly Datasets ETL)

I built a modular set of 6 custom Claude Code skills that encode the entire ETL convention for the factly-datasets repository, so any dataset project — new or existing — gets built to the same directory structure, coding standards, and validation contract without me having to re-explain it each time.

dataset is the router I built: it resolves the repo root via git rev-parse, enforces that every path stays inside projects/, blocks direct writes on the develop branch (prompting to check out or create a feature branch first), and hands off to one of three flows depending on what’s needed.

dataset-create (Flow B) is what I use to scaffold a brand-new dataset project from scratch: it runs my team’s cookiecutter template, then generates a scraper (scrape_raw_data.py), an optional interim-cleaning step, and the final create_dataset.py — all with consistent path-resolution (Path(__file__).resolve().parents[N]), docstring and naming conventions, and incremental-update logic (concat, sort, dedupe on save). I built it to handle four source formats, including a genuinely detailed PDF/OCR pipeline I designed: dynamic page detection by header text (no hardcoded page numbers), a dual extraction path that routes text-rich pages through pdfplumber and image-only pages through an open-source OCR library (e.g. Tesseract), idempotency skips so re-runs don’t reprocess existing output, and indicator/category-name cleanup logic (footnote-digit stripping that’s careful not to mangle things like “by age 18”, plus OCR category canonicalization maps). It then calls my validation and workflow sub-skills, updates pyproject.toml, and runs poetry install plus each script with an automatic error-diagnose-and-fix retry loop I built in.

dataset-correct (Flow A) is the audit-and-repair counterpart I built: given an existing project, it checks for missing pipeline files, flags code issues (missing docstrings, single-letter variable names, wrong sort order, fiscal-year sort edge cases, missing append-then-sort logic), regenerates anything absent, and re-runs the same validation/workflow audits and script-fixing loop as dataset-create.

dataset-add-category (Flow C) is the same flow as Flow B, which I reused for adding a new sub-path/category inside a project that already exists, skipping the cookiecutter bootstrap.

dataset-validation is a skill I built to generate or audit the Pandera schema (rules.py) and validations/config.json for a project, then run valid validate-all with an error-fix loop I backed with a lookup table of common schema errors (dtype mismatches, nullable mismatches, incomplete isin= enums, duplicate-key issues) and their fixes.

dataset-workflow generates or audits the src/workflows/config.json that defines the cron schedule, script dependency graph, and validation step — the exact config file I have the Workflow Failure Resolver (below) read to know what to run and in what order.

I gave every skill the same safety scaffolding: abort keywords (stop/abort/cancel) checked on every turn, an interactive/non-interactive mode toggle, and checkpoints with “go back and re-run from step N” navigation, backed by an explicit dependency table of what needs regenerating. Together, the six skills I built form a self-consistent pipeline-generation system — that’s agent/skill engineering in its own right, not just prompting.

Automated Workflow Failure Resolver (Factly Datasets)

I built a production scheduled agent that monitors the “Workflow Notifications” Google Chat space for failed data-pipeline runs on the factly-datasets repository — which produces government/public datasets like CPCB air quality, NPCI, GST, RBI remittances, EPF, MNREGA, and similar releases — and autonomously fixes the ones on an approved whitelist.

I designed it to: read the latest failure notifications via Chrome browser automation, extract workflow names by regex, filter against a whitelist file (workflo-names.txt), skip anything already replied to, post an acknowledgment to the relevant Google Chat thread, create a git fix branch, run and validate the affected dataset script (falling back to my dataset-correct skill above when a script needs an actual fix), generate the rclone upload commands, then commit, push, and open a pull request against develop before checking back out.

I iterated on it repeatedly rather than building it once: I swapped Step 1 from screenshot-based reading to a direct DOM query via JavaScript for reliability, added a text-based fallback for detecting thread replies (Google Chat’s reply indicator isn’t a stable selector), added a safety check that halts the whole run if the local repo isn’t on the develop branch, and extended the final step to handle the full git → PR → checkout sequence automatically. I run it unattended on a Monday–Friday schedule, and I’ve watched it correctly identify runs with nothing in scope, or halt and ask for human input when Google Chat needed re-authentication or the sandbox couldn’t reach docs.google.com.

I also produced a small supporting toolkit in helper-functions/ (fiscal-year lookup, validation logging, Wasabi upload script) with its own README, and worked out a token-cost estimate for running the scheduler regularly (roughly $1–3/month quiet, $5–10/month during active failure periods).

Stack: Claude Code scheduled tasks, Claude in Chrome (Google Chat automation), git/GitHub CLI, rclone, Wasabi.