Recommendations you can check.
Machine learning engineer and independent AI consultant; writes about LLM evaluation and tooling at hamel.dev.
10 things
HexService
You might be skeptical of using synthetic data. After all, it’s not real data, so how can it be a good proxy? In my experience, it works surprisingly well. Some of my favorite AI products, like Hex use synthetic data to power their evals
His long guide to building an LLM-as-judge, including how he uses synthetic data.
2024-10-29·hamel.dev ↗
Next.jsSoftware Vercel
I now use different languages for different tasks. For web development, I prefer the Next.js stack.
'Why I Stopped Using nbdev' — on how AI coding tools changed which development tools he picks.
2026-01-18·hamel.dev ↗
Arize PhoenixSoftware
The panel had a generally positive view of Phoenix, with one panelist calling it one of his “favorite open source eval tools.” The tool is positioned as a developer-first, notebook-centric platform.
A write-up of a live panel review of the main LLM-eval vendors, with his own criteria and per-vendor notes.
2025-10-01·hamel.dev ↗
AmpSoftware Sourcegraph
My daily drivers now include Amp, Cursor, and Claude Code. I still enjoy notebooks, but only for data analysis, machine learning, or other exploratory workflows where the iterative, visual nature of notebooks shines.
'Why I Stopped Using nbdev' — on how AI coding tools changed which development tools he picks.
2026-01-18·hamel.dev ↗
CursorSoftware Anysphere
My daily drivers now include Amp, Cursor, and Claude Code. I still enjoy notebooks, but only for data analysis, machine learning, or other exploratory workflows where the iterative, visual nature of notebooks shines.
'Why I Stopped Using nbdev' — on how AI coding tools changed which development tools he picks.
2026-01-18·hamel.dev ↗
Claude CodeSoftware Anthropic
My daily drivers now include Amp, Cursor, and Claude Code. I still enjoy notebooks, but only for data analysis, machine learning, or other exploratory workflows where the iterative, visual nature of notebooks shines.
'Why I Stopped Using nbdev' — on how AI coding tools changed which development tools he picks.
2026-01-18·hamel.dev ↗
Jupyter notebooksSoftware
As for me personally, I tend to use these tools as a backend data store and use Jupyter notebooks as well as my own custom built annotation interfaces for most of my needs.
A write-up of a live panel review of the main LLM-eval vendors, with his own criteria and per-vendor notes.
2025-10-01·hamel.dev ↗
nbdevSoftware
Because I have invested thousands of hours into nbdev, it's difficult to admit there are better tools for the outcomes I want.
'Why I Stopped Using nbdev' — on how AI coding tools changed which development tools he picks.
2026-01-18·hamel.dev ↗
LangsmithService LangChain
The overall workflow is intuitive, especially for those new to formal evaluation processes. The UI guides you through creating datasets, running experiments, and annotating results.
A write-up of a live panel review of the main LLM-eval vendors, with his own criteria and per-vendor notes.
2025-10-01·hamel.dev ↗
BraintrustService
The panel had a generally positive view of Braintrust, highlighting its clean UI and structured approach to evaluations. The tool’s emphasis on human-in-the-loop workflows was a significant strength.
A write-up of a live panel review of the main LLM-eval vendors, with his own criteria and per-vendor notes.
2025-10-01·hamel.dev ↗
Every entry is a verbatim quote and a link to the public post it came from. Nothing is paraphrased. If it is not on record somewhere public, it is not here.
The verdict is what they actually said: loved and liked are explicit; recommended means they told others to get it; uses means they only say they use it; read is a book on their public shelf with no verdict; mixed and disliked are kept too.
A dashed affiliate link, sponsored or their own label means the post disclosed a relationship. It is shown, never hidden.
The shading behind a date shows how dated a recommendation is for its kind of thing.
The shading behind a date shows how dated a recommendation is, for its kind of thing. Fully green is recent. It drains a quarter at a time as the post gets older, and is empty once it reads as dated.
What counts as old depends on the thing. A laptop praised four years ago has been replaced twice; a book loved twenty years ago is probably still loved. The shading is gone after about:
Stripes mean the post carries no date and the date shown is an estimate. No date at all, no shading.