MEL MLOps Community #19 | Advanced Evals & Deep Agent Harnesses
About the event
We are stoked to announce our next Melbourne MLOps Community Meetup of the year!
We’re gathering at Mantel to unpack the realities of building reliable, production-ready AI agents. Join two top-tier industry experts to learn how to move beyond basic evaluations and build the structural harnesses that turn fragile demos into real-world tools.
🎤 Our speakers and topics:
🧪🎯 The Lesser of Two Evals: Why Your Agent Passes Benchmarks and Fails Users Josh Gillies, Senior Software Engineer @ Prefactor Traditional LLM evaluation metrics (correctness, groundedness, and hallucination checks) are necessary starting points, but they aren't enough to guarantee a great agent. In production, agents encounter unexpected multi-turn branching, bad conversations, and tools which can fail in a myriad of ways. In this talk, we’ll explore evaluations beyond using a single-prompt LLM as a judge and look at Agent quality as a holistic system.
🏗️🧠 Your Agent Needs a Harness: How deep agents actually work Lisa Sherin, Founding Deployed Engineer @ Langchain Most agent demos work because someone is watching. The gap between a demo and something you'd let run unattended isn't the model, it's the harness around it: how you structure the loop, hand it tools, manage context, and recover when a step goes wrong. This session pulls apart the agent harness pattern and how Deep Agents use it to plan, delegate, and stay on task over long-running work.
Networking
The Melbourne MLOps Community is committed to fostering a space where AI/ML engineers and practitioners can connect, exchange ideas, and build meaningful relationships.
Come ready to discuss:
• The tools you’re experimenting with • Trends you’re seeing in ML and MLOps • Challenges you’re tackling • Ways to find peer support
We’ll have food, drinks, and the perfect space for great conversations — so feel free to bring along anyone interested in ML, MLOps, or AI more broadly.
Agenda
🍕 5:30pm – Networking, drinks & food 🙋 6:00pm – Welcome 🎤 6:10pm – Presentation by Josh ❓ 6:30pm – Questions for Josh 🎤 6:40pm – Presentation by Lisa ❓ 7:00pm – Questions for Lisa 🍻 7:10pm onwards – Networking, drinks & food We’re looking forward to seeing you there!
P.S. If you can’t make it in person, we’ll be streaming the event via Google Meet. Google Meet link: [https://meet.google.com/fxh-hpsg-ryt](https://meet.google.com/fxh-hpsg-ryt)
