Inside the Loop with Mercor: RL Environments, Benchmarks, and Evals

Inside the Loop: Evals → Real-World Data → RL Environments → Better Models A Meet & Greet with Engineering Leadership at Mercor NY 📅 Tuesday, October 20 · 7:00–9:00 PM The way frontier models get better has changed. Scaling pre-training used to be the main driver of progress. Now a lot of the gains come from post-training: rigorous evals, high-quality real-world data, and RL environments that test models on the work people do. Mercor sits inside that loop. We work with leading AI labs, so we get an early look at where models are falling short and what the next generation needs to learn. Join engineering leaders from Mercor and Deeptune for a candid fireside chat on the flywheel we’re building and what it shows about where AI is going. What we’ll cover How model improvement has changed: why the frontier has moved from “more compute” to “better signal” Why post-training is critical: how evals and real-world data turn into measurable gains in model capability Building RL environments at scale: the engineering problems behind simulating real work for models to learn from Seeing the future early: what working with frontier labs every day tells us about what comes next Speakers Timothy Marks, Engineering Lead, Applied AI, Mercor (Moderator) Arty Vohmincevs, VP of Engineering, Mercor Kuai Yu, Head of Data Acquisition Engineering, Mercor Tim Lupo, CEO, Deeptune Shweta Sharma, Eval Platform Lead, Mercor Dylan Pourkay, Tech Lead Manager Aksh Garg, Head of Research Engineering, Mercor Agenda 7:00 PM: Doors open, food and drinks 7:30 PM: Welcome 7:35 PM: Fireside chat and audience Q&A (25 min) 8:00 PM: Open floor. Meet the speakers and the engineers building this work. 9:00 PM: Close Who should come: Engineers and technical leaders in NYC interested in working on the hardest problems in AI, like evals, data infrastructure, RL and applied AI, and who want to hear how frontier models are being built right now. Seating is limited and every RSVP is reviewed. We’re building our…