SPEAKER_01 0:10

Today I'm joined by David Windt, a founder and an operator who has built scale and sold companies and is now building again at the frontier of AI with his new venture, Agent Work. David previously founded PeerGrid and EdgyFlow, taking them through Y Combinator, growing them largely through product-led growth and word of mouth, and ultimately acting to Multiverse. Along the way, he's had a front row seat to what it really takes to scale a product, a team, and an organization, both inside a venture backpath and outside of it. In this conversation, we are going to talk about three big things. First, how to design a product that effectively scale themselves through reforms and self-serve. And David, I'm hearing that there's an interesting story about the Romanian church. So I would love to hear more about it. Second, what David has learned about sustainable scale and why he deliberately chose not to chase headcomb heavy hypergrowth. And third, how he's thinking about scaling with AI agents at agent work, what a digital workforce actually looks like in practice, and when founders should use agents instead of hiring more people. If you're building or scaling a company and you care about efficient growth, clear unit economics, and pragmatic ways to use AI in your operation, this episode is for you. Welcome, welcome, David. Thank you. So as you as you heard from the intro, the premise really is for us to get to know you, how you've scaled the previous two companies, and kind of like what are you doing differently this time around? So help us shape the thinking in terms of how you're building now versus how you were building five or ten years ago.

SPEAKER_00 1:58

It's a good question. I think in many ways, in the same way. Like we're trying to not change most of how we used to do things because I think we had a company that was running quite well in Noe. I think there's obviously some different things happening in the world right now with AI, and that impacts specifically like how we think about hiring and how we think about building. So the big difference, obviously, that most people have already observed is that what used to take us three months for a full-time engineer to build, you can fundamentally like build in an afternoon as a founder, which means that you can you don't need to hire a bunch of engineers first and foremost. You can you can get a lot done. And like engineering is not the bottleneck anymore. But it also means you can you can test things much faster. If you have a good idea, or at least you think it's a good idea, you can build it in an hour or two hours, and then you can put it in production and you can see if it actually works. And if it doesn't work, you can just delete it again. So it's much cheaper to make mistakes, which means if you can move fast, you can iterate extremely rapidly. But if you can't, then it's you're kind of at a disadvantage compared to your competitors who might be able to do that. So practically we hire fewer people, and on the other side, like we can ship product much faster than we used to, even with a much smaller team.

SPEAKER_01 3:23

So, besides uh writing code and building faster, what like what are your hacks? Kind of how are you doing things differently now than you were doing before? How do you get in front of customers? Um, like what other accelerants have you found that that work?

SPEAKER_00 3:44

It's a good question. And I actually think in some way that's becoming harder. Um, not because the modern AI stack for go-to-market and marketing and so on isn't getting better as well. But given that writing product isn't the bottleneck for most teams anymore, uh, there's a lot more products out there, all like going for the same attention of the same people. So like breaking through the attention span of a customer is actually getting more complicated. Anybody can like sign up for an automatic uh email, cold email sender in five minutes and then click a button and send a thousand cold emails to everybody, which means everybody's getting a lot more personalized emails. And when everybody's getting a hundred personalized emails, then they don't read any of them anymore. So I think there's like we're trying to do some of the same things that everybody else is doing, and are kind of obvious that, like, okay, we need to use all the modern tooling here, but but in the end, I think it's actually going to be much harder to get distribution for your products now that that everybody has all these AI tools.

SPEAKER_01 4:49

And so does this mean then we're going to see um a shift from digital to analog? Uh, are you seeing more like a transition back to kind of like I guess what what used to work in the past with uh like events and active labs and activations, like things that they're just kind of going back to like would we see a continuous rise in the third spaces where people are just congregating together?

SPEAKER_00 5:17

Yeah, yeah, I think we will see more physical things happening, and I think another thing that will probably become even more extreme now is that if you already have a following, if you're an influencer on some social media, that will be even more powerful because that is basically the biggest hack you can have right now is to have the customer list available right now. So I think the people who manage to grow an audience before or can figure out a way to do it now will have a big advantage when when anybody can build a product, right? You can just if you have the audience, you can just bycode an alternative product and sell it to your audience, which you've seen people do that in the past, right? Like Mr. Beast launching a burger chain or something, right? Like it doesn't make any sense, but if you have the audience, you can just uh like whip up any product, label it with you, and like ship it to your customers. And I think that's we're probably gonna see that entering more into the startup world as well. Like uh vibe code and alternative to obsessed product and then send it to your audience at a subscription price.

SPEAKER_01 6:20

So I mean our audience are scalar founders, that's who we write for, this this is who we serve. So if they're hearing this for the first time, and if they're not Jimmy um or Mr. Beast, how do they how do they go about building an audience? Because you know, there's this different growth strategy from product led growth to sales that grow to community led growth, um, and and obviously communities emerged to be the most powerful, but that's predicated upon you already having a community. So if you are in the early uh stages, how do you build a community?

SPEAKER_00 6:58

I think that's probably like I'm not the right person to ask because I'm not very good at it. I think it's been one of those things where I've always wished. I think like so, okay. I think I've always wished that like I wish I had a community. I wish I had a big uh following on X, but I think it's like very classical. I would like to be the person who has it, but I'm not actually the right person to build it, right? I'm not super excited about sitting and writing like thought leadership content on X or something. It doesn't like it's not what I wake up to do in the morning. I like doing other things, and I think it's super hard. It's equally hard to building another type of business. It's an art form, it's stand-up comedy on social media, right? It's it's tough. So I don't think I think the problem is with a thing like that, you can obviously like you don't have to be the world's best or the world's largest audience to have any value from this, but but it's like with many other things, if you do it half-assed, it doesn't give you any value. And if you do it really well, you get a ton of value. So there's some like power law going on here where somebody will say, like, oh, we should I should have an audience. They're gonna write a LinkedIn post uh a few times a week for three weeks, and then they're gonna give up because nobody cared. Yeah, and like it's it's not enough, right? You can't just click a button and get an audience, you have to do all the hard work. So I think it's for sure for most people, it's kind of like an either or like if you do it, do it well. If you if you don't do it well, then don't do it at all, probably.

SPEAKER_01 8:26

Well, and and uh it it like it's it's like everything in life, it takes time to master. And uh the other thing is that you have to be authentic to yourself because a lot of people go down, like I mean, there's a saying, what enrages, it engages, right? So a lot of people go down the clickbaity routes like being very provocative or uh uh controversial, and that's great for the moment, but but then it begs the question well, who are you as an influencer or as a scalar founder, whatever it might be.

SPEAKER_00 9:02

Uh but I read something or heard a podcast a long time ago that inspired me a bit, which was like it's not actually that hard, surprisingly, to make the world's best resource on something.

SPEAKER_02 9:13

Okay, right?

SPEAKER_00 9:14

It's like it can't be like the world's best book on negotiation, like that's a very big topic. Like a lot of people have worked on that before. But if you're like, no, but but I work in let's say car sales or whatever, then it's like I'll write the best thing about specifically selling electric cars or whatever, like you knees down enough so that there's not a ton of competition, and then write an incredibly good thing, or build a good thing, or make a good thing. It has to be the best in the world, yes, but if you knees it down enough, it's not actually that complicated. And you see a lot of technical blog posts where like some engineer somewhere is writing this like very detailed blog post about this very niche technical subject. Maybe a hundred people read it, maybe a thousand people read it. But for them, that is like the the best thing in the world, and I think that's it's actually easier to make the best in the world type of content on a niche subject than trying to be like LinkedIn here and a little bit of X there and so on. Like it doesn't nobody cares if it's not the best. So I think that'd be my recommendation like find something that's small enough that you can write the best thing in the world, yeah, and then spend the time it takes to make that thing.

SPEAKER_01 10:24

For sure. I couldn't agree more um with my with my digital and content team in Buenos Aires. We that's one of our core focus. We we want to produce content that uh Scala Founders found useful. And so we one of the articles or in-depth guides that we published was on forward-deployed engineering model, and so the FDE model got popularized by Palantir. And so if you look at the content that we write, it typically ranks in top three. Um, and and to be honest, it takes a lot of time to research. Like writing a content piece easily takes us eight plus hours. Um, because it's not just putting a prompt and it's coming out with whatever comes out. Um we go through a citation, we go through references, we go through competing articles, see what's there. And and then once you have developed the article, this is kind of just the first gate, just the first milestone. Then we have uh a mechanism in place where continuously monitoring the competing articles to see if they update them. And the algorithm, both on um the LLM models and in the search models, are such that it rewards the recency, right? So we're constantly trying to update content with the most relevant information in order to stay in the top three, because essentially, what it's AI citation or Google mentions or whatever it is, it's it only rewards the the top the top five. If you're below top five, you really don't get in any traffic.

SPEAKER_00 11:59

Yeah, and I think like the other hack to doing that is like have data or have something that other people don't have, right? It's very hard to write a post that's very good about something that is not uniquely yours.

SPEAKER_01 12:11

Yours, yeah.

SPEAKER_00 12:12

So what we did for our old company, EGFlow and Peer Grid, was that we used our data, right? So we had a ton of data from our users using the product in interesting ways, and we would like anonymize it in the right way, and then we would produce like basically research. Like this is like something that nobody else can answer, but we can, because we have access to some secret data, and that allows us to make very interesting content that is uniquely ours.

SPEAKER_01 12:37

So let me ask you a question then, David. Um, because I want to riff on that point. You said it's super easy to vibe code, it's it's super easy to build. Um, get in front of the audience becoming harder. Um the speed and velocity has uh increased exponentially since you built your first company. How does a scalar founder think about creating a mode? Um, so that it's it's something that gives them an enduring, lasting competitive advantage.

SPEAKER_00 13:10

It's it's a good question. And I think there's probably many modes to build that they're all classical. I think the number of features is the one that has disappeared the fastest, right? Like in the old days, it would be kind of hard to build a new learning management system, which is what we did at Eduflow, because there's just like a hundred table stakes features like SCORM support and single sign-on and quizzes and so on. You have to build a hundred things. You could do that in a month now, so that's not the problem anymore. So then you have the other types of modes, which is like you have a lot of customers, they're like integrated already, they don't want to churn. You have a good brand, you have data that you can use for some advantage, different types of network effects. I think all of the other modes are classical and they are probably still around. I think a lot of people will think about data as a mode, and I think it is a strong mode if you can really make it useful. But I think for most people it's quite hard because the models, for example, if you think about AI models today, they're growing so fast, they're becoming better so fast that even the data gets outdated in a way, and like all the data you thought was very valuable turns out to be not that interesting in the next iteration. So I don't know, like that's an open question for me. But you can see products like Lovable, for example, and there's like a bunch of competitors, uh Replit and V0, and so on. And they're all kind of similar products, right? And like one of their biggest advantages for, for example, Lovable is that they have an incredibly strong brand. People know about it and they want to use it. The same with Chat GPT, like in many ways, Cloud is probably better for many things, but ChatGPT was the thing that everybody used in the beginning, and it's like become synonymous with AI now. Um so having a brand, being out there, like talking about it is it's a huge benefit. And if everybody, if every product converges to the same thing, then you're just gonna stick with the one you know, the one your friends are using, like people are gonna be simple about it.

SPEAKER_01 15:13

So um a lot of our clients are asking a question, what will happen to software? And I'll give you kind of context because it it triggered me with the with the previous point. We we're launching the future ventures academy, and in our search for the future ventures academy, which is the course curriculum, the active lab, the activations, um, we went down the traditional path. Let's let's explore learning management systems. So we went to like looking at BetterMode and Micro Networks and Hybrid and blah, blah, blah, blah, blah, a whole bunch of lists. And so, and then we went down the open source route. And I think we've settled down on an open source system for a moment. Um but if you can build an LMS within a month, then what's the value proposition for a SaaS company? Do you believe that the SaaS market will go through a bit of disturbance and only the biggest, most entrenched, the companies with a high switching cost will continue to drive? Kind of like what's your perspective? And if you can break it down into like large or complex software, like SAP, mid-grade, and kind of like more entry-level niche product.

SPEAKER_00 16:31

I think it's probably gonna be a slow but then like sudden transition in a way. Like, if you think about simple products in a way, like DocuSign is everybody's favorite example of a company that has a very simple core product and they have like 5,000 people or something. Yeah, um, it's not like yes, people can vibe code a docusign tomorrow, but that doesn't mean docculies is going to zero tomorrow, right? It's gonna take a long time because people are like integrated into this product and so on. And it's not like people are gonna wake up tomorrow and say, I'm gonna vibe code my own doc design for for a big company.

SPEAKER_01 17:05

But what's happening? So it's kind of like I don't know. I I'm assuming that's in there, Mark, you guys don't have Costco. But what the one of the examples here is that it's a big warehouse, uh yeah, and and so some of the examples that's being given is like or or or maybe Amazon is more accurate than Costco. If a product becomes really successful, they they build out the name, the no-name brand, or they build like the Amazon version of a product. And so I but because we deal with a ton of founders, we um about a month or so ago, we got a document, an NDA to sign, and it came out of a Google Docs. So now we see the super mega cap the companies rolling out features that previously were companies like Docker. So now we're signing documents via Google Docs rather than DocuStyle. So do you because of the speed with which things can be developed, do you now think that some of those niche companies are going to disappear?

SPEAKER_00 18:02

I think it's gonna be incredibly fragmented. I think like every company will do everything. I think is kind of what I'm seeing as a trend right now. I saw like I love the lovable product, I've been following it for a long time. And then I saw this morning, I think, that lovable is now releasing a new version that can do anything. So it's not just building websites, now it's like an everything agent. Like every other company. Like there's no company right now in tech that's not building an everything agent. Like they're all building some variation of Claude Cowork or OpenClaw, whatever. Every single company, Airtable, used to be a beautiful table product. Now it's an Airtable agent thing. You can't even make a table anymore. And like it's like every single company is converging to the same product, trying to be everything for everybody. And I think that's really weird, and I don't know how it's gonna work, but I think you'll see you'll probably see dynamics like um Microsoft acquiring Slack and so on, where like if you have a really strong enterprise footprint and you have the customer base already and you have the data processor agreements, you have a big advantage because you can just now ship so many products and features into your product ecosystem. You already have the customers, you just upsell them on new things. So, on one hand, anybody could build a docusign, but because everybody can build a docusign, it's gonna be a thousand docusigns. And the one you're actually gonna use now that they're all kind of comparable is the one that's already in the stack you use. So I think actually this is gonna help some of the incumbents quite a lot because they can now just expand their product repository to like anything goes, like Google adding docusign features, right? Yeah, so it's chaos, it's complete chaos. I don't know what's gonna happen, but it's definitely weird.

SPEAKER_01 19:50

So can we now transition a bit and talk about agent work, uh, your your new company? Um, because we're seeing the same thing, we're seeing everyone uh and anybody trying to roll out some kind of an agentic uh platform. What is Salesforce, what is like Hotspot, what is like you can build your own agents? So tell me how you were thinking about agent work.

SPEAKER_00 20:17

Yeah, in a way, it's kind of like I'm I'm doing a dumb thing, right? I'm also building an agent product, and I've just been saying that like everybody and their mom is building an agent product. So that seems kind of ridiculous. I think our approach is a little bit different. So, like the core idea behind agent work is that it's a product that has an agent inside it. You can give it a task like you would give to Claude Cowork, but then what we try to do differently is that as soon as the agent kind of hits a block or is thinks it's done, but it's not necessarily correct, there's a human in the loop that checks everything and ensures that the output is correct, and that the agent will actually finish. So just think about it as like a David.

SPEAKER_01 20:57

Um, who is the ideal customer profile for agent work? Kind of like I'm assuming it's somebody that maybe has some technical predilection, but it's not necessarily um super knowledgeable.

SPEAKER_00 21:09

Just hopefully it's not um it's not an engineer, it's not somebody who gets excited about running their own open claw or something. So agent work is for people that are like non-in-engineering roles, so that could be like sales, marketing, legal recruiting, ops, like people in companies who have different types of computer problems, not like my computer isn't working, but like problems you would solve with a computer, like making slide decks, generating reports, doing research. Um they have a problem that's big and complicated. They could, in theory, hire a person to do it, but that's too complicated and costly. But Chat GPT can't solve this problem. This is like too big or too important. Um so this is a way to like give that problem to a to somebody and get it solved, and then what 18 World does is like it Use AI as much as it can, but there is a human in the loop who ensures that everything is going to be good enough when you get it back, and you're not going to have to manage an agent now. You're you're getting a result back. That's kind of our approach.

SPEAKER_01 22:12

And so, David, where does agent work excel in structured tasks? Like, for example, like you in cloud, you can build a skill, right? Like, um, or in an unstructured task, kind of like the one-off. Like, where where would it excel vis-a-vis uh another product or a platform?

SPEAKER_00 22:31

So I think like where we've started and where we excel today is things that relate to like finding information online. So that could be generally research problems, but it's specifically like lead generation, web scraping, and so on. Um, and that could be like one of the reasons we can take an excel at that is that a lot of the data you often need is behind like gated gated services, LinkedIn profiles or stuff like that. It's also like sometimes these like very vague problems where I need a bunch of things that like live up to some criteria, but you don't know how many there are. It's kind of you need to find them on weird sources, you have to judge a lot. Like, so we have a lot of people using it for lead generation of different sorts. Like, give me like founders who exited at least two years ago and are now working on something new, but they haven't raised capital for it yet. Stuff like that, which is like hmm, that's like a hard query to make. So you have to like make a good plan and execute it and so on.

SPEAKER_01 23:29

So let me give you a real scenario. Um, about a month or so ago, um we decided to start building our own uh platform, the capital intelligence platform. And and and the reason why we did so is we raised money on behalf of companies, and uh um have like there are products, there are a ton of products that do this, that that would give you uh uh like an investor database, but that that wasn't our primary motivation. Uh our primary motivation was speed, quality, efficiency, and precision. And so we started to build our product, and so we we had to go through laborious effort to build out, of course, we started with our theorem, to build our database. We have 17,000 investment firms at the moment, being private equity venture capital firms, uh family offices, uh, angels, and so on. Now, can I use agent work to continuously enrich the data where one of our 64 fields is blank and not populate at the moment? Because essentially, what we're trying to do is in the algorithm, we have we have we have done simple matching, like a company A, this is the this is the like this is who they are in terms of context, background, geography, check size, they look, and then we're trying to match the investor. So, how can I use agent work to populate, continuously populate this uh this database?

SPEAKER_00 24:53

Yeah, so agent work can actually be used in in two, maybe three ways, but at least like two ways. One is as a human, looks like any other product. You log in, there's a chat, you talk to the agent, you give it a task. I think that would be more useful for the first step of like, hey, I need a list of a database from scratch, and then you can use agent work as an API or or through an MCP server. So you could have basically an automation that says whenever we have an empty field in our CRM, or whenever we add a new uh entity to the CRM, tell Agent Work, like, hey, I have this row, fill it out, and I want the information to be correct, and then our system will go figure it out, solve the problem, and send you back the data. So you can call Agent Work as an API as well, which is useful for kind of continuous uh work.

SPEAKER_01 25:38

Nice, nice, nice. Um I'm not a developer, I'm a business guy. Um everybody's a developer now. Well, it's funny that you say this because I I'm a CPA by background, like a 25 years in consulting. Um but I uh on a continuous basis, I have Visual Studio Visual Studio and Cloud Code running on my machine on my third screen on a continuous basis. And so in in cyber intelligence platform, we have uh plugged in a whole bunch of uh API keys. Like, I mean, I have my Apollo and my Snowview key and OpenAI and Tropic, like just the list is on and on and on. But what I find um this is just experiential learning. The the LLM that they find to be the best for kind of enrichment is Grok.

SPEAKER_00 26:30

Like open AI is data, it's better data, right? That's better data.

SPEAKER_01 26:34

But I think that they also might be some governance mechanism because cloud is amazing at executing difficult tasks. OpenAI is kind of like PG, everything comes out, everything that comes out of ChatGPT is PG. Um, Cloud is basically on a technical, and Grok is like there's no almost like there's no limit. Like everything is like whatever you want it to be.

SPEAKER_00 26:56

Yeah, I think that's correct, right? And I think we're we're like operating in some variation of those spaces as well, where like we like currently the current model of aging work is built on top of ChatGPT, okay, but it has access to cloud as well, so like it can execute claud code if it needs to, and it can use any APIs that we have available and like different services. So like AgentWork is not trying to be the best web scraping product in the world or the best code writing product in the world, it's the best place to go with a task, and then we will figure out what the best tool is and we will use it for you. So you don't have to know, and you don't have to care how the source it is made, you just want your table, yeah, yeah, yeah. Exactly.

SPEAKER_01 27:36

Okay, um, and so how does agent work continuously learning in in the background? Because I'm assuming that some tasks you've anticipated, but sometimes are maybe different. Kind of like what are some of the edge cases that you would have seen kind of happen at the agent work?

SPEAKER_00 27:55

Yeah, so uh we constantly see tasks that are like, hmm, okay, I didn't expect this. Like, yeah, that's kind of weird. How do we do we even want to solve this task? But I so like I guess there's two types of questions, right? Like, what is agent work? What's more more expected and unexpected? And I think for that, the way we think about it is like things you could in theory give to a freelancer is good for agent work, right? If it if it leans on like an incredible amount of context that only you have or your team have, and you don't give that context to agent work, then it can't help you, right? You can give it all the context, and sure, then agent work can solve it. But if it's just ask a random freelancer without context to solve a problem, they couldn't do it either. Um, I think agent work currently is very good at like one-off tasks. Like, here's just a task, make a spec, give me the results. Okay, whereas like an employee is more like on an hourly basis kind of thing. That's the next step for agent work, it's more like long horizon employee-like uh interactions. And then I think um what's the other question you asked? Or maybe I was just trying to say something. Edge cases. Yeah, I think like we see we see all sorts of hard problems, right? Like um, yeah, everything from like document processing and so on. But obviously, when people come to us and say, like, I need to build an app, what ATWork should do, and what ATWork does, it's it's writes back and says, like, I think you should just use cloud code for that. Like, we don't want to necessarily entrench on like where we don't see ourselves being the best product. Um but we have like things that we're really good at, which like research type problems right now.

SPEAKER_01 29:32

So, how do you move like and I'm assuming that it's it's not just agent work, but there's so many other companies that I'm assuming that they're going to um completely disrupt like the the business of companies like AppWork and Fiverr that that have like uh they build an army of freelancers. So, how do you move from one-off tasks to teach the employees that you've done everything that you need to do in terms of them building institutional knowledge and then they have the context that they need in order for them to build this if it as if they're one of your employees?

SPEAKER_00 30:10

Yes, I think like and I also remember the other thing I wanted to say, so I'll try to remember that as well. But like I think fundamentally, my perspective on this is that an employee is a long-term thing that executes tasks over and over, right? So they have like a long-term memory, but then what they do is tasks, fundamentally, right? So the current version of agent work is just the task part. You give it a task, it solves it, and it gives you back a result, one-off. And then employee mode of aging work is just a long-term horizon thing with memory, but every time it needs to do something, it uses tasks, right? So you can take what we already have for tasks, and then you can just do task after task after task and remember what you've done before and update your knowledge and memory and so on. Um, so in that sense, I think it's kind of easy for us to bridge the gap. If we can solve tasks well, it's not that hard to make an employee mode in agent work. The other question you asked that I didn't answer was around like how does it get better? How does it learn? And I think that's one of the big advantages of how we think about agent work is that every time you run a task through agent work, we don't just get the model to solve it, we also get a human to help and verify. And that means that every time we solve a task with agent work, we now get a very nice piece of data that we can use next time you get a task. So we obviously reuse all of these learnings every time. So whenever you're done with a task as a human and you verify it and you say, like, this is now done and good, the last step for you as a human is to like put those learnings into the database and say, like, next time we see something like this, we should probably not do that or that, we should do this instead. This is what kind of the feedback is from this task. So we build a big database of feedback and learnings from using the product, um, which is how agent work in itself can get better day by day. Um, and also like that that's just like an incredibly interesting piece of data for us to kind of work with.

SPEAKER_01 32:06

And when you say the human, you mean the human that's actually asking for the task to be performed, then human in the middle. No?

SPEAKER_00 32:14

No, no, no. Our our people. So like our human experts, as we call them. So a human expert at agent work is like an employee of ATWork who is helping agents solve tasks. So if you come in with a task, our agent will try to solve it, and then a person will be associated to the task, and their job is to make sure that the result is actually good enough. And that's kind of that's on us, right? That's on the house that there's a human involved because that's the way we ensure that the output is good, but it's also the way we build better data for the future.

SPEAKER_01 32:43

So, how do you scale? I mean, this this obviously you over time I mean some problems are going to be repeatable as they're solved, you don't need a human. I get that. Yeah, but how do you scale if if you always have human uh on your on your end?

SPEAKER_00 32:59

So, like, obviously, like the the way to do that is to make the humans more and more effective. And like inside agent work, also in the current version, you have like an accuracy scale in the beginning. When you give it a task, you can decide like, do I just want the agent to give a stab at it? No humans. Do I want high accuracy? Which means we'll run the task multiple times in parallel, we'll cross-check against each other and do very proper QA, but still just the robot. And then we have like, no, I want like human in the loop. I actually need a human to verify this. So you can choose like a lower version. Today we give we have humans on everything because it's important for us to like make sure the data is good. Um, if you pick the high version, right, which we charge more per token, basically. So like the task will cost more to run. But in the beginning, obviously, a human is idling a lot, they're not very effective, but over time we're we're gonna see, according to my hypotheses and calculations, right, that a single human can work across potentially a hundred different tasks in parallel, where an opwork freelancer is working on maybe like three tasks or two tasks at the same time, we can like make them incredibly efficient because they don't actually have to do the work, they just have to guide and help and orchestrate and validate and check and so on.

SPEAKER_01 34:16

So, let me ask you a question. So, about a month or so ago, a company out of Canada here approached us to partner with us, and I I don't have the technical knowledge to assess this. Their claim is that they can get to zero hallucinations, they can move from a probabilistic model to a deterministic model, and so you were like it. I wouldn't mind if you can give us a bit of a background on your on your education because you were trained in this, you worked at CERN, like you like this is a question that you you you can knowledgibly answer. Like, is this possible?

SPEAKER_00 34:58

I don't think so. Okay, yeah, so I have I have I'm a mathematician, right? I have a PhD in ML, but it was like pre-the-lm days, of course. So I don't I haven't like studied extensively transformal models and so on in as part of my PhD, but I have a fairly good grasp of of the field, right? I think can you stop hallucinations? I don't see how that would ever be possible, given that humans hallucinate, right? Like it's not like that's some bug in the LLM architecture that gives you hallucinations. Hallucinations come from the fundamental idea that the model can make a wrong prediction, right? Like hallucination is just a term we've chosen for that. Um, obviously, LLMs have a way to hallucinate in some interesting, specific LLM type ways. Um, if you ask it in Danish what a beefsteak tomato is, it'll like make all sorts of crazy claims that are totally unsubstantiated. But so it's like it's it it hallucinates in weird ways, but but you can see the same for humans, right? I think one of the examples of of humans also being kind of broken is there's a lot of these like classical visual puzzles of like you see something and it looks like one line looks much longer than the other line, but they're actually the same length. It's kind of like a human, a human bias, right? It's a human error in the brain, and humans aren't like the computer would never make that mistake, it would just like calculate the length of the lines, yeah. But humans are also idiots. We just we're used to how humans are idiots and not how AIs are idiots. So could you make the model never make mistakes? Obviously not, but could you make it you could make all sorts of heuristics, right? You can look at like the probability of each token and say, like, oh, this looks this is like this was generated with like a lot of variability in it. So you can you can tune it, you can make it better, you can like highlight the challenges. Can you make a hallucination-free model?

SPEAKER_01 36:52

I think that's philosophically challenging to uh essentially it essentially the claim they have is it's it's almost kind of like a blockchain. Uh information flows through it. That sounds like a scam blockchain?

SPEAKER_02 37:06

What the hell?

SPEAKER_01 37:07

No, no, so that's just like a stamp saying, like this is this is verifiable information. So I was trying to understand is this like a uh a rack model that pulls information because you know that it comes from a structured data, the data is verified, but I'm like, okay, but uh this is a assembly of information if it's just a rack model, like so like if you if you constrain the model to only talk about what's in the secret database, then sure you can make it hallucination free, but then it's also not very useful because it only talks about like very few things.

SPEAKER_00 37:34

Like you can you can make a hallucination free calculators are great, they're hallucination free, right? But they're also only able to multiply and add numbers and so on. So like I don't think that's that's not very interesting. I think what's cool about LLMs is they can talk about anything, yeah, also things they've never seen before, which is also that's the place when they get into trouble, right? If you give them a problem and don't give them web search and ask them to reason about something that makes no sense, they're gonna try and they're gonna fail. Um, but that's also what makes them cool, right? They can create poems, for example. How can you do that without hallucinating? For example, yeah, yeah.

SPEAKER_01 38:12

I mean it's it's amazing. Like now I'm like there's these unique apps, like there's an app called Suno that that creates music and beautiful. I've used it, I love it. But now I'm seeing the LLMs go into this basic kind of like to your point around big companies are gonna become the the companies of everything, and so it it kind of makes me question as to what they are. But yeah, one question they have for you is on reinforced learning. A lot of people and a lot of companies use AI in a very simplistic way, it's almost like a uh a search box in in on a browser rather than to perform tasks, and so even performing the tasks it it in in many ways is very simplistic. How do you how do you build a reinforced learning layer on top? Like like how do you build something that continuously learns itself and it because you're right, a job is a combination of tasks, but how do you continuously get more and more tasks off a human plate and onto a digital worker's plate?

SPEAKER_00 39:18

That's a hard question. I think I think the best strategy for many companies though is to make simple AI implementations. Like you don't necessarily want to change your product into a big text area and like now you just talk, talk to the product. Like, obviously, that's the feature everybody first, everybody got an AI chatbot, and then everybody got a talk to the product model where you can like ask the product to do certain things, which is like that's cool. I think a lot of the powerful AI features are not that visible because they're like happening behind the scenes, um populating certain data based on other data, like ingesting PDFs and converting them into something. I think the best way to as a company though to get more digital and more automatic is to use more software. Because like once you've put something in software, then you can automate it. I think the the biggest challenge is like taking something that you're not doing in like some vertical SAS and then putting it into a piece of vertical sass. Because once you do that, that that product will help you push your automation forward, right? If you're writing all your blog posts in Google Docs and then copying them into your blog, then that's very hard, right? But if you write them in some blog writing product, then that product can build all sorts of cool features for SEO optimization and stuff like that. So I think just like use more software and use software that is great at using AI, that that kind of helps push everybody forward very fast.

SPEAKER_01 40:53

So, how would you think about it? And and and and we we've built workflows around because we write the the content in one piece and we have to obviously propagate it and but have to run it through SEO and AR. Um how do you build this orchestration layer so it doesn't become like a Frankenstein system of like I have this and this and this and everything?

SPEAKER_00 41:15

I think it will be for a while. I think it will be like a Frankenstein's layer, and I think that's because the world is moving so fast, like there's not like everything isn't connected yet, which is one of the biggest challenges for like LLMs right now, and like where agent work has a big step up compared to just ChatGPT, is that the world is like full of software, none of it really talks together. There's all sorts of attempts with like APIs and MCP and whatnot, but like it doesn't actually work that well. Uh so none of the products actually communicate in a nice way, and everybody has their own agent and so on. But I think we're not that far away from the world, I guess, where a person has an a core agent in some way that they talk to. There's like the main the main agent, and that could be like an interface into multiple agents, but like I go into one place and I write, I want something done, and then that agent is now responsible for talking to all the other agents and all the software I have out there and saying, like, okay, David wants a blog post. Let me use the blog software he has and let me use uh Webflow to push the blog out and let me write somebody an email, and like and the problem is every company also wants to own that initially, right? Everybody company wants to be the agent you go to first. So right now, complete fragmentation, yeah, and then we're gonna see at some point some sort of like convergence to a standard or uh there's a product called conductor, for example, for coding, right? Which is like it's a product just for managing cloud code and other tools. Okay, but it's like so you open that instead of opening clawed code directly, because then you can use cloud code more effectively and you can use codecs at the same time and so on. I think we're gonna see these like types of meta products um appear for different things. And open claw is one of the classical examples of this. It's like, hey, here's a product that can do anything, it will just and you give it full permissions and it can go completely haywire. But but the premise and what people liked about open claw is that yeah, it's just it's a thing that does it all. Does it work? Usually sometimes is it scary? Yes, but like but it's the it's the UX we want, right? The thing that people want right now and they can't really get it yet, is that thing where like there's just one product and you always talk to that one product and it solves any problem you have.

SPEAKER_01 43:35

So let me just juxtapose the two strategies. So open claw can do anything and everything, but there's no human in the middle or human in the loop. Uh agent work is human. So you get high like significantly higher quality when you have a human and in in using this to to essentially accelerate the enforced learning cycle. Um If you are comparing scaling strategy, obviously, if if I'm a user, I want the highest quality possible because it gives me precision, gives me accuracy. Could this be a strategy that open cloud is using to say, well, we're just gonna go and acquire the data and then figure out a way to use this data to to make the system better? Kind of like compare the quality versus the speed.

SPEAKER_00 44:24

I think like the data we're gathering is very interesting, right? So that way that that data is an interesting acquisition target one day, I guess. But I think like the way I think about open claw and agent work is like open clause should just use agent work sometimes. Like when you give it a task and say, like, make no mistakes, like this can't go wrong. Okay, then open clause solution should be like, well, uh my client here clearly wants high accuracy. Let me find a way to do that. And that could be using agent work or something else, right? By saying like, okay, I'm gonna spend the money now and go and actually get a human to validate this because I really need accuracy here. Um that's also one of the ways to think about 18 work. One part is we use humans to actually get you to the goal, but also we use humans to give you a stamp of validity. Which we're not promising that the result is correct, we can never promise that. But what we can promise you is that a human have looked at it. So if there is an error, it's a human error, which you can never really get. It's hard to get much better than like the combination of robot and human. Yeah, if they both approve, then that's best best in class, I guess.

SPEAKER_01 45:29

So let me put my own my own hat. So I'm uh uh I'm a CPA, and so in the in the in the audit world, we have kind of three levels of assurance. Um, we would have like no assurance, which is like we've we've done we've just it's it's called compilation game. We've just compiled data. Then we have negative assurance, meaning that like nothing has come to our attention, so that's a review engagement. Then we have audit where we have done everything possible within reasonability to confirm that the information is is correct. We there's no absolutes, but it's a reasonable level of assurance. Yeah, so um with with agent work, would it be like kind of like the comparison? Like uh I can choose my task and I can apply a review engagement, I can apply audit level of assurance, and then open claw is kind of like no assurance. It's just like we've just assembled stuff. And can you toggle within agent work? What level of assurance would you would would you give on the output?

SPEAKER_00 46:31

Yeah, because that's one of the first boxes that appear after you give your task. It's like, do you want fast mode, which is like basically your no assurance, or do you want high accuracy, which is kind of the like we're gonna QA and we're gonna do it in multiple times, and we're gonna do that and not find anything, then we're gonna send it to you. And then audit mode, basically we call it premium, right? But like there is gonna be a human that looks at this before you get the result. So when you get a result back, like some human has looked at it and verified that this is actually living up to expectations, and then send it to you.

SPEAKER_01 47:02

And what does the human look at? Does it look at it kind of like the the code that's being executed to to execute a task to make sure?

SPEAKER_00 47:10

Yeah, so they they yeah, they engage in different ways. They have like the the most important thing is like at the end, they actually look at the results and say, like, does this does this actually make sense? Does this actually solve the problem? Do all the links work and so on? Yeah, but then they're also engaged along the way. So like at some point the agent might like realize, okay, I'm actually like this is going weird, like this is not really looking well. Like, let me engage the human already now, and then the human can come in and say, like, no, that's a bad path, don't go that way, go in this way instead. So they also help get to the goal, and then the agent helps give a proposal for what the human should verify. So if the agent knows, like, this is kind of my weak spot, I think this is what you should be checking, or I would like spot checking on like five rows in this table. Like, these are five rows you could check. Because obviously, for agent work to work, it can't be that the human have to solve the whole task again to verify that the solution is correct. That that wouldn't be very effective, then it's just a human and not in the loop. Um, so we have to find a way to say the agent should do the work, and the agent can be quite good at figuring out like what are the weak points, and then we'll generate a proposal for like what we think the human should check. But then they can also do their own checks basically to say, like, well, actually, I think this I should check as well. And then finally, the the client gets the solution back, and then they verify. Like, if they're not satisfied with the solution, we're notified, and then we give it another round, right? And we figure it out, and then that's a big learning check for next time. Like, actually, there was a problem here, and we didn't catch it. That's that's a bad thing for us, right? So we need to go on and improve.

SPEAKER_01 48:50

Wow, that's amazing. Um, what um what's your kind of prediction? Like if we're if we're I'm not trying, but if we're trying to be a little bit more provocative and controversial, what does the world look like in 5, 10, or 15 years? I'm not asking for 30 or 50 years, but kind of like, what does the world look like in 5, 10, 15 years?

SPEAKER_00 49:17

Honestly, I think I'm quite a techno-optimist and I am very AI pilled. I've been AI pilled since I was 13 and trained my first neural network. I think that we will not be working as much or at all, many humans. I think that it's gotta be a while before like robots and so on are solved properly, but like it's just a matter of time. And I think I just don't see like people compare it to the industrial revolution and so on, are like, oh, like no, the machines become better and then humans get new tasks. But I just don't see humans catching up to this one. Like, if you actually get to proper ATI where robots are better at everything than humans, humans will do things that we want humans to do, like take care of babies and like talk to your elders and stuff like that. And I don't and like luxury things where you want it to be done by a human, but like for everything else, I don't think humans are necessary, and like will we all just like lie on the couch? It's probably like not realistic, but we will engage in things that make us happy, like playing chess, even though the robots are better at chess than us, we still do it. Um, we'll read books and not just get summarized books because we like reading a book, and we'll we'll do work for fun, like we'll do work because it fulfills us. Um it's kind of like Maslow's hierarchy, right? Like we're gonna satisfy all the bases pretty easily, at least in some like unevenly distributed way, right? Like it's gonna happen in the Western world first and for the rich people first, and so on. Um, but but we are going towards a future where the society has to kind of adapt in some way, right? Because if suddenly people stop working, like what actually happens, I'm not smart enough to figure that out, right? Like it's it's complicated. Taxes, what do we do with taxes? What do you do with labor? What do you do with salaries? Like, I don't know. It's it's all very complicated. But what I will think is is gonna happen in the short term is that it's not like the world changes tomorrow and like suddenly everybody gets fired. I think we're gonna see people competed away. Like we're gonna see new people. We see some companies firing a lot of people, but I think what's more important is like new companies not hiring. Like we're gonna see more and more people starting new ventures, not hiring anybody, doing it all with AI, and out-competing the established players who are not doing that because they don't like firing people and like they like to be managers of many people and so on. I don't want to be a manager of many people, so I'll try to not hire, and then my unit economics are better than yours, and then I can outcompete you, right? So I think it'll be a slow transition, but suddenly it will appear to be very fast that everybody's gonna have to adapt to a world where humans are not that important um as like labor force. Yeah, they'll have to be important for some other reason.

SPEAKER_01 52:05

Oh, for sure. And and I think that probably the biggest disruption that we're going to see, it's uh um not necessarily, I mean, there's definitely going to be elimination job uh of jobs, but what I'm envisioning or what I'm seeing is that if you think about a job as an assembly of tasks, that assembly of the tasks is going to shift the most most dramatically. It's not that humans are suddenly going to be all humans are gonna be out of a job, it's just that the tasks they were doing before versus the tasks they're doing now are just different. And in some cases, it's gonna be radically different, and it's gonna eliminate some jobs as a job title completely. In some cases, it's just going to transform what's included in this job. But I wanted to ask you, um, a lot of AI currently resides in kind of in the digital world, but you're starting to see the shift into the physical world with these robot farms, the training kind of how do you envision agent work moving to the physical world?

SPEAKER_00 53:06

Oh, I'm not sure we will, honestly. I don't think like I don't think agent work is like built for the physical world, but on the other hand, like the good thing about agent work is that it's kind of like universal in a way. You can you can write to agent work right now, like, hey, I need a cup of coffee delivered to this place tomorrow. The agent will say, I can't do that, but then a human in the loop will say, like, I'll order the coffee, and then I'll call the coffee shop. Like, I'll be the human in the loop, for example, call the coffee shop and order you a coffee, and then you'll get your coffee. So there's no reason why agent work can't solve any problem. That doesn't mean we should be ordering people's coffee, right? That's not the code of aging work, but it it's kind of an everything product in that sense.

SPEAKER_01 53:50

So, do you see aging work kind of being like the ingress point for tasks that that uh transcend the digital and the physical world? And I just uh I was just gonna walk through an example. So yesterday I was listening to a podcast, and uh uh I'm I'm not a gamer, I've never been a gamer, but the example was Pokemon Go. And so Pokemon Go um apparently sold the game for billions of dollars, but they they kept the video data. And so now they're licensing the video data because obviously people were walking through with a uh device recording, and so like kind of like the way Google Maps goes in the Street View, they they're now selling the data. So if I was to play through an example, um somebody's ordering a coffee, um um, they go to agent work, uh, the the order gets passed on to a delivery app, the delivery app goes out and and hires uh uh um a human to go and deliver. Is that a possibility?

SPEAKER_00 54:49

Okay, absolutely, and then the next time the agent will know that the last time this happened, then what the human did was order a coffee through the service. So, like, I'll just try to do that now, and then hopefully next time it's all automatic. But like, we won't deliver the coffee, some service provider will drive out the coffee, obviously.

SPEAKER_01 55:07

Amazing, David. This has been a fantastic conversation. We've covered a ton of ground. Thanks again for joining, being uh so open about the lessons learned along the way. Um, I love having a conversation with you. I love seeing you build uh agent work that and and helping people adopt AI because what we're seeing is that this there's still this fear about using AI. And and and and our advice uh when we when we meet with with companies and clients is just put the hands on keyboard. There's no such thing as like I'm gonna go and learn it by watching a video or and whatnot. So thank you so much for joining us. That was amazing. Um, thank you. We'll see you in the next episode. Thanks, David. Bye.