00:00:00:03 - 00:00:08:04
David
And it just takes one thing to go wrong. Like we've seen this week and that whole house apart starts to fall apart.
00:00:08:06 - 00:00:15:13
Martin
This is brought to you by harness where we break down software delivery is actually changing in the AI era. I'm Martin Reynolds.
00:00:15:14 - 00:00:19:09
David
And I'm not Adam Mariano.
00:00:19:11 - 00:00:41:08
Martin
Super excited to have David Cannon with us on the show today. And he is director Americas at People Search. He is also an executive VP at infinity three, and he's the founding director at Sears, which he is also doing. You know, this is a busy man. So when he says he is getting stuff done, he he's really getting stuff done.
00:00:41:12 - 00:01:10:06
Martin
So welcome to the show. I want to get right into it with our first story, which just I mean, it just has to be Nvidia. So Jensen Huang resigned as the Nvidia CEO on November 12th after 33 years, effective immediately with CFO, Kress named us interim CEO, and the stock fell as much by 13%. At the open, Nvidia owns stakes in Intel Space Core.
00:01:10:06 - 00:01:44:12
Martin
We've has placed up to 10 billion to anthropic, and that 530 billion of commitments and guarantees in its last filing and things like core we've and nebulous those NIO clouds you know they're hit really hard. Bloomberg essentially said that Nvidia's up to $10 billion anthropic pledge is back in focus as anthropic heads towards its IPO, and Yahoo Finance is counting this number just blows my mind 530 billion in total commitments and guarantees.
00:01:44:13 - 00:01:47:05
Martin
I mean, what's your initial thoughts?
00:01:47:08 - 00:02:17:00
David
My initial thought is that there's been a lot of talk about the the possible Chinese connection and the motivation behind this, but in reality, we're facing we're facing a situation where the entire industry is being disrupted. And that that to me is the biggest thing about the story, is that we have these pledges being made. If one organization does this, then will come up with this $10 billion and so on.
00:02:17:04 - 00:02:37:18
David
And that falling through means that we have we have a lot of risk in organizations that have invested there, and their technology solutions on this hardware and software kind of combination between, between Nvidia and and others.
00:02:37:19 - 00:03:02:11
Martin
Yeah, that circular financing is insane. When I look at this stack and the way everything is lined up for the whole AI boom, what scares me a little bit is that there is no competition in the chips space. It is all on one vendor. Is that true capitalism at work? And and is it pushing those chip vendors as far as they could go?
00:03:02:12 - 00:03:10:07
Martin
If there were, if there was more competition in that space, how do you feel about maybe regulators coming in and saying, you know what, we need to break up this company.
00:03:10:08 - 00:03:30:10
David
I'm not sure that I stand on the on the side of we need the regulators to step in and break this company up. I'm on the side of the fact that we are facing a potential for serious vendor locking, and when that happens, we're all at risk. And it just takes one thing to go wrong, like we've seen this week.
00:03:30:10 - 00:03:39:10
David
And that whole house of cards starts to fall apart. I think that the market.
00:03:39:12 - 00:04:08:03
David
The market really dictates that there needs to be more diversity. There need to be other players in this. Do we need the regulators to step in and make those calls and drive that perhaps, you know, we've been we've been in the situation for very long where this kind of, this kind of growth has been encouraged, has been boosted, has been formally welcomed.
00:04:08:05 - 00:04:18:06
David
And I think we're facing a situation now where in the consequences that are becoming very clear and the market needs to address this.
00:04:18:08 - 00:04:44:05
Martin
One of the other things that was super interesting to me in this is, you know, he's a founder with dozens and dozens of direct reports and absolutely no public succession plan. So, you know, the key, like one key person for essentially the whole AI supply chain, you know, is there is this just an end to problem or are we seeing this across the space?
00:04:44:07 - 00:05:07:16
David
We are we're seeing a number of high profile, highly dynamic, highly charismatic leaders in a similar situation across different industry sectors. It's not the first time we've seen this in in the industry. It is not sustainable. As I've just said, it just takes one of those, one of those linchpin to fall through, and then suddenly the whole thing is exposed.
00:05:07:18 - 00:05:32:07
David
Another thing that comes out of this for me is the fact that we have kind of been treating AI as a limitless resource. We did this with the cloud, right? You can scale infinitely with with this technology, and we've kind of treated AI in the same way. It's just never ending. We're going to keep building new models, new and new ways of doing this infinite cloud capability to deal with the workloads that we're throwing at this thing.
00:05:32:13 - 00:05:50:04
David
And what this is showing us is that this is not infinite. There are limits. And as I said, it just takes one component in that chain to to fail or to weaken. And, and then it just exposes the limits of, of the industry. Really.
00:05:50:05 - 00:05:50:11
Martin
Yeah.
00:05:50:12 - 00:05:51:17
David
I so.
00:05:51:21 - 00:06:12:05
Martin
I don't want people to feel like I'm picking on Nvidia this week, but I'm picking on Nvidia this week. So one of the other stories that's happened over the last week was fireworks AI, which is backed by Nvidia and is an inference platform. They told customers on November 9th about its second breach in weeks, and the new disclosure widened to the billing data.
00:06:12:05 - 00:06:32:00
Martin
And for some accounts, secrets, data sets and incomplete model weights. Notion says that chat transcripts for fewer than 1% of its workspaces were accessed, and the first notice it got called the data synthetic Test data. I'm sure you've heard about this. I know you'd like to keep things running. You know, how do you feel about this one?
00:06:32:04 - 00:07:00:10
David
Obviously it's very concerning, but I think what we're seeing here is that we're exposed to security threats that are not even our organization. We're we're being exposed to, you know, to our data, our transactions, our our interactions being exposed through something that we don't even control. That, to me is the biggest concern about this one.
00:07:00:12 - 00:07:22:05
Martin
Well, yeah. And the one that that caught me out and was the fact that the second breach was not even, it wasn't even fireworks. AI it was fireworks AI cloud provider that was breached. So it was like, you know, where does your supply chain end when your vendors vendor is now part of your risk assessment?
00:07:22:06 - 00:07:43:12
David
You know, we try and mitigate this through through having contracts in place and making sure that we understand what is going to be delivered and how it's going to perform and what security measures are in place. But to be honest, this world is so uncertain. It is it is so volatile that there is no real guarantee. The things are changing so quickly.
00:07:43:14 - 00:08:24:17
David
There just is no real guarantee. And this, this makes it this makes it really difficult for individuals, especially that I'm thinking, you know, we're talking about citizen developers, right? Citizen builders. And it makes it really difficult for those individuals who have not had the decades of experience of how to manage these complex technology environments, assuming that what they're building is fine is, you know, is secure, is is robust, and it's not dependent on what they're building, is depending on something upstream, and they don't even know to look for that necessary.
00:08:24:21 - 00:08:48:21
Martin
I was smiling then, David, because I have to say, I love your optimism about people because like, genuinely, the fact that a citizen developer would think about resiliency, reliability and data security makes me chuckle a little. But that being said, you do make a very serious point. So let me give you a like the full breakdown of this.
00:08:48:21 - 00:09:17:23
Martin
And then I do have a couple of extra questions. So fireworks was originally hacked in October 1st through compromised internal credentials credentials reportedly exposing API keys for a limited number of accounts. Then on October 23rd through 24th, a third party cloud provider that hosted its research and development machines where an attack. Right. So, you know, notion heard that they had synthetic test data.
00:09:17:23 - 00:09:33:11
Martin
And then four days later, you know, possible customer data. I you mentioned contracts, but how do you contractually require AI vendors to give you warning, accurate notice. And you know, what's what's the notification clock?
00:09:33:17 - 00:10:03:02
David
Well, it's incredibly difficult to do that if they're unable to detect these things. So what are they going to notify you in advance. There is none. It's we're we're building solutions based on plumbing that is inherently unstable. We're building things that we considered to be stable, but they're not supported by the infrastructure and environment that is stable. So the plumbing is leaking.
00:10:03:03 - 00:10:20:19
Martin
Are you implying that these companies maybe aren't, you know, necessarily building the the, the standard kind of safety constraints that we might normally build? Are you, you know, that they're so focused on innovation, they just want to, you know, just get on with that. Don't worry about the security and the.
00:10:20:20 - 00:10:30:08
David
I'm not saying they're not worried about security. I will say that the focus is more on innovation than it is on, on, on the ongoing sustainability of those solutions.
00:10:30:13 - 00:10:45:04
Martin
So I would describe that as risk deferment. And what I mean by that is they're essentially saying, hey, we we're doing all this great innovation. And and so it's not our risk. It's your risk. Well is that really fair?
00:10:45:06 - 00:10:58:10
David
No, it's not at all fair. Right. Because the expectation of people who are using these solutions is that these APIs are secure, that I can use them, and that I can get what I need and be safe in doing so.
00:10:58:12 - 00:11:17:10
Martin
All the stuff that we know, as you know, good engineering practices don't go away just because we're using AI. And I think we're slipping into old bad habits. And we need to absolutely, you know, take a breath, make sure we're following good security practices and then kind of move on. If you sort of mean and keep moving forwards.
00:11:17:10 - 00:11:18:12
Martin
But you know.
00:11:18:14 - 00:11:19:08
David
Yeah, absolutely.
00:11:19:12 - 00:11:25:23
Martin
But if you want to give me your credentials for your bank, I really need to go on a spending spree. If you just drop them in the chat.
00:11:26:00 - 00:12:04:04
David
We seem to we seem to be very keen to suspend all of the rigor that we've learned over decades in the interest of pursuing this innovation, but really the most effective use of this new technology, all of these new technologies, is when we use the tried and trusted approaches that we've used before. And to give an anecdote, I was talking to somebody last week, one of the senior developers in a very large organization, and who said, very concerned about the quality of development with, with that's being done by many of the members of my team using AI.
00:12:04:06 - 00:12:19:10
David
And as to what makes the difference between you and what they're doing. And he said, what makes the difference is I'm still using the standard development practices, the checks and balances that I have used over the last 25 years. They're not.
00:12:19:12 - 00:12:43:12
Martin
Yes, I love that. I genuinely love that. I was actually on a webinar panel earlier this week with Patrick Du Bois, the, you know, of DevOps fame, and he said exactly the same thing he said he said, I still follow all the good practices that I've always followed. They haven't stopped just because I'm now using AI. Okay, I'm going to throw a surprise story.
00:12:43:14 - 00:13:10:04
Martin
You just because of the story that you just told about, you know, that kind of let's keep moving forward. Let's not stop. So I don't know if you've seen this, but a group of nonprofits sued OpenAI anthropic, SpaceX's AI division, Meta and Google on November 12th. So yesterday, asking a San Francisco court to block further development of their most advanced AI models.
00:13:10:04 - 00:13:22:00
Martin
They argue that under California, unfair Business Practices and Public Nuances Law, citing the company's own leaders warnings that AI could destroy civilization.
00:13:22:01 - 00:13:50:13
David
No, I had not read that story. But but I have to say, you know, this is we're all in this together. The whole industry revolves around relationships between the between these these organizations that are pioneering and the organizations that are consuming. And I guess what we're trying to see happen here is a restoration of balance, of power between these, between these two, between the supply side and the consumer side.
00:13:50:15 - 00:14:05:22
David
And yes, I guess that's kind of what's happening. But to push the entire thing onto the developers and say, stop them innovating. This is a very pessimistic view. And I'm an optimist, I.
00:14:06:00 - 00:14:08:21
David
Yeah, maybe put some guardrails in there.
00:14:08:23 - 00:14:35:17
Martin
But my favorite topic, regulation guardrails is regulation. Yeah. So let me just summarize that up a little bit. So essentially they argue that no other industry is allowed to treat unsafe products as a or inevitable or, you know, or to self-regulate. Right. So it is the it's honestly the latest running. It's the latest in a whole bunch of lawsuits that happened.
00:14:35:17 - 00:15:22:13
Martin
So there was the Florida one that sued OpenAI over in June about child safety, and then in September asked the court to bar OpenAI from developing new models without without outside oversight. And then there's a motion still pending, after which OpenAI said it paused training its most capable model. Then last month, a group of eight states sued OpenAI in courts across the country, alleging ChatGPT coached users, including teenagers, to harm themselves or others and seeking financial penalties and an order barring selfie in an order barring self improving models unless they can be aligned with human values.
00:15:22:15 - 00:15:46:15
Martin
I mean, that's a it's not like I pulled this one out because it is a little bit of a it felt like a one off until I did a little bit more reading. And I'm like, well, no, this is this is a trend. And what this says to me is there needs to be some regulation and maybe not even regulation, just some transparency around this.
00:15:46:17 - 00:16:08:17
David
We can trace us all the way back to industrial revolution, even before, right? Every single wave of new technology brings with it the potential to to harm and not only the potential, but actual harm. And we're seeing this this is a little bit different, though, because here we're not just dealing with with things we can predict. These are not deterministic models.
00:16:08:18 - 00:16:33:08
David
This is not if we do X then why will happen. This is probabilistic. And what this means is that there is a very high likelihood that these models can go off the rails and cause damage that we actively try to prevent. So I'm all for putting those kind of boundaries and those guardrails in place. But that raises a whole bunch of new questions like, who are the governors?
00:16:33:08 - 00:16:36:11
David
Who are the regulators of this?
00:16:36:13 - 00:16:52:19
Martin
I mean, one of the things that that scares me is if you're looking at your, you know, your organization that you work in, if you know the organization where you work today, I'm pretty sure you're using AI, right?
00:16:52:23 - 00:16:55:04
David
Absolutely. Yeah, we're using AI.
00:16:55:06 - 00:17:21:13
Martin
And what happens if frontier model gets restricted by court tomorrow for something for some reason and it's no longer available, what happens? Like how does that impact your business and how prepared are you, me, anybody who's consuming this stuff for something suddenly not being available. And I could overlap that with, hey, an outage rather than a court. I think that, you know, either apply, but you know, how prepared are you?
00:17:21:14 - 00:17:23:00
Martin
How prepared are we?
00:17:23:01 - 00:17:49:00
David
I don't think I don't think we're very prepared at all. And I think the current legislation that's emerging, I think Europe is a little bit ahead on this current, current legislation that is emerging, I don't think is able to keep pace with the with the scale and rate of change that's happening. So we're laying foundational, kind of foundational kind of legislative framework for this.
00:17:49:00 - 00:18:19:04
David
But but it's going to be a while before we see how effective those were. I still come back to, you know, this is a time for us to own our individual responsibility and to perform due diligence in what we're using and how we're using it, instead of running headlong into into the functionality race. Because functionality is great, but unless you can sustain it in a responsible and ethical way, functionality will do nothing for you.
00:18:19:04 - 00:18:33:02
David
In fact, it will kill you. So I, I just, I hesitate to put everything at the feet of regulators. I think we all have a responsibility here.
00:18:33:03 - 00:18:36:07
Martin
Okay, I.
00:18:36:09 - 00:18:37:15
David
Yes, I know.
00:18:37:18 - 00:18:43:05
David
I'm optimistic. I'm assuming that everybody is just going to do the right thing. They're not. We know that. And so they have.
00:18:43:07 - 00:18:59:00
Martin
And I do feel like sometimes we're like children when we're with AI, we get our hands on AI and we're like children. We forget all the good things we learned. And AI says, hey, come over here, look at this great hot stove. And we put our hands up and go, oh yeah, that burned. But let's keep going towards the hot stove.
00:18:59:00 - 00:18:59:19
David
Because I'll.
00:18:59:19 - 00:19:04:17
David
Agree with you on this. And, you can put that on your, on your meter.
00:19:04:19 - 00:19:05:12
David
Excellent.
00:19:05:14 - 00:19:21:06
Martin
That's two now sweet. One of the things that we ask all of our guests before we wrap up the show is, what's the one thing that engineering organization, engineering teams are getting wrong today about the software they deliver.
00:19:21:07 - 00:19:53:18
David
That teams are too willing to embrace untested approaches, and they are not bringing forward the tested approach that they have grown up with. And that means that they are they are not using a disciplined approach into what they're assuming that the technology they're using is building the discipline in. And unfortunately, it's not. So I think the mistake is just making too many assumptions about the technology that we're using.
00:19:53:20 - 00:19:54:03
David
Well.
00:19:54:04 - 00:19:54:19
Martin
And we know.
00:19:54:19 - 00:19:55:01
David
What.
00:19:55:06 - 00:19:58:02
Martin
Assumptions does to you and me.
00:19:58:04 - 00:20:00:23
David
Exactly. Well.
00:20:01:00 - 00:20:08:02
Martin
Thank you so, so much for stepping and and being a much better looking Adam.
00:20:08:04 - 00:20:13:20
David
No, no, no, I you know, I don't have the I can't compete. I'm sorry.
00:20:13:22 - 00:20:28:06
Martin
So that ship to you by harness. If David's point of view was was not yours then subscribe and tell us why. Until next time, stop talking. Start shipping.