From Cost Center to Business Driver: Making Security a Strategic Asset

Episode 2 October 30, 2024 01:16:11
From Cost Center to Business Driver: Making Security a Strategic Asset
Security Program Transformation Podcast
From Cost Center to Business Driver: Making Security a Strategic Asset

Oct 30 2024 | 01:16:11

/

Show Notes

In this conversation, Robert Wood, CEO of Sidekick Security, interviews Tyler Healy, CISO of DigitalOcean, discussing the evolution of security leadership, the importance of security as an enabler for business growth, and the dynamics of building a security team. They explore the challenges of engaging with customers, fostering internal relationships, and the balance between security and usability. Tyler shares insights on incident management, materiality assessments, and the significance of understanding how a business makes money to effectively align security initiatives with organizational goals.

Takeaways

Chapters

00:00  Introduction to Security Leadership
06:02  Navigating Security as an Enabler
09:56  Building a Security Team from the Ground Up
15:54  Engaging with Customers and Stakeholders
20:00  Fostering Internal Relationships for Security
24:03  Product Security and Developer Enablement
29:59  Balancing Security and Usability
36:03  Incident Management and Materiality Assessment
42:04  The Role of Availability in Security
48:01  Third-Party Risk Management
53:51  Transforming Security into a Business Enabler
View Full Transcript

Episode Transcript

Robert Wood (00:00.861) Hello everyone, this is Robert Wood, the CEO of Sidekick Security. And today I am joined by Tyler Healy, the CISO of DigitalOcean. And I'm gonna turn it over to him so as not to steal his thunder around his introduction. But today we're gonna be talking about everything from security enablement and digital trust and kind of how you work that into a product setting. And I'm super excited about this. And so Tyler, take it away sir. Tyler (00:13.713) introduction. Tyler (00:28.702) All right. Thanks for having me on. Yeah. As you mentioned, I'm the Chief Information Security Officer at DigitalOcean. We're a cloud infrastructure and platform as a service provider. I've actually been with DO for six and a half years now. So when I started, it was still a bit of the Wild West startup days. And now we've gone through IPO to kind of our next arc and journey in enabling cloud for businesses and developers around the world. Before DO, spent some time in the federal defense space and then some time in the private sector consulting land, primarily focused on telecom technology and biotech businesses. So kind of crossed a bunch of different lands before getting to DO and it's been an exciting journey helping to get DO to where it is today in building the security team where we cover trust and safety, core security issues, privacy. enablements and actually have started pulling some of the core IT and business systems as well into the security fold. Robert Wood (01:37.263) I love it. And before I jump into the meat of the topic, one thing that is also one of the things that I want to do with this podcast is interview and have a trade off between public sector leadership and private sector leadership. you know, mean, myself, I went through that journey. I started in startups and then I went, I did the whiplash thing and went into big government. And then I came out and now, you know, not in that anymore. And Tyler (02:02.478) Yeah. Robert Wood (02:06.749) So for you, how do you feel like your background in consulting, which is you kind of drink from the fire hose and see everything, going into the public sector space, which very regulated, very mission oriented, and now here, how do you feel like that has served you from a perspective or like a leadership experience standpoint? Tyler (02:29.416) Yeah, definitely got a lot of different perspectives and saw a lot of things that I liked and a bunch of things that I didn't like as well in terms of how security teams operated. I certainly started to shy away from the heavily regulated spaces for sure. I spent enough time with pharmas and learned enough about GXP and how that actually can sometimes negatively impact a security program because once you've actually built something that's producing a drug or maybe it's a piece of technology or equipment that's been certified the numerous ways that you need to in order to be able to bring it to market. It can be incredibly difficult to change and incredibly expensive to change. And we know security is all about being able to change sometimes as quickly as possible. So you get these competing environments where the regulatory component of it, which is intended for safety, Robert Wood (03:16.081) Yeah. Tyler (03:27.064) can sometimes make things even more unsafe, at least from a cybersecurity standpoint. Robert Wood (03:31.921) Right, it's like compliant to the neglect of everything else. That's the only one you can't waiver on. Tyler (03:36.288) Yeah. Tyler (03:40.78) Yes, yeah. And even in some of the security regimes, we see that too. the dozens of different ways that folks have to be compliant. I mean, I grew up in an era of the federal defense space with a lovely thing called Gold Disk. I don't know if that rings a bell for you at all, but like just the entire process of... Compliance checkbox in your way through an operating system through multiple hundreds of controls now obviously with our hardened environments now It's much easier to scan for those things but back then it was so slow it almost feels like you were putting the the the Compliance mindset over the actual security mindset and that was something I learned really early on that I did not did not like too much Robert Wood (04:29.649) Yeah, appreciate, feel like that, you know, learning those things that you don't like, or that, really, it's like, know, the things not to do, the things you don't want to replicate all that are just as important as, know, the things, those like mental models, you really want to latch onto it and build on. Tyler (04:35.874) Yeah. Tyler (04:42.04) Yeah. Tyler (04:47.086) Yeah, 100%. I mean, lots of good to learn in the public sector environment as well, too, right? I mean, there's just a ton to take away in terms of the way people think about sharing information and controlling information flows and who's ready to what scenarios. And there's just a whole bunch of good operational security side of things that the Gov side does extremely well. But being nimble isn't always one of them. Robert Wood (05:14.717) No, So my deputy always used to say like, you know, we're in the business of the government and like, you know, it's not like a fail fast and break stuff kind of environment because there's real stuff that are bigger than pictures getting leaked or something that is on the other side of it all. Tyler (05:28.536) That's right. Tyler (05:37.324) That's right. Yep. Yep. And as long as it's, if the programs are built in a way and funded in a way that allow for that change at the pace of what we're learning about the threat landscape and the attacker landscape, then that can work. That can still work really well. Yeah. Robert Wood (05:53.703) Yeah. Well, so I'm going to, I want to kind of use that like change at the pace that you need to change thing as a lead in here, because I love that. Now, one of the, one of the big things I want to focus on today is how you see and how you are kind of navigating and working around this, this dynamic of like security as a traditional cost center. Tyler (06:02.866) Yeah. Robert Wood (06:23.149) in an organization and how you are working towards effectively enabling DOs strategy, mission, growth, customer trust, et cetera, like using security as a means as opposed to this kind of checkbox end. Tyler (06:24.909) Yeah. Tyler (06:34.946) Yes. Tyler (06:40.076) Yeah, yeah, yeah, exactly. I mean, I think you nailed it with the word customer in there. It's just about as getting as close to the customer with the security program as you possibly can. And there's going to be. Graduation in there, right? Even within an individual business and within the security program, there's going to be the stuff that no customer is ever going to really care about. So how well we do our user access reviews for SOX compliance. They don't care. They really don't care. I mean, they want to know that we're a compliant business and that from a public company responsibility standpoint, we're doing what we need to do. But that's not going to change their mindset or their attitude about DigitalOcean as a business in the market. But what is are the security capabilities are in front of them. Some of the more public facing compliance regimes that they need to be able to comply with in order to serve their customers. You know, in my particular case, in the experience with DigitalOcean, the fraud and abuse and trust and safety side of the world was probably the first wedge that we built into bringing security out of this sort of like obscure cost center compliance area into something that was really, really meaningful for the business. because a lot of the security capabilities that we built for fraud and abuse and trust and safety have immediate impact on the customer experience, both potentially very good and also potentially very bad. There was one experience probably 18 months into my tenure at DO where we accidentally shut down a customer that had been part of our startup program called Hatch and did it in a way that was not particularly customer friendly. was a couple of big misses in terms of just human doing human things and a couple of errors that happened in that space, but it ended up being a widely visible public event where I think the register headline was something along the lines of like Digital Ocean drowned my business. that was, Yeah, and it was just a, it was a mistake in terms of. Robert Wood (08:51.815) Yikes. Tyler (08:57.39) some of the anti crypto mining technologies we'd put in place really to help our customers. Because the more we could reduce the fraudulent crypto mining on the platform, the less problem we would have with something called noisy neighbor, which takes CPU cycles away from good customers because it's a bad customer on the same piece of hardware potentially mining against it. yeah, that piece of automation and a couple of manual follow ups later going rogue. we were battling a pretty big PR situation. For better or for worse, those formative moments were really piece of how we recognized just how important security was to the ability for us to grow and for our customers to grow in a safe and trusted way. Robert Wood (09:42.705) Yeah, I love that example. when you refer to building this function, what does that mean in this context? How did you structure your team? Was there certain processes that you had? And maybe even more importantly, who are the stakeholders, like your peers? Maybe it's PR, legal, sales. Who are the peers that... Tyler (09:56.813) Yeah. Tyler (10:05.506) Yeah. Robert Wood (10:10.279) were most important to you in facilitating that. Tyler (10:15.094) Yeah, I mean, so from a building the team standpoint, it was pretty close to ground up. I started at the same time as a peer of mine who is now the senior director of security operations. He's part of my overall security leadership team. He and I actually started on the same day. We, came in as the director of security engineering. He was starting as director of security operations. There was CISO at the time and about four other people on the team. It was, it was really quite small for a. a cloud business. mean, not entirely surprising given where the business was in terms of its arc, was still relatively small, scrappy startup, but it was definitely the right time to get things moving in terms of a scaled security program that could cover all the various challenges of running a public cloud. so, yeah, we started by pulling in the trust and safety team, which had been part of the support function, actually. Robert Wood (10:51.197) George. Robert Wood (11:11.165) So it was an existing team that came in here. Okay, got it Tyler (11:13.32) As an existing team, yeah, exactly. So there was a couple of different teams. There was the trust and safety team, which was really just part of the support function. You know, we would get inbound complaints into our abuse inbox or legal ad inbox or other places where we would have high volumes of things are bad coming from your platform and you need to do something about it. And that was all just put through a. hands-on keyboard ticketing system in which our trust and safety teams were managing the problem with headcount, really. That's how they'd scaled it. So we saw that, brought that into the fold, because really the trust of our platform and the outbound abuse from our platform is such a big component of how we're viewed and trusted amongst our customers and others on the internet. And so that's absolutely a major security component of our platform. The other thing that we did is stood up a dedicated engineering team. And I'm not talking about just like security engineering in the traditional sense of knowing security tools and technologies, but a team that could actually build software and code. Yeah. And so I think that's one of the things that we found to be very successful is to be able to feed ourselves in terms of building and scaling automation for detections and Robert Wood (12:24.071) code. Tyler (12:36.408) particularly again in that trust and safety and fraud and abuse space, we were able to own our own destiny by having the operations in one side and the engineering team on the other working closely together. That's not always perfect, but having those two teams next to one another has been huge in our ability to ship code, build automation and do things at scale. I we still run a fairly small ship when it comes to the number of head count we have in the trust and safety fraud and abuse and security operations world. it's definitely enough to, get the job done, we pretty much entirely scaled based on automation. Robert Wood (13:14.749) Got it. And I feel like that one of those like misnomer, misnomer things in the industry right now is we use the term security engineering all the time. And I've found in most places that are not technology companies, basically what that ends up meaning is they're more like system administrators from a security standpoint. You know, they're like configuring and tuning, splunk clusters and setting up Tyler (13:37.218) Yes. Yeah. Robert Wood (13:44.199) kind of integrations and things like that, but they're not, and they might be like scripting is part of that, but, and this is a whole nother probably rabbit hole that I may or may not want to go down today, maybe not, the, but you know, there's this, like we talk about security people coding and it's in many places synonymous with just like basic scripting. Tyler (13:49.986) Mm-hmm. Tyler (13:58.755) you Tyler (14:11.446) Right. Robert Wood (14:12.133) And that is very, very, very different than like writing, maintaining, operating like enterprise software or services. Tyler (14:20.866) Yup. Yup. That's exactly right. And I've found the same thing, the ability to, I don't know, run your Nessus environment to be able to scan. There is an engineering skillset there, but it's not... It is. It's just very, yeah, it's very different. you're right. mean, we, on the Digital Ocean Security team, we run what we call... Robert Wood (14:36.566) Yeah, it's Tyler (14:46.966) like SEV1 services, meaning that if it were to go down, it would cause customer outage, customer visible outage. Robert Wood (14:52.957) You've got SREs, you're doing monitoring. Tyler (14:57.379) Yeah. yeah, we are plugged into all of that. So like even our vault cluster, mean, you sort of consider that to be some system administration, but all of the stuff that sits around it to make it scalable and accessible and resilient, like multi-region deployments, load balanced, reads out of the environment, even we run our internal LDAP cluster as well. the entire identity system is a combination of LDAP and Okta and how those two things communicate together and then scale out via deployments up to the production environment for access. Like those things all require actual building of services that are going to run and be scaled and not just do big IT sysadmin stuff. so, you know, again, some of that's going to be customer visible. And then some of those things are just going to be internal. We got to keep it running, keep the lights on. Yeah, it's definitely one of those things I think moved us away from just being viewed as an internal shared services cost center kind of situation into something that's a lot more customer connected and viewed as critical to the success of our customers. Robert Wood (16:11.207) Yeah, following question to that is, security teams are normally, unless you're being audited in some way, or form, you're normally not seeing the customer, visible to the customer. There's no bi-directional kind of visibility or communication that's happening there. That's probably being filtered through sales, customer support, other channels basically. So how did your team get plugged into those channels and like basically be able to deriving that insight instead of just, you know, assuming it. Tyler (16:51.015) Yeah, yeah, I mean we definitely connect with the customer on a regular basis. I mean, I probably talk to a customer once a week and it's a couple different ways that we did that. You asked before about the stakeholders and one is just building that relationship with the stakeholders internally so they know what are the capabilities that we have when a customer comes and asks about something related to security. And then the second piece is just there are some Unique things about running a public cloud where our security posture definitely does affect a customer directly. So, you know, things like dealing with IP reputation issues or, you know, maybe the customer is using our platform to do something that, that might get them blocked or something along those lines. Sometimes we have a lot of security companies that actually run on our platform and do testing off of our platform. We need to make sure that we're doing that safely. Robert Wood (17:25.935) yeah. Tyler (17:47.726) And so I think, you know, to your point about the, the stakeholder groups, you know, we've got marketing from the self-serve funnel standpoint, which is just, you know, how many we've 40, 50,000, customers that will sign up and start using our platform each month and managing the influx of those customers and what they're doing and how they're using our platform all the way down to the more, you know, white gloves style customer engagement where we're working with a large customer who might be. bringing a workload to our platform that they want to make sure is, is, operating safely and isn't going to cause them or us issues. yeah, I mean, that's a, that's a big piece of it. So I think, a lot of it comes back to that customer bruh buying criteria, which is that they care if they care a lot about security when it comes to choosing their provider. that's when I've seen it. It, you know, work well to make sure that there's a conduit between the security team and. You know, we can try and train this folks in sales and CSMs to, bring the right information to customers. But I found that it's. It's hard at least at our scale to have that knowledge really well baked across the team. they still, the CSMs and the sales folks still definitely rely heavily on the group we have internally called trust and governance. that helps bring the, the security messaging as part of the. the conversation to customers. So for example, if you go to digitalocean.com slash security, we absolutely had help in marketing launching that, but all of the content, all of the thought process put into it, you know, what kind of messaging we wanted to structure, shared security responsibility model, all those things all came from the security team with the, you know, partnership with our marketing folks, but it wasn't the other way around of, you know, the marketing folks trying to just. feel their way around in the dark sometimes when it comes to security. Unless of course you're a security products company, then you probably have that part pretty well figured out. Robert Wood (19:49.733) Yeah, yeah, yeah. Okay. Now, now the other thing, you know, so you guys are presumably grown past the, you know, four, four to five people, you know, Mark. So as far as those internal relationships go, how, how are you working with your, your team and getting other people to go out, build bridges, you know, is it part of like their goals? Is it just kind of an internal culture? Tyler (20:00.773) yeah. yeah. Tyler (20:17.941) yeah. Robert Wood (20:19.917) That way, know, everything's not riding on you and you know, you don't have the Tyler bus factory or lotto factory, depending on your general outlook on life. Tyler (20:29.494) Yes. Tyler (20:33.791) Yep. Yeah. No, it's, it's, it's been actually part, it's both really, it's, actually something we have to continue to reinforce. but it's also, it's also part of the culture that we built when we started. like I used to say that I wanted to build a relationship based security team. That's just kind of how I would frame it to the folks on our team and to the other stakeholders that we would work in the base business is because. Robert Wood (20:57.405) Yeah Tyler (20:59.862) After spending a lot of time in different sizes and shapes and styles of businesses and in the public sector space too, I learned that you cannot do security alone. Everybody says security is a team sport. I know that's probably something that's been used way overused at this point, but it is true. But I definitely knew heading into this that it had to be a relationship based security team. And so just spend a ton of time figuring out how to make the other stakeholders successful in what they want to go do. And so for, you know, our marketing team, they are incentivized to bring more and more and more and more customers onto the platform, which makes it difficult for our capacity team. When the capacity team is responsible for making sure that our hypervisors are available and clean and free for good customers want to come use them. And sometimes those two things were at odds with one another with security kind of right in the middle. And so figuring out how to make both groups successful, how to measure it and how to prove it to them that their lives have gotten better over time because we've done things like, you know, put good fraud detections in the final funnel in place or build more advanced crypto mining detections to help get bad workloads off of the hypervisors faster. I think it was figuring out and pinpointing where each one of those teams was like how they were motivated and making. them successful in their goal, which in turn made us successful in what we wanted to accomplish. Robert Wood (22:34.193) Yeah, I love that. I love that. so one thing that I ran into a lot at CMS was this, there were silos, of course. Like there's always silos in any organization. And one of the ways that we pursued breaking down those silos was really trying to make sure that teams were, that we found points of alignment between teams. So, Tyler (23:03.554) Mm-hmm. Robert Wood (23:03.963) instead of kind of reinforcing this like trope of everyone has to collaborate with everyone or we're going to run a flat organization or whatever, badgeless culture, et cetera, et We tried to be really intentional around the flows of like, call them value streams, call them whatever, where people, where there were real points of necessary collaboration. And that included Tyler (23:14.188) Yep. Robert Wood (23:33.905) within our team, but then we included outside stakeholders where there was handoffs. Like incident management was a good example of that. There's operations, there's us, there's privacy, there's potentially general counsel, there's, in our case, we had like the office of legislation and they were the ones dealing with Congress. And then we had the chief operating, kind of administrator's office where the political sat. Tyler (23:35.114) Mm-hmm. Tyler (23:50.413) Yeah. Right? Robert Wood (24:03.409) And so there was, like, we really had to be intentional around ironing out these kind of like flows where people were intentionally collaborating. then at the point of like two teams interacting with one another, figure out what the, like the best way to create alignment between those two were so that way they were not at odds. And so as the... Tyler (24:18.166) Yeah. Tyler (24:29.154) Yep. Robert Wood (24:32.167) The long-winded way of backing into a question here, with that context in mind, is do you find that there are certain teams or departments or functions where it is easier to find that alignment, the alignment of incentives over others? Tyler (24:52.942) Wow. Yeah, that's a good context and a great question in terms of like thinking through the, yeah, no, it's the organizational dynamics. Absolutely. And I mean, there are moments when it's got to be process driven. think especially in incidents or other circumstances where there's not a lot of room for error or miscommunication. Like you want that to be process driven. And then there are other pieces where you really want it to be. Robert Wood (24:58.011) that hit a point. Tyler (25:22.606) you know, finding that common ground more naturally and aligning these incentives in the right way. So I'll give you an example. like the security design review process that we have in place when a team wants to bring a new product to market. Robert Wood (25:37.117) Okay, it's like a wall of a new thing. Tyler (25:40.49) Yeah. So like, you know, just product design life cycle stuff where like from the very inception, I guess it's, it's really at the, this stage where we're, you know, proving out, potentially a technical architecture. They're doing an RFC on a new, new architecture, new product. there's, there's security engagement across the entire lifespan from when we start thinking about what, what a product should look like. and how we'll integrate for customers through the point in time where we're delivering it and potentially testing it as it's going live. That's pretty intentional. like, we try not to use a lot of room for error in that one. We still have to make prioritization decisions. We don't have a large enough product security team to do every single nook and cranny feature that might be coming through the development pipeline. But a lot of the big stuff, we want to make sure that we're... doing what we need to do to launch a secure and safe product for our customers. And we don't want to miss those steps. And I got to be honest, it really is dependent on the team. Sometimes it feels like we are forcing their hand to come engage with us and we don't like that. We don't want that to be the case. We want it to all be heavily collaborative and to your point that you said before, like the words of a CISO or somebody else rattling around in their heads to say, let's make sure that we're launching this thing. safely. So let's go talk to security. But it's it's it is it is highly dependent. I've found that our partnership with marketing is actually incredibly strong, which is, think, not what you might expect or kind of unique. But that happens to be one of the strongest ones that we have. And then certainly in like the infrastructure space. But I think that the further we go up stack, And you start to think about the further abstractions away from infrastructure where teams are doing more heavily, just like software releases that don't really necessarily have hypervisor or capacity impact. It's easier for teams to forget that a lot of the primitives that they're looking at in building still need to die back to the right authenticity, capacity considerations, limits considerations, like how many, how much do want someone to be able to deploy? Tyler (27:59.85) even if it's something that we consider to be, you know, fairly, fairly safe. still don't want to run out of capacity, on any point in time. So, yeah. So I think that's probably if I had to pinpoint is, is where the further that we were going up stack, the more we have to make sure it's super intentional to engage. but we definitely have tight relationships across legal finance. Robert Wood (28:08.561) for sure. Robert Wood (28:19.141) Okay. Okay. Tyler (28:25.558) I mean, you know, a lot of the considerations for privacy all flow back through those organizations. And to me, that just feels like we're always locking arms to come up with the right solution. Robert Wood (28:36.795) Love that. Okay, now I'm gonna dig into the product security thing a little bit. Have you ever listened to any threat modeling talks by Jeevan Singh, by chance? Is that name? Tyler (28:43.053) Yeah. Tyler (28:49.066) I have, the name doesn't ring a bell, no I haven't, I don't believe, yeah. Robert Wood (28:52.315) Okay, so he used to run like a security engineering and threat modeling team at Segment, which was then bought by Twilio. He's over at Ripley now. He's a friend of mine for a while now. He's big in the OWASP community. And one of the things that he pushes, others like Adam Shostak and other folks far smarter than me have kind of championed this, Tyler (29:00.352) Okay. Yep. Got it. Tyler (29:08.31) Nice. Robert Wood (29:20.969) is this idea of a, like the developer led threat modeling or developer led security design. And, and this was something like back when I was at Sigital, this was a big kind of hotly contested topic of like how much lives out in the developer space where they are empowered through tools. Like maybe it's, it's secure templates or libraries or, you know, documentation and things like that or training. Tyler (29:27.17) Yes. Yes. Tyler (29:47.255) Yep. Robert Wood (29:50.405) and how much is run in-house and, you know, kind of delivered in this like as a service model. And cause you know, just, just that decision represents a huge, like those are, those are very different things. And if you wanted to say like, all right, we're going to start doing security architecture work as a security team, you know, you're kind of like stepping into this like bifurcation of like, all right, what direction are we going to go? Are we going to enable teams to do it? Are we going to do it ourselves? Or are we going to kind of. Tyler (29:59.491) yeah. Robert Wood (30:19.665) have a foot in both and maybe do it ourselves on mission critical stuff and enable teams to do it themselves so it's more scalable. so I'm curious, with that product development dynamic you mentioned, how do you guys think about enabling, and maybe it's beyond security design, because that's just one part of product security. You've also got the static analysis and the SAS, the DAS, the SCA. Tyler (30:42.496) Right. Yeah. Robert Wood (30:49.329) you know, throw all the acronym soup at it. like, so how do you guys think about like enabling those teams without having to be like stretching your products, your product security teams so thin that they're everywhere? Tyler (30:49.558) Yes. Tyler (31:02.176) Yes, that is a, I feel like a perennial challenge for every security organization. Yeah, the way, one of the things that we've done is we definitely have like the secure by design guidance that everyone can find. So we have a portal, an internal portal that's focused on engineering templates, if you will. mean, it's just like a lot of like, what are the paved paths for engineering? Robert Wood (31:06.678) Yeah. Tyler (31:29.832) And, we, we have a security portion of that. for things like, all of our internal service, all of our services use MTLS. So they're mutually authenticated across each other and you know, that uses taps into vault and our vault architecture. And so we have some really well-paved paths on how to go do that. And so we don't need to have a product security engineer sitting there with someone who's building a new service to say like, here's how you talk to and communicate to all the other services, the environment, because it's all. Right there, same thing with a lot of the LDAP and role creation and various app roles and that kind of thing. And so what we try to do is push as much as possible into those paved paths. mean, we just have a security engineering baseline document that says here are the, I forget how many categories, it's between five and 10 categories where it just says, if you're doing these things, and your feature is approximately about this size, you can probably start working on it and making sure that you're covering these areas. You could create a security design review, but we're likely to deprioritize it against something that's a much more complex launch. So we try to give our engineers as much self-service as possible in that space and then figure out where we want to invest from there. So a great example of that is we recently launched Robert Wood (32:41.703) Yup. Tyler (32:54.114) something called RBAC standard roles. So now on our platform, we have a set of five or six standard roles rather than the super complex custom roles, which we are building for some customer use cases. But we know that the majority of customer use cases, everyone has the same mindset in terms of like what they want the folks on their team to be able to do from a crowd operation standpoint against the environment and said, all right, let's be really opinionated about the standard roles that we want to bring forward. We dedicated. some serious product security engineering time into that space because it's business logic. And I think that's where we generally want our product security team to be spending a lot of time is when, from a threat modeling standpoint, it's the business logic that is most exploitable that we want to spend a lot of time thinking through it. It also maximizes the brain power and the engagement of those teams because if they're just kind of going through doing the product security, if they're just doing more of the check Robert Wood (33:41.649) Yeah. Robert Wood (33:52.253) to a scholarly. Tyler (33:53.016) Checklist stuff like we don't that's not great time spent. want them really thinking hard about what the business problem is How the customer is engaging with it what the threat model looks like when it potentially becomes exploitable and what are all those? corner case business logic things that can come up that might Bite us down the line and that often are the hardest to discover once something is launched Robert Wood (34:16.433) Yes, yeah, the business lot. feel like even doing something like a penetration test. That's always where the most interesting stuff is. Maybe you have some complicated chaining scenarios and whatnot that you can work through, and that's just more fun, but the business logic manipulation is good. I remember years ago when I was at Sigital, we were working for a Tyler (34:27.828) Yep. Tyler (34:33.297) Exactly. Robert Wood (34:46.503) financial service company, customer of ours, and they were launching this big new product. And there was a really subtle bug in it. They had a few vulnerabilities. They had username enumeration. You could register an account without email verification, stuff like that. it was like, okay, those are normally low risk issues, et cetera. But there was something that you could do. You could inject this, you could tamper with this. Tyler (34:57.762) Mm-hmm. Tyler (35:09.719) Yep. Robert Wood (35:15.705) one parameter in the signup flow. And it would basically corrupt the user in their back end user table. they couldn't, somebody had to manually go in and unscrew this thing to fix it. so, and basically we kind of worked out this scenario where you could imagine like, Tyler (35:19.636) Mm-hmm Tyler (35:27.429) wow. Tyler (35:34.242) man. Robert Wood (35:42.875) just signing up for a whole bunch of accounts without the email verification, basically corrupting all of them where their customer service team can't launch this product successfully. And it was like, you know, we're like working with the security team initially on like these low risk issues and they're like, hey, you know, like, okay, like, you know, what app doesn't have these things, these kinds of things. And, you know, you kind of like work your, crescendo your way towards the like, and this is like what it actually. Tyler (35:47.66) Yep. Tyler (36:03.885) Yeah. Robert Wood (36:10.813) could mean and it's like, that's not cool. We don't like that. Tyler (36:11.735) Yes. Exactly. Yeah. Yeah. That's, that's, that's a great example. I mean, we have one recent, very recently that's now fixed, but I think it was probably, I forget if we did the root cause for how long it had been a bug. but speaking of email verification, I mean, there was a bug that was allowing customers to upon registration, essentially send themselves to email verifications. and if you didn't click the first link first, and you change your email address and then read redid the email verification, it would, you know, you basically then have to confirm your emails and two different inboxes. One of what you might not even own or even it might not even be a real email address. But if you've changed it and then reset the verification before you completed the sign up stage and then click the confirm on the first one that you sent, it would actually confirm your email for the second. Robert Wood (37:09.533) So you have like weird kind of race condition thing. Tyler (37:12.18) Exactly. Yeah. And so that's one of those things where it's not a, you know, it's not a major bug, but it certainly was something that we wanted to get fixed as fast as possible because it was being exploited. People, we could see signups that were attached to email addresses that were like, that company is not on our platform. That doesn't look legit. And so we had a bunch of weird email addresses coming through the signup funnel that didn't really make sense to us. And it took a while to dig into this and eventually found that Robert Wood (37:31.644) Right. Tyler (37:41.324) fraudsters who were keen on trying to make it past our fraud mitigation components were manipulating their email sign up through the sign up flow, which was pretty wild. Robert Wood (37:48.285) Yeah, that's one like little bit of signal that you have in that whole, that whole process. Yeah, that's fascinating. Now, I want to ask one more question about this, like this kind of internal paved road process. And I'm going to try to connect the two, like, you know, some of the work that you were talking about with your marketing team and that, now, do you find that Tyler (37:57.196) Exactly. Exactly. Tyler (38:05.741) Mm-hmm. Robert Wood (38:16.903) Well, all right. So one of the things that I think is such an underrated set of skills and security is this whole idea of effective communication and storytelling and salesmanship, which is an icky word in our field. But effectively, you are presenting and kind of evangelizing in a way security inside of your organization, because security happened through other teams, not oftentimes by the security team. Tyler (38:27.565) Hmm. Tyler (38:31.459) Yeah. Tyler (38:42.53) Yep. Tyler (38:47.0) Exactly. Robert Wood (38:47.165) And so do you find that like when your team is working with marketing and kind of seeing how they do things, what they do, how they approach it, that there's a like a reciprocal benefit on how you guys engage and show up and evangelize internally? Tyler (39:05.594) Wow. Interesting. I think subtly, yes. I mean, I think that we oftentimes try to learn from marketing and our product managers. But I also would say that is something that I believe in so strongly. And so it's also something that I'm kind of asking my leaders to always think about again, in that relationship based security mode. It requires changing the mindset away from You know, luckily we didn't have this problem with DO, but the security being a team of no, right? I mean, that's historically been a challenge. and one that I wanted to make sure we addressed right at the outset of building the team. so I had a lot of leaders who have been super helpful and effective in doing that. fact, Tim Lisco, and Ari Kalfis, who are in our security engineering team. they presented it at OWASP this time last year of, Of exactly that of like security team, enabling their organization, not being in department of no, I forget the exact name of their talk, but something along those lines. was fantastic. It was really cool to see them get up there and not even just evangelize within DO, but out to the broader community about how we connect with our, our peers across the business. so it's, it's very intentional and I feel like it has to be. It's not one those things that comes naturally to a lot of us who. come from the security side of things, it's easy to be negative or super risk averse to the point of being difficult to engage with. Like there are a lot of pitfalls that come with having a security mindset. And if you can break yourself free of those in the right moments, still have them and still have that be your core ethos, but like break yourself free of those in the right moments to communicate effectively, I think is incredibly powerful. Robert Wood (40:58.523) Yeah, I think one other dynamic that I've observed over my own career is I think our field has, and I think this is slowly moving in a positive direction, but I feel like for the longest time we prized this somewhat abrasive rock star culture. you know, and then put the, you know, the DEF CON, like, you know, drop zero days, shame everyone for not being security con, you know, that kind of thing. And, and, and I think those folks got put on pedestals and that's what people aspired towards was this, you know, this, this kind of very technical, but somewhat kind of standoffish egotistical like personality. And, and I feel like in this, in this, Tyler (41:25.313) Yeah. Tyler (41:30.538) security problem, know, that kind of thing. Yep. Tyler (41:47.458) Yep. Robert Wood (41:54.513) security culture that you're trying to build, like you can't have that. You can't have the abrasive, hostile, standoffish security person, because nobody wants to work with that person. Tyler (42:04.94) No, yeah, absolutely. mean, it's, it's, there's still a ton of value in, in the folks who can do what you just described, which is come up with something so esoteric that they can be like that. Like it takes, it does take like a certain kind of person. It's not me. Like I'll say, I'll be the first one to say it. Like I, I don't have that, like I love to break things, but There's a certain limit that I'll be able to get to in terms of my level of dedication in breaking something that some people just have the ability to surpass and that they're going and they're going to figure out the like the most esoteric thing. It's wild, right? And so there's still value in that, it does take in order for the rest of the world and the rest of our companies to get that value, it has to be presented them in a way that they can consume it. Robert Wood (42:45.917) We'll the same cases and... Robert Wood (43:01.179) Yes. Tyler (43:01.246) and so I think that there, we all, both, you know, everybody has those roles there. And so I think we, we do still have a couple of folks within the security team that I would say lean more towards that, OG security mentality, which is, which is totally cool. And I, I, I love talking with them and especially in like one-on-one settings and just getting them going about whatever various topic it may be. but there is a bit of a ceiling at least from. Robert Wood (43:18.759) Yeah. Tyler (43:31.186) a, I wouldn't say a technical leadership, from like an organizational leadership standpoint. Because eventually you'll run into areas where someone it's just, you're not able to connect with someone to get the outcome that you want. Robert Wood (43:45.233) Right, well, and in the CISO role or VP, whatever it is, that organizational kind of, on the organizational ladder, so much of your work is not technical, it's not even strategy, it's strategy and technical, but it's very relational and it's yeah, it's. Tyler (43:50.829) Yeah. Tyler (43:59.955) Yes. Yeah. Tyler (44:10.53) Yeah. Robert Wood (44:11.101) It's all that other stuff. It's kind of like, have likened it to like a yin and a yang of skills that we need to be investing in. I mean, we do, know, conferences and trainings and all of that have just been dominated by technical content, which is super important. But if you're operating at like a hundred percent of 50 % and like 10 % of the other 50%, then like, you know, you put them all together and it's, it's not that impressive, but if you're. Tyler (44:16.193) It is. Yeah. Tyler (44:34.53) Yeah. Tyler (44:39.104) Right. Robert Wood (44:39.965) a little less technical, but you're an amazing communicator and you're friendly and approachable and you're reasonable and all those things. Your overall effectiveness is likely going to rise in a non-trivial way. Tyler (44:50.143) Yeah, yes. Tyler (44:55.692) Yeah. Yeah, completely agree. mean, when I'm looking at, you know, future leaders for Digital Ocean security space, I mean, it's the folks who do both well. I mean, I still really believe the technical side of things is a must. It has to be, but you have to have that balance. It's like you said, Yin and Yang, it's like it is a balance. I think like anything else in life that striking the right balance is the right way to find. success, mean to over rotate and, know, there's arguments to be made like you so big, but rock star mentality, right. Being an incredible guitarist or whatever it might be that just, you know, vaults you into the, the hall of fame. Like that's, that can be a goal. can be an outcome that you have to be. So incredible. Exactly. Yeah, that's exactly. There's no room for not being in the top 10 of whatever that field might be that you're talking about. And then the rest. Robert Wood (45:49.149) absolutely superb. Tyler (45:59.08) what success can look like is much more of striking the right balance and figuring out how to make others around you, successful rather than, bringing them down. Robert Wood (46:06.738) Yeah. Yep, yep, totally agree with that. Now, speaking of relationships, so Dio is a publicly traded company. So you're dealing with all the fun SEC rules and guidelines and stuff. So there's all the hubbub about the rules changes and whatnot. I guess it wasn't recent, but relatively recently. Tyler (46:25.473) Yep. Robert Wood (46:36.253) at least in near-term memory, especially around breach notification, incident management, breach notification, reporting of material risk. Now, what was it like for you, maybe for you and then some of your direct reports that were involved in this process to iron out and prepare for the... Tyler (46:56.867) Mm-hmm. Robert Wood (47:03.741) Like the communication side of incident management, because I mean, I, and I guess backdrop is like, I incident management was one of the things that roll up under me at, at CMS as it, as it does in most security teams and organizations. And there is like, what I observed was like, there was a lot of like very strong technical proficiency in terms of, you know, digging apart a ransomware attack or, or doing a log analysis or whatever. And then. And then as soon as something had to like, you know, kind of flip from this very technical thing into a more polished executive product that would go in front of Congress, it's like, you know, it was kind of like, you know, my, my kids building, like my four year old, kind of assembling, you know, his like makeshift car versus like, you know, building like one of these like technics sets that's like super glued together and like ready to go out, you know, down a track or something. And it's like, Tyler (47:43.616) Yeah. Tyler (48:01.408) Yeah. Robert Wood (48:02.191) One of them is gonna fall apart relatively quickly and the other one is gonna hold up a little bit better as it goes through the trials. so I'm curious what that process was like for you, specifically on the non-technical preparation front. Tyler (48:09.858) Yep. Tyler (48:18.796) Yeah, yeah. mean, good news is that we had a lot in place already. I think that the relationship that I have with the audit committee and kind of some of the established patterns we have around incident reporting. we had a security incident response process that during a major incident, we have a couple of different criteria. It's like if I would ever, if we'd ever need to write out to customers. you know, something along the lines of GDPR notifications or other situations where it would be customer visible or potentially hit PR, know, with, gradiation between various incidents. But ultimately it would result in, you know, me putting a fairly formal communication forward to the audit committee during that incident response process. Generally, it would hit at the end of the containment phase of our incident response process. Robert Wood (49:13.18) Okay. Tyler (49:13.326) to, to, to do that communication, cause that's also when we line up with our GDPR, 72 hour, reporting window timeframe starting. and so we at least had that paved path of, you know, me up through audit committee board level to, to start, which I think if any organizations don't have that, even if you are not a public company, I would like, that's the first get used to that and build that relationship and build that bridge because it's obviously an incredibly important one. So we already had that and then what we needed to do was just build out what we had as our materiality scoping committee and So we worked with legal we worked with internal audit work with finance to figure out I mean we already have a materiality threshold when it comes to us as a business overall It was more about what how do we know whether a security incident is reaching a level of materiality? Massive gray area. Yeah Robert Wood (50:04.061) I was gonna ask about that because I... Yeah, right. Very much not a prescriptive thing, and I guess nor should it be. And so there's all the talk about risk quantification and loss modeling and whatnot, and then there's more just criteria-based assessment. Tyler (50:15.534) No. Right. Tyler (50:25.646) Mm-hmm. Mm-hmm. Tyler (50:31.446) Yeah, yeah, we did a combination of both ultimately. Yeah, so we've got a workbook that we put together. It's technically owned by legal, but then it's signed off by myself and our general counsel after we've gone through a materiality assessment. It's about 10 different criteria, fairly high level, just to get to that. documented point in time where we say we believe this is material or not. And we, you know, we say our process can go back and change that because sometimes you learn something much later that, that is different. then we go back and change it. But what we did is we built that out, built out a process, had outside council look at it. and then tabletop it like two or three times and just, yeah, through about this time last year, through December, around two or three different scenarios, some of them based in real things that had happened to, to digital ocean previously. Robert Wood (51:12.989) I love that. Tyler (51:23.398) to come up with how we would have assessed materiality. And we've used it once or twice in live fire scenarios since December. Both of them were not even close to material, but we were doing it for good diligence anyway, just to make sure we had it. So I'd say, yeah, exactly. Exactly, yeah. So we've done it probably five times in total between... Robert Wood (51:28.678) Okay. Robert Wood (51:38.413) Yeah, I mean, it's build that muscle over time. Tyler (51:50.38) Tabletop exercises and real life fire incidents. Robert Wood (51:55.291) Okay. Now one of the, so last episode I talked with Joe Lewis, who's the CISO over at CDC. And one of the things that he experiences this, I experienced this was in the public sector, and I'm sure you did as well when you were in the defense space, is financial impact is not really as important. Tyler (52:05.474) Yeah. Tyler (52:23.629) Right. Robert Wood (52:24.157) You know, have these, like, so CMS is a trillion dollar organization. like the collective, yeah, I think they're operating projects. It's like 1.4 trillion or something like that. A lot of that's program money, of course, but still, it's just like, you come in and start talking to somebody about a $5 million risk if you tried to quantify it. And it's like, at that level, at that scale, it's like a rounding error. Tyler (52:31.34) Wow. Yeah. Tyler (52:37.932) my goodness. Tyler (52:52.28) Great. Robert Wood (52:53.179) And it's not to like be reckless about any, any kind of loss, but it's just not, it's not on the same level. so, so you end up instead shifting your conversation to talk about more qualitative things like, like mission impact or mission impact. You may be able to quantify a little bit in terms of like, you know, the people impacted states impacted, you know, whatever, or like reputational harm, you know, in our case, it was like, Tyler (53:07.842) Yeah. Yes. Robert Wood (53:20.943) Is the Washington Post going to write an article about something? Is Congress going to inquire about something? Those kinds of things got people's attention a lot more than a financial figure. And so I'm curious, given the relationship you have with marketing, you have a mission that's related to the stuff your customers are building on top of DO. And then you've got your own company's mission and goals, which are centered in part around growth. Tyler (53:22.816) Right. Tyler (53:29.976) Got it. Tyler (53:44.605) Right. Robert Wood (53:51.013) And so, you know, like the material assessment is good for, obviously abiding by regulatory requirements and such, but as part of your incident response process, do you factor in those other types of risk? And if so, how do you go about kind of approaching it? Tyler (54:10.974) Yeah, we do and I'm forgetting exactly how we have it framed in our materiality assessment But it's something along the lines of do we think that we will lose X number of top customers or are we at risk and what percentage of that risk? You know, is there a 50 % chance we're gonna lose our top 20 customers over something like this? Robert Wood (54:25.341) Hmm. Tyler (54:31.886) cause we have an enormous long tail of customers, right? So it's like 600,000 plus customers on DO at any given time who are, who are active. you know, just, it's a, it's a really broad scope. but really the focal point for us in terms of materiality determination is really that top five, 10 % of customers are even, even like two and a half to 5%. cause you know, if you're running your personal Minecraft server on DO, it's like. Robert Wood (54:31.997) Okay, I like that. Robert Wood (54:53.319) Gotcha. Tyler (54:59.798) We love you, but like, we're probably not considering you as material to our business if you were happened to leave. But, know, someone who's running a mission critical business on top of us that if they left would be material, you know, potentially a large customer loss. does tie back to that revenue loss standpoint. So we do still bring it back there, but there's that reputational impact too, right? Which is future revenue growth. there's. Robert Wood (55:05.714) Yeah. Robert Wood (55:21.468) Yes. Tyler (55:23.498) existing revenue churn, but then also the potential impact of future revenue growth absolutely is a part of the equation. And so, we definitely talk about it from the customer lens and standpoint. Like I was a customer DO, how would I view this? How would I guide my business? If I was a CISO, any different business looking at DO, how would I view this particular incident when I found it, you know, on X or wherever else that it might popped up in terms of, PR. Robert Wood (55:37.201) Yeah. Tyler (55:50.39) So that is part of the equation, not the easiest to quantify in terms of our assessment, but I think a really big part of it. Robert Wood (55:56.571) Yeah, you know, something subjective, but still, you know, kind of a directed, directed question. Out of curiosity, have you ever gone through the process of, I don't know if anything like this exists at DO, but having almost like an advisory board-like resource of maybe it's customers or, you know, maybe, you know, peers to like look at, look at an incident Tyler (56:00.792) Yeah. Robert Wood (56:25.615) even if it's a mock hypothetical incident in the tabletop and get their take on whether or not they're, let's say they were gonna be buyers or decision makers at a customer to get their sense of how you're calibrating versus the actual outside perspective. Tyler (56:37.304) Right. Tyler (56:45.57) Yeah, I don't think that we've done that specific to security incidents. We definitely think about that from a overall resiliency standpoint. And that's actually another really interesting gray area on the SEC guidance because I'm pretty sure availability is stated in their guidance around reporting materiality. And of course, if we had something that took a data center offline for X number of days or hours, whatever it be, we would report that regardless. But it's really interesting to think about it from that standpoint, because, what if there's an undersea cable cut that forces a bunch of late rerouting that creates a bunch of latency for our customers in Europe? I don't know. Just made that up. Definitely an availability impact that we have to think about that our customers are concerned about because all of a sudden, you know, we're, looking at a ton of late latency that might be impacting those customers business. But is that an SEC reportable event? I mean, it's an undersea cable cut. Like I don't like that. There's a very interesting gray area, but yeah, mean, to the question about customer advisory, certainly on the availability and resiliency side, I've thought about pulling together kind of an advisory board on the security side as well. But yeah, I think that the availability gray area is an interesting one on the guidance. Robert Wood (58:00.797) And I feel like availability is, know, like when you're in school, if you're, you know, to have to take any kind of like security course, you know, everyone like hammers down the CI. Like you get out into the real world and availability is like pish posh in so many situations. And, you know, it's, it's, I think one of the more interesting elements of the triad in the sense that it has, I think the most Tyler (58:10.338) The triad. yeah. Tyler (58:16.204) Yeah. Robert Wood (58:30.727) Well, integrity, I'd say if I had to rank them, would go availability, integrity, confidentiality in terms of like having the potential for cascading failures. Availability is obviously, I think it's a little more obvious is something's down, something else that's reliant on it is down. Integrity is a little more subtle, but has that potential. And I think like, you look at like the SEC guidelines and Tyler (58:41.708) Yes. Tyler (58:49.378) Yep. Robert Wood (58:58.621) And it is very focused on your organization, not necessarily the ecosystem of organizations that you're a part of. Tyler (59:07.791) Right, Yeah, what is your take? Is availability that is not related to an outside threat actor, is that a security issue? Robert Wood (59:22.001) I'll ask the questions here, sir. I think, I mean, that's a really interesting question. I would say, it's not necessarily a security issue, but I think it's something that you would maybe wanna report or at least disclose. If nothing else, from the standpoint of what is your Tyler (59:45.848) Yeah. Robert Wood (59:52.271) your organization's business continuity plan look like in relation to its downstream supplier needs? I was actually gonna ask you about this next is like, about your, so DO is a third party to many other organizations. Your third parties though, how do Tyler (59:58.349) Right. Tyler (01:00:02.37) Yeah. Tyler (01:00:12.877) Right. Robert Wood (01:00:18.417) they factor into your incident response process. Because I think this is also one of those. Tyler (01:00:22.074) Mm-hmm Robert Wood (01:00:25.137) this is one of those areas where it's so easy. I mean, I wasn't doing this intentionally, but wearing the Spider-Man shirt, pointing at each other, this is one of those classic who done it type scenarios where everyone's just blaming everyone else and it's so easy to just cast away blame in a third party failure scenario. And my personal sense is, Tyler (01:00:33.036) Yeah. Tyler (01:00:39.106) Yeah. Robert Wood (01:00:53.979) organizations need to, they need to kind of own and have like full accountability around like their operating processes in relation to their suppliers. like, like if, if, know, if you're using a tool, you own the configuration and all of that, that's what you own the monitoring and the setup and you know, the use of that tool. If it's something you're reliant on. Tyler (01:01:03.384) Yes. Absolutely agree. Tyler (01:01:17.666) Yep. Robert Wood (01:01:20.343) You should be factoring that into your BCP and all that to like make sure that if there's, if that thing goes down, you might like, you know, in the healthcare world, it was like the, change healthcare breach. like that ended up creating all these cascading failures and like, you know, kind of highlighted it underscored how poor business continuity planning is slash was for so much of the sector. And because like this supplier was unavailable and, and, and even when the supplier started to come back up. Tyler (01:01:30.402) Right. Tyler (01:01:43.03) Yeah. Robert Wood (01:01:50.375) there was all these business processes that were kind of tied into and had these relatively tight couplings to that supplier. And that caused all these failures. And that I think is, it's just such an interesting case study to be thinking about like how your organization really manages itself in relation to its suppliers. Tyler (01:02:13.682) Yeah. Yeah. I mean, for us, it's exactly what you said. Like there is no excuse. A supplier could go down and we have to have a plan for it. Yeah. I mean, there's just no, our customers won't care. They don't care if it's a, if it's a Ram issue that made a bunch of hypervisors die or if it's an undersea cable cut or, you know, if cloud flare was down or Robert Wood (01:02:22.301) You all set? Tyler (01:02:38.658) Like there are dozens of or, or, or that they just don't care about. want their service to work with us and us to be reliable for their business. so we absolutely treat it that way from a shared responsibility model standpoint, where it gets really interesting is from a security standpoint, one of the examples I've been using recently, to talk about potentially requiring two factor authentication for all customers. Robert Wood (01:02:45.149) Yep. Tyler (01:03:07.746) which I'm a big proponent of, but I know it can be a challenge to implement depending on, especially if you're in a heavily business to consumer environment where a little bit of both B2B and B2C, but it's something I'm thinking about very seriously, is the whole situation with Snowflake over the summer. Yeah. Robert Wood (01:03:12.285) Sure. Robert Wood (01:03:25.117) I was gonna bring that up. In relation to secure by design, but also in relation to what third party risk management should really look like as a CISO. And there's two really meaty things to unpack there. speaking of like two of a, there's, cause like you guys operate a public cloud. So like there's a zillion and one bells and whistles that you can kind of turn on and off. Tyler (01:03:37.39) Exactly. Robert Wood (01:03:54.883) know, tune and all that stuff. And, and, and yeah, like having a kind of default, you know, same, same deal with like the internal paved road experiences for the engineers. Like what does the paved road, you know, out of the box secure experience look like for the customers and how do you build that? Because, you know, customers like anyone are likely going to take the path of least resistance. And, you know, and the snowflake thing, it's like, Tyler (01:03:57.154) Yep. Tyler (01:04:07.16) for customers. Exactly. Yeah. Tyler (01:04:20.257) Yeah. Robert Wood (01:04:24.391) Snowflake probably carried some, I think they do carry some blame in the sense that like that wasn't a mandatory feature. It's like AWS allowing public S3 buckets by default for the longest time. And like all these people just getting burned, like sure, you know, they carry some amount of blame, but ultimately, you know, it's really on the shoulders of the people who didn't, you know, configure it properly. Tyler (01:04:32.003) Right. Tyler (01:04:40.3) Yeah. Tyler (01:04:47.266) Yeah. At the same time, though, and I think this is kind of the message that I was driving home even with our leadership team is nobody cares that it wasn't Snowflake's fault necessarily. It's still the fact that they, you know, were in this place where a bunch of their customers were going through a pretty hectic security time and they were the epicenter of that, regardless of whether or not it was explicitly their fault. that's Robert Wood (01:04:56.283) Nope. Robert Wood (01:05:11.889) and they could have done something and they did. Yeah, and that's not gonna shame them or whatever, but I think it's a good lesson to, they're like, all right, if we have, if we as an organization can be thinking about security meets user experience meets product goals, or what the outcome somebody is trying to do with a product, how do we approach that? Because... Tyler (01:05:13.644) Yeah, and they didn't. Exactly. No. Yeah. Tyler (01:05:32.216) Yes. Yeah. Robert Wood (01:05:37.809) Like oftentimes you have to buy in the security features and you know, it's the enterprise version and you're upselling and all that. Most businesses are like, I don't want, know, I can't, I'm not, know, five Xing my cost on this tool. Like get out of here with your SAML tax. Tyler (01:05:40.663) Mm-hmm. Tyler (01:05:52.652) Yeah, I was just going to say the SSO tax is one of my funniest ones, which I, yeah. Yeah. And I think it's, it's interesting because at least in our circumstances, there's a very large dependence on the shared responsibility model because we can put as many things in front of our customers as we can. And I want to do more. think that's one area that digital ocean has some room to go, which is to getting really novel security capabilities into the hands of our customers natively on the platform. Robert Wood (01:05:56.669) It is. Tyler (01:06:21.87) is a goal of mine. But at the same time, a customer can show up and ignore the fact that we highly recommend and even put it in these interface to use key based auth for droplet and pop in a 12345 password and that droplet is going to get that instance is going to get compromised within minutes of being on public facing internet, literally minutes. And then start dousing outbound or phishing or whatever it may, bad actors are going to do with it. We can't prevent all of them, but as long as we're making that, that you brought the paid path again, like that's exactly it. Just making it as user friendly and simple as possible to do the right thing for security. That's, that's the most we can do. Yeah. Robert Wood (01:07:05.553) And I think that that message really transcends like internal, external, personal. So like a kind of zooming in or like, you know, out of a work setting, like for a while I had bought the family version of like one password. And, you know, one password is fantastic, but I think it's a little more Tyler (01:07:10.816) Yes. Tyler (01:07:26.788) yeah. Tyler (01:07:30.669) Mm-hmm. Robert Wood (01:07:33.647) advanced, then a lot of like non-technical people are really willing to, willing to kind of work with, you've got the secret keys and it's like the little, it's a little harder to just like get it signed in on multiple accounts and just go. And, you know, if you're a non-technical user and, you know, I struggled to get my household, you know, in-laws and whatnot like rallied around using a Tyler (01:07:56.737) Yeah Robert Wood (01:08:00.473) using a password manager for a while until I changed to something that was a little more like user friendly and still, you know, was an upgrade in security, you know, but they would actually use it because like the usability drove the overall value that that thing was able to provide. And I see that that same dynamic very much exist in the workplace. Tyler (01:08:10.894) . Tyler (01:08:19.598) Yep. Tyler (01:08:25.772) yeah, it absolutely does. mean, knowing your employees, knowing your customers, knowing what they need to be successful with, is, is enormous. mean, a lot of, a lot of people talk about like heavy MDM solutions on, on phones. I think that we found that that's not even that particularly good in most settings. And it is in a lot of, a lot of circumstances, especially gov. think it's highly important, but like, there are a lot of private company work settings where, you know, having two phones doesn't really make sense anymore. mean, the zero trust architectures, I suppose, if we want to throw around some buzzwords have helped us get past that to a large degree, which is great. But like, yeah, I think that's exactly it. You have to know your customer. have to know exactly what they need to be able to be successful. think having a highly technical business within DigitalOcean has made us really successful in some circumstances. It actually kind of difficult in others because we have some like Linux power users who are like, screw you. I don't want to have anything else running on my endpoint in terms of exactly. Yeah. And I know that's the same for customers too. Right. So if I'm thinking about, you know, we're an all Linux cloud, you know, it's a, it's something that I would love to be able to offer to customers in terms of a simple and low. Robert Wood (01:09:32.069) and get your EDR out of here. Tyler (01:09:52.426) operational expense and complexity security for Linux platform that would allow them to quickly define and deploy what they want and just kind of get it out of the way. And the biggest problem is that, yeah, there hasn't been much of that in the market historically. I think it's getting a lot better. There are a few things that I've seen that have popped up recently, but like it's just not been a market historically. Robert Wood (01:10:14.511) Yeah, I totally agree with you, which then pushes you into a situation of like, right, is there even an option to build versus buy? And where do we kind of fall on that spectrum? because at your guy's scale, that would be a very potentially large buy decision. It's usually a really value-add thing, because it's not something you're getting out of the box with the competitive landscape. Tyler (01:10:24.14) Yeah. Tyler (01:10:34.721) Right. Yes. Tyler (01:10:43.394) Yep. Robert Wood (01:10:45.817) I'm a company trying to build a FedRAMP ready application or some mission critical thing that has all these compliance requirements. most of the time in a compliance ecosystem, at least from what I've observed, you've got your user workstations, they've got your EDRs, the server environment is just like the Wild West. We monitor it, we do configuration, like CSPM stuff, but Tyler (01:11:03.747) Yep. Tyler (01:11:08.97) Yes. Robert Wood (01:11:14.909) Beyond that, it's like server endpoints are kind of neglected from an asset protection standpoint on that level. I I mean that I could see that tension around that decision being there. Tyler (01:11:21.442) Yeah. Tyler (01:11:26.178) Yeah. Tyler (01:11:34.806) It is. Yeah. And I think it's an interesting one because I, you someone will just throw a CIS benchmark on a, a, on a, you know, instance and say, it's done. like, yeah, I think that there's a more to go in, in, this space. And what I try to imagine is how to make it as simple and as integrated and opinionated as possible. Cause at least for a lot of our customers, they may not even have a security team or a large security team or someone who's even dedicated to thinking about. Robert Wood (01:11:38.013) Totally. Tyler (01:12:05.032) I'm sure they probably have that responsibility internally, but they might not have a CISO with a CISO team. Some of them do, but a lot of them, especially in their early stages, don't. so how do you create the... nor should they, yeah, because it's going to be distracting from innovating on whatever they're innovating on and they'll never make it to market if they're just going to only focus on security of the infrastructure that they're building. So the more we can abstract away those complexities and do the thinking on our own first and then put it in front of them. Robert Wood (01:12:15.323) And more should they necessarily cause. Robert Wood (01:12:24.829) Mm-hmm. Yep. Tyler (01:12:34.818) In an opinionated way, the better, that's a, yeah, that's a tough nut to crack. And I think we're always thinking about that as we're bringing new products to market, like what are those security primitives that we can come built in? Robert Wood (01:12:45.627) Yeah. And, and, and I love the, I love the thinking or I love the term opinionated in this kind of, in this platform discussion, because that is, I think, I think when you start to shift away from being overly opinionated, it like you end up with just so much variance that you need to then manage. And that, that variability is like it's complexity that will end up either biting you long-term or Tyler (01:13:04.802) Yeah. Yep. Robert Wood (01:13:15.285) it's just something that is those kind of like you have to be hands off and just not manage it. And that then represents some amount of like residual risk. Tyler (01:13:24.982) It does exactly. Yeah. I mean, you could, you can configure just about any security technology out there to be so noisy that it becomes useless. And so, and then you can also do nothing with it and therefore it's not representing any risk mitigation. So I think that there is somewhere in between that allows us to say for these large bulk of use cases that we're able to do something that is reasonable and makes sense, provides a reasonable level of assurance. It's never going to necessarily be perfect, but I think that. Robert Wood (01:13:34.897) Yes. Tyler (01:13:54.892) that in those circumstances, especially with a fast growing technology company building in cloud infrastructure, perfect isn't the answer. Something is better than nothing in many circumstances. Robert Wood (01:14:05.425) Yeah, my God, yeah, I feel like our field, like the tagline for our field on some level, or like what I'd want to see pushed into like university settings is like, yeah, don't let perfect be the enemy of good, flash better. Tyler (01:14:19.342) 100 % yeah, yes, yes Robert Wood (01:14:22.663) And yeah, I don't know if you made this mistake, but like early in my career, especially when I was coming out of consulting, like I was so like drinking the Kool-Aid on wanting things to be like really secure. And like I was so invested in that and made so many just wildly stupid decisions. yeah, it's unfortunate. And I feel like now with time and Tyler (01:14:36.481) Yes. Tyler (01:14:45.419) Yeah. Robert Wood (01:14:52.015) and gray hair and, you know, kind of less emotional attachment to these things. can kind of see them for what they were and I've grown from them and learned from them. But I think that that is such a, it's such a prevalent tendency, you know, thinking about that risk aversion we talked about earlier to really want to make sure that something is secure, not like good enough or better. And Tyler (01:14:53.735) Hahaha. Tyler (01:15:08.854) Yes. Tyler (01:15:18.882) Yeah. Robert Wood (01:15:21.149) That's a hard pill to swallow. Tyler (01:15:24.138) It is. Yeah. And I mean, it's, the old conversation of like, we, of course, it would be great to have everything be unhackable, but we know that that's not how it works. And that can't be the goal. It's all about making sure that the, the, it's economics, right? The time investment put in should be, should, should be, you know, difficult enough. It commensurate with the value at risk. mean, yeah. yeah, I, I agree. It's, kind of funny reflecting on that. I feel like I've become. Robert Wood (01:15:43.805) to measure the thing that. Tyler (01:15:54.86) I wouldn't say more risk acceptance. I don't think it's that it's just better identifying when to really lean in on, on those moments of like mitigating or removing risk. And when, when, when not to, think it's to your point, it's just like, I would, you would expect someone who spends more time in security to become more and more and more more paranoid, but I've actually found myself being like, no, I've seen that a dozen times. It's really never going to not going to bite us. Like, let's not worry about that as much. Robert Wood (01:16:23.505) Right. Yeah, or if it does, it's not, you know, we've got a path forward. Yeah, for sure. Well, I want to ask you one last question. Now, thinking about this whole security as an enablement factor to transformation, to growth, to turnarounds, et cetera. Like what if... Tyler (01:16:27.99) We've got a path. Exactly. Exactly. Yeah. Tyler (01:16:47.448) Yep. Robert Wood (01:16:51.301) somebody was stepping into a new role or maybe, you you yourself in the future stepping into a new role. And that's the scenario that you're, that you find yourself in or that they find themselves in. Like what's the one big thing that you would recommend to that person in terms of like making sure that security is a part of that transformation story and not just, you know, kind of the, the, the, the distractor. Tyler (01:17:20.098) Yeah, it's probably going to sound somewhat cliche, but it's just true. Just figure out how the business makes money. Robert Wood (01:17:29.063) No, that's not cliche at all. I think, so now I love that because I think it's something that's never talked about. Like my deputy and like all the power to him, he's amazing, he still is amazing, but was amazing in the role that he was in. He had spent so much time at CMS and he had this this Tyler (01:17:43.224) Yeah. Robert Wood (01:17:58.429) presentation and this kind of organizational map of how CMS worked. The different centers, what they did, how data flowed, all that stuff. you can't really function effectively in your job if you don't know how the, what's happening around you is supposed to work, how it works. Tyler (01:18:04.515) Yeah. Tyler (01:18:22.668) Yeah. Tyler (01:18:27.372) Yeah. Robert Wood (01:18:27.399) you know, is the strategy based on, you know, is it based on like massive growth of users? Is it based on marketing? it, know, what's the, you know, what's the conversion kind of acquisition process, like all that stuff. then, and then you can kind of like figure out where to, you know, you've got, you've got a bundle of sticks that you can, you know, kind of spend on security. Where are you going to put them? Tyler (01:18:34.446) Exactly, exactly, yeah. Tyler (01:18:47.17) Yeah. Right. That's exactly it. mean, we talked about before the idea of incentives of your stakeholders, like who are the main stakeholders and how are they incentivized? And at the top level, what are the customers doing and how are they incentivized to engage with the business, which ultimately means how you're making money. So I was kind of saying that tongue in cheek, there's a lot of different like nuances to that piece of it, but it's really like, why do the customers care about the security program and how do they, how would they potentially view it? So, you know, for example, like you. don't necessarily want to show up to a life sciences business and say, all right, we got to do PCI compliance. And obviously, hopefully, no CSO would actually go to that. But it's like, unless you're collecting credit card data, you're probably going the wrong direction with that particular path. you think you have to really leave behind all the preconceived notions of wherever you've come from. And say, you know, use those learnings and experiences to guide, to guide your, team and your thought process. But the first question and really opening your ears to, after you've asked the question is, you know, how, how are we making money? Why do our customers care? Robert Wood (01:19:55.389) Yeah. Yep. And I like asterisk that with anyone who's in a nonprofit or a mission style organization, does the mission work or how does whatever it is, like why do you exist and how does that work? Yeah. Cause if you don't understand that you're gonna be in a, you're gonna be in a fun spot and it's not. Tyler (01:20:06.721) Yes Tyler (01:20:10.712) Right. Tyler (01:20:15.658) Exactly. Tyler (01:20:23.355) It's gonna be tough sledding for sure. Robert Wood (01:20:27.205) Yeah, for sure. Well, Tyler, thank you so much for the time, for the conversation. I thoroughly enjoyed it. And I hope that we get a chance to catch up again at some point. Tyler (01:20:32.526) Of course. Yeah, likewise. Yeah, for sure. No, thanks for having me on Robert. Appreciate it. Robert Wood (01:20:41.319) All right, thank you everyone.

Other Episodes

Episode 1

October 01, 2024 01:02:59
Episode Cover

Tech Debt, Compliance, and Strategy: A Deep Dive with the CDC’s CISO

Summary In this conversation, Robert Wood and Joe Lewis discuss the complexities of leading cybersecurity efforts within a large organization like the CDC. They...

Listen