Oh lordy, the "two crews" bifurcation fully written down. What a fantastic way to ship until it becomes far too expensive to ship anything good.
Look, when we break the feedback loop back to the people who wrote the software in the first place, they get happier for a bit, you make some other people sadder for a bit, and then slowly your feature crew never want to be interrupted or bothered again and your customer crew can't get enough resources to fully fix anything.
Worse, your feature crews aren't learning anything beyond how to get those lines out the door, which will somehow get slower and more expensive as time goes on. Why? Because you removed the one fitness function on good software development which is to fully re-incorporate the negative feedback back into the source of development.
A real CTO leadership handbook would say clearly "it's your responsibility to help your developers improve, especially while shipping, and they're not always going to be happy about it."
One way I try to get my head around things like this is to skip to a section I understand deeply and see what they said. Here, the claim is made:
Don't try to get a compliance certificate at the last minute. Preparing for and conducting an audit such as for PCI DSS or SOC 2 from start to finish is a lengthy process, ranging from six to twelve months for most startups. Starting early and maintaining compliance is cheaper than starting late and doing rework.
This is basically the opposite of the advice I would give a startup. SOC2 attestations in particular are easy to get, and are a waste of money to obtain preemptively before there are purchase orders on the line for them.
There are things you should start doing early that lay the groundwork for attestations, but you should be doing them anyways, even if you never plan to get a SOC2 (and if a big-ticket customer never demands it, you shouldn't SOC2). That's stuff like setting up single sign-on and having protected git branches; simple best practices.
Anyone else want to spot check other parts of this document? I wouldn't feel qualified to challenge most of it.
Great approach. I ctrl-F'd for databases, good info there generally. The only thing that gave me pause: a startup doesn't need to focus on SQL vs. NoSQL in 2025 with such good json support in the most popular SQL databases. Just use PostgreSQL or MySQL -- whichever your engineers have more experience with -- use CloudSQL or RDS which will take care of the hard stuff like backups and replication for you, use read replicas for BI with a good visualization tool, you'll be good with that for a good while before you need to fork over 5/6 figures for Snowflake or anything else.
> use read replicas for BI with a good visualization tool
Put up 2 or 3 read replicas, split your queries so writes happen to main and reads come from replicas (supported out of the box by many modern ORMs), and you can scale to millions in daily active users for most startup workloads.
Really the hard part of BI is that folks who need the info don’t wanna learn SQL. The ones who can do SQL, will struggle to keep up with your changing schema.
I give them Metabase. Metabase pointed to read-replica-3; and via Metabase API one can add lots of meta-data about tables and fields so the BI folk can point & click to build reports (and keep up with schema changes (which I mostly resolve with views anyway))
I was just commenting to a colleague recently about the significant improvements RDBMS have gotten for json support over the last decade. For instance, keys below the first level in Postgres jsonb fields were not indexable around a decade ago. Now you can do GIN index and other options that are rather sophisticated.
Agreed. I can't think of anything that would convince me today to use a document store over Postgres as the primary (or likely only) database. Most of the time JSON fields augmenting the RDBMS seems like the way to go.
Yeah, in my experience, most companies who are going to 1) do business with early stage startups and 2) want SOC2 report, are going to be totally fine with writing “startup X will get their SOC2 type 1 in the next six months” into the contract and moving forward, so long as someone technical can get on the phone with their IT people convince them you are reasonably competent.
I am new to compliance but this seems super strange to me. Based on my cursory read of SOC2 you need a ton of evidence gathering for months leading up to your audit. How wold you know what to retroactively have if you didn't spend time on it?
SOC2 attestations being easy to get also runs counter to what I have heard from every single other person on this topic. Generally what I hear is that it is extremely hard and time consuming. What am I missing? I would love to be wrong here and for this to be easy.
Using something like Vanta or Drata makes life a lot easier. I've done SOC2/PCI audits in fintech where we change tools every year (meaning we reinvented the wheel every year), and I've now done it at my own startup using Drata. Auditors feel more comfortable, you'll feel more comfortable, etc. Even if you're not planning on doing it right away, just sign up and have it start tracking your progress.
It's time consuming, but not all consuming. I think I spend <2 hours a week on compliance now that we're set up.
The "fun" part was engineering ways to implement things like PHI scanning and WAF protection as cheaply as possible. There's almost always a nearly-free cron job/python script/slackbot alternative to every "mandatory" 5-6 figure SaaS subscription in the space.
By all means use tools like these, but be very careful, because they (and auditors that use them) will lead you into engineering changes that are not required for SOC2 and may not be what's best for your team. For instance: there is absolutely no need to set up PHI scanning or a WAF to get SOC2.
Made an account just to say that I respectfully disagree solely when it comes to accounting and supply chain processes in an enterprise ERP. Unwinding un-auditable processes costs so much f’ing time and money while the business still has to run that I’ve found it to be cheaper and better to be auditable from day 1, in this one specific instance.
There’s being auditABLE and being auditED. Honestly I think the article’s take is smarter for a less experienced or skilled founding team and tptacek’s is better for a more experienced team. Paying auditors to look at screenshots and CSVs is a giant waste of money until it’s not, but at the same time, letting bad practice ossify until it’s expensive to remove is also a mistake.
I built one of the Trade Promotion Management platforms used in the NA market, and couldn't agree more. It's a nightmare trying to be auditable if you didn't think about it from the start.
I was at (insert infamous unicorn) and they spent a ton of money (all relative, the money meant nothing) but more importantly 18 months attempting to get SOX compliant and never made it, because running the business was too important. Of course it all came down to lack of leadership to enforce policies but even if we had it, it was objectively super fucking challenging.
When I do get a chance to implement compliant processes at the beginning, it’s one of those amazing IT things where we prevent WW3 but never get the credit for it.
> There are things you should start doing early that lay the groundwork for attestations, but you should be doing them anyways, even if you never plan to get a SOC2 (and if a big-ticket customer never demands it, you shouldn't SOC2). That's stuff like setting up single sign-on and having protected git branches; simple best practices.
This is in many ways the spirit of SOC2, no? There are a lot of startup founders, far more than I'd like, who would purposefully eschew such "simple best practices" unless they had an axe like a SOC2 audit swinging over them.
I think you're both right, for what it's worth, and my take is that you are more aligned with TFA than you perceive.
I'm pretty sure that's not what the author meant. Again: those are things you should do regardless of whether you're ever going to get SOC2 (and a lot of startups shouldn't).
The section on performance management is circular and vague: a good one is motivating and a bad one is demotivating. OK. Glad we got that out of the way.
The whole intro reads like a puffy resume and lots of gilding. Even a section of gushing testimonials.
And he puts his name on the title so you don't gotta read the author byline. Total cheese.
Having gone through quite a number of compliance audits... the one thing that is good in that advice, is that many items in an audit are just a checklist of questions, such as
do you have a policy for XYZ?
or confirm you have a process for "thing"
So what ends up happeneing is if you feel stressed about an audit, just getting a list of the audit, you will realize how much you can just say "yes" to and feel less daunted by the audit.
So, its a good self-check even if youre just crossing out the things you should have already have a framework for.
I like things like this not because I'm going to use it as a bible, but because it provides good coverage of responsibilities & concerns.
Unless you've got some great advisers or you worked under someone really great, no one's going to take you aside and give you a list of stuff you need to take care of once you're in this position.
For each section I'm asking - what's our answer to this? Do I agree with this? Is our process better? What have I missed? It's helpful.
I read all the time about folks who become a VP/CTO and stop coding. Management skills are not coding skills. I know it. But I can't for the life of me figure out why folks hang up their keyboards and let their first super power go to waste. You can be a technical CTO from start to finish. Treat your team and the company like a service that needs active contribution, maintenance, and on-call support; and also, get your hands dirty building by yourself and with your team.
I've also had the experience where the CTO was activly coding, 80% of the code base were theirs, and the company was hiring software engineers who could and wanted to fix up their stuff - there was this true luxury problem for this start-up: bad bugs everywhere, but patient and resilient customers. They found 4 willing engineers with good chemistry at first, at least up until they were constantly vetoed by the CTO in their decisions, because the teams best practices conflicted with the CTO-way of "getting things done" - it's a rigid hierachy after all, and not a democracy.
In my own experience it's a matter of only having that much time in the day. For 7 years I had somewhere between 20-25 people I was directly or indirectly responsible for. There was just not enough time to get anything useful done in the code and my time was much better spent solving problems that others couldn't. A few times I was able to pick up some really simple change just to get the experience first hand to go through all our processes and see where we can do better.
I always kept coding nights and weekends but it's just not the same and over time you are gonna get a little rusty. That said I greatly enjoyed getting my hand dirty all day during a sabbatical I'm taking.
When you're not just an IC, you have other priorities. That means your IC work can be derailed at any moment. _That_ means you can't take on work anywhere near the critical path or you're just blocking others or handing things off.
Reviews? Sure. Design meetings? Sure. But taking critical work will end up causing issues.
I don't think there's a right answer here, but there's definitely a point where your code contributions are much lower leverage than for example trying to recruit the next set of critical engineers, working on the technical roadmap to keep ahead of the competition, or making sure the engineering org is aligned with the rest of the company.
Any lines of code the VP/CTO could write, could likely be written by someone else on their team (and their team's quality could be even better) - but all the other items I listed is likely only something the VP/CTO could do the best at in the company. It's quite a rational decision to largely give up hands-on technical work for what's more important for your team and company.
For me, the hacker super power — the one worth carrying forward as you progress in leadership — is being able to prototype something that works and proves a concept.
Realistically, a proof of concept is also only 20% of the work an engineer needs to do in order for a change to become production worthy and I respect that my sketched ideas need a lot more care and craftsmanship than I have time to give them. Where I can help other ICs is having that initial 20% idea around which they can then build a working idea, and do so autonomously.
It feels very cringey to write — oh brave new world that has such people as me in it! — but I can easily reassure myself by remembering all the times earlier in my career where I was very grateful to be initially pointed, with quite a lot of prompting, in a particular direction and then being given the chance to deliver on it.
I’m just a lead, but I can imagine part of being a CTO takes the same form as what I’ve described.
My last CTO role (team of 40) had me absolutely over capacity from day one, and I am _good_ at time management. I would rather have been programming 50% of the time, but there just was no time, and no support structure in place I could hand stuff off to; I had to painstakingly build that, which was yet another reason I had no time.
I like the idea of continuing to code, but usually that’s not what you’re being paid for, and while I consider myself a very strong developer, they can be purchased for less than the CTO’s salary, rather than the more expensive CTO doing the work. FWIW I went back to IC after a few years and plan to stay that way for the rest of my career.
Has anyone worked in a "two crews" system where there wasn't resentment? Or where people didn't want to naturally migrate to the "future crew".
I like the idea of this on paper. I have a hard time believing it can work in practice. The closest I've seen are library teams that build some service (say a design system + components) that other teams utilize.
At my last job we had a version of our on-prem product where the company sold a super extension for a version that was supposed to go out of support. We had a small team (I think three people) whose only responsibility it was to ensure that version was supported, pipelines worked and we're ready to ship a big fix at needed. That was their responsibility but as long as that was covered they were allowed to work in their spare time on what they wanted as long as they saw value for the company in it. It was a good bargain. Everyone else was grateful the small team was doing the dirty work and the maintenance team was delighted they could use the remaining time working on things that had always bugged them, do research, etc. This was a few years ago and I forgot details but I vaguely remember that we got some really cool improvements and research from them. The people on that team also were really excellent and self-motivating which helped make this a success but also more expensive.
I am not sure this is great advice. I am sure it makes sense for certain things, like UI or animation. But, generally reading text is more efficient than video watching. It's also easy to seek to the important parts of a text than a video when you are in a hurry.
Its interesting that the original HN discussion talked a whole lot about recorded meetings. Feels like another world just a few years later with AI transcript embedded in every video conference.
Oh lordy, the "two crews" bifurcation fully written down. What a fantastic way to ship until it becomes far too expensive to ship anything good.
Look, when we break the feedback loop back to the people who wrote the software in the first place, they get happier for a bit, you make some other people sadder for a bit, and then slowly your feature crew never want to be interrupted or bothered again and your customer crew can't get enough resources to fully fix anything.
Worse, your feature crews aren't learning anything beyond how to get those lines out the door, which will somehow get slower and more expensive as time goes on. Why? Because you removed the one fitness function on good software development which is to fully re-incorporate the negative feedback back into the source of development.
A real CTO leadership handbook would say clearly "it's your responsibility to help your developers improve, especially while shipping, and they're not always going to be happy about it."
One way I try to get my head around things like this is to skip to a section I understand deeply and see what they said. Here, the claim is made:
Don't try to get a compliance certificate at the last minute. Preparing for and conducting an audit such as for PCI DSS or SOC 2 from start to finish is a lengthy process, ranging from six to twelve months for most startups. Starting early and maintaining compliance is cheaper than starting late and doing rework.
This is basically the opposite of the advice I would give a startup. SOC2 attestations in particular are easy to get, and are a waste of money to obtain preemptively before there are purchase orders on the line for them.
There are things you should start doing early that lay the groundwork for attestations, but you should be doing them anyways, even if you never plan to get a SOC2 (and if a big-ticket customer never demands it, you shouldn't SOC2). That's stuff like setting up single sign-on and having protected git branches; simple best practices.
Anyone else want to spot check other parts of this document? I wouldn't feel qualified to challenge most of it.
Great approach. I ctrl-F'd for databases, good info there generally. The only thing that gave me pause: a startup doesn't need to focus on SQL vs. NoSQL in 2025 with such good json support in the most popular SQL databases. Just use PostgreSQL or MySQL -- whichever your engineers have more experience with -- use CloudSQL or RDS which will take care of the hard stuff like backups and replication for you, use read replicas for BI with a good visualization tool, you'll be good with that for a good while before you need to fork over 5/6 figures for Snowflake or anything else.
> use read replicas for BI with a good visualization tool
Put up 2 or 3 read replicas, split your queries so writes happen to main and reads come from replicas (supported out of the box by many modern ORMs), and you can scale to millions in daily active users for most startup workloads.
Really the hard part of BI is that folks who need the info don’t wanna learn SQL. The ones who can do SQL, will struggle to keep up with your changing schema.
I give them Metabase. Metabase pointed to read-replica-3; and via Metabase API one can add lots of meta-data about tables and fields so the BI folk can point & click to build reports (and keep up with schema changes (which I mostly resolve with views anyway))
Definitely - I've been surprised at some very complex pipelines built with pandas, etc. because someone didn't want to use SQL...
I was just commenting to a colleague recently about the significant improvements RDBMS have gotten for json support over the last decade. For instance, keys below the first level in Postgres jsonb fields were not indexable around a decade ago. Now you can do GIN index and other options that are rather sophisticated.
Agreed. I can't think of anything that would convince me today to use a document store over Postgres as the primary (or likely only) database. Most of the time JSON fields augmenting the RDBMS seems like the way to go.
Yeah, in my experience, most companies who are going to 1) do business with early stage startups and 2) want SOC2 report, are going to be totally fine with writing “startup X will get their SOC2 type 1 in the next six months” into the contract and moving forward, so long as someone technical can get on the phone with their IT people convince them you are reasonably competent.
I am new to compliance but this seems super strange to me. Based on my cursory read of SOC2 you need a ton of evidence gathering for months leading up to your audit. How wold you know what to retroactively have if you didn't spend time on it?
SOC2 attestations being easy to get also runs counter to what I have heard from every single other person on this topic. Generally what I hear is that it is extremely hard and time consuming. What am I missing? I would love to be wrong here and for this to be easy.
Using something like Vanta or Drata makes life a lot easier. I've done SOC2/PCI audits in fintech where we change tools every year (meaning we reinvented the wheel every year), and I've now done it at my own startup using Drata. Auditors feel more comfortable, you'll feel more comfortable, etc. Even if you're not planning on doing it right away, just sign up and have it start tracking your progress.
It's time consuming, but not all consuming. I think I spend <2 hours a week on compliance now that we're set up.
The "fun" part was engineering ways to implement things like PHI scanning and WAF protection as cheaply as possible. There's almost always a nearly-free cron job/python script/slackbot alternative to every "mandatory" 5-6 figure SaaS subscription in the space.
By all means use tools like these, but be very careful, because they (and auditors that use them) will lead you into engineering changes that are not required for SOC2 and may not be what's best for your team. For instance: there is absolutely no need to set up PHI scanning or a WAF to get SOC2.
I posted two guides downthread. It's hard because people make it hard, or let people make it hard on them.
Made an account just to say that I respectfully disagree solely when it comes to accounting and supply chain processes in an enterprise ERP. Unwinding un-auditable processes costs so much f’ing time and money while the business still has to run that I’ve found it to be cheaper and better to be auditable from day 1, in this one specific instance.
There’s being auditABLE and being auditED. Honestly I think the article’s take is smarter for a less experienced or skilled founding team and tptacek’s is better for a more experienced team. Paying auditors to look at screenshots and CSVs is a giant waste of money until it’s not, but at the same time, letting bad practice ossify until it’s expensive to remove is also a mistake.
Yea agreed, my comment was more of a sidenote than a direct response.
I built one of the Trade Promotion Management platforms used in the NA market, and couldn't agree more. It's a nightmare trying to be auditable if you didn't think about it from the start.
I was at (insert infamous unicorn) and they spent a ton of money (all relative, the money meant nothing) but more importantly 18 months attempting to get SOX compliant and never made it, because running the business was too important. Of course it all came down to lack of leadership to enforce policies but even if we had it, it was objectively super fucking challenging.
When I do get a chance to implement compliant processes at the beginning, it’s one of those amazing IT things where we prevent WW3 but never get the credit for it.
> There are things you should start doing early that lay the groundwork for attestations, but you should be doing them anyways, even if you never plan to get a SOC2 (and if a big-ticket customer never demands it, you shouldn't SOC2). That's stuff like setting up single sign-on and having protected git branches; simple best practices.
This is in many ways the spirit of SOC2, no? There are a lot of startup founders, far more than I'd like, who would purposefully eschew such "simple best practices" unless they had an axe like a SOC2 audit swinging over them.
I think you're both right, for what it's worth, and my take is that you are more aligned with TFA than you perceive.
How are we both right? I think you literally should wait until the last minute to start a SOC2 process.
GP's point is that having SSO and protected git branches _is_ starting the SOC2 process.
I'm pretty sure that's not what the author meant. Again: those are things you should do regardless of whether you're ever going to get SOC2 (and a lot of startups shouldn't).
That and having a ticket system (e.g., Jira) to track why you touched prod and you can answer just about every question.
Exactly; lots of over engineering/pre-optimisation in this. It's less for startups and more for startups-burning-vc-money-while-team-builds-resume.
Do you know of a good resource which describes these simple best practices?
I wrote two:
https://www.latacora.com/blog/2020/03/12/the-soc-starting/
https://fly.io/blog/soc2-the-screenshots-will-continue-until...
Thanks!
The section on performance management is circular and vague: a good one is motivating and a bad one is demotivating. OK. Glad we got that out of the way.
The whole intro reads like a puffy resume and lots of gilding. Even a section of gushing testimonials.
And he puts his name on the title so you don't gotta read the author byline. Total cheese.
Having gone through quite a number of compliance audits... the one thing that is good in that advice, is that many items in an audit are just a checklist of questions, such as
do you have a policy for XYZ?
or confirm you have a process for "thing"
So what ends up happeneing is if you feel stressed about an audit, just getting a list of the audit, you will realize how much you can just say "yes" to and feel less daunted by the audit.
So, its a good self-check even if youre just crossing out the things you should have already have a framework for.
I like things like this not because I'm going to use it as a bible, but because it provides good coverage of responsibilities & concerns.
Unless you've got some great advisers or you worked under someone really great, no one's going to take you aside and give you a list of stuff you need to take care of once you're in this position.
For each section I'm asking - what's our answer to this? Do I agree with this? Is our process better? What have I missed? It's helpful.
I read all the time about folks who become a VP/CTO and stop coding. Management skills are not coding skills. I know it. But I can't for the life of me figure out why folks hang up their keyboards and let their first super power go to waste. You can be a technical CTO from start to finish. Treat your team and the company like a service that needs active contribution, maintenance, and on-call support; and also, get your hands dirty building by yourself and with your team.
I've also had the experience where the CTO was activly coding, 80% of the code base were theirs, and the company was hiring software engineers who could and wanted to fix up their stuff - there was this true luxury problem for this start-up: bad bugs everywhere, but patient and resilient customers. They found 4 willing engineers with good chemistry at first, at least up until they were constantly vetoed by the CTO in their decisions, because the teams best practices conflicted with the CTO-way of "getting things done" - it's a rigid hierachy after all, and not a democracy.
In my own experience it's a matter of only having that much time in the day. For 7 years I had somewhere between 20-25 people I was directly or indirectly responsible for. There was just not enough time to get anything useful done in the code and my time was much better spent solving problems that others couldn't. A few times I was able to pick up some really simple change just to get the experience first hand to go through all our processes and see where we can do better.
I always kept coding nights and weekends but it's just not the same and over time you are gonna get a little rusty. That said I greatly enjoyed getting my hand dirty all day during a sabbatical I'm taking.
When you're not just an IC, you have other priorities. That means your IC work can be derailed at any moment. _That_ means you can't take on work anywhere near the critical path or you're just blocking others or handing things off.
Reviews? Sure. Design meetings? Sure. But taking critical work will end up causing issues.
I don't think there's a right answer here, but there's definitely a point where your code contributions are much lower leverage than for example trying to recruit the next set of critical engineers, working on the technical roadmap to keep ahead of the competition, or making sure the engineering org is aligned with the rest of the company.
Any lines of code the VP/CTO could write, could likely be written by someone else on their team (and their team's quality could be even better) - but all the other items I listed is likely only something the VP/CTO could do the best at in the company. It's quite a rational decision to largely give up hands-on technical work for what's more important for your team and company.
For me, the hacker super power — the one worth carrying forward as you progress in leadership — is being able to prototype something that works and proves a concept.
Realistically, a proof of concept is also only 20% of the work an engineer needs to do in order for a change to become production worthy and I respect that my sketched ideas need a lot more care and craftsmanship than I have time to give them. Where I can help other ICs is having that initial 20% idea around which they can then build a working idea, and do so autonomously.
It feels very cringey to write — oh brave new world that has such people as me in it! — but I can easily reassure myself by remembering all the times earlier in my career where I was very grateful to be initially pointed, with quite a lot of prompting, in a particular direction and then being given the chance to deliver on it.
I’m just a lead, but I can imagine part of being a CTO takes the same form as what I’ve described.
> You can be a technical CTO from start to finish
My last CTO role (team of 40) had me absolutely over capacity from day one, and I am _good_ at time management. I would rather have been programming 50% of the time, but there just was no time, and no support structure in place I could hand stuff off to; I had to painstakingly build that, which was yet another reason I had no time.
I like the idea of continuing to code, but usually that’s not what you’re being paid for, and while I consider myself a very strong developer, they can be purchased for less than the CTO’s salary, rather than the more expensive CTO doing the work. FWIW I went back to IC after a few years and plan to stay that way for the rest of my career.
Elon Musk, for example, appears to be wholly self taught as a coder. Do you want Elon doing your code reviews?
Has anyone worked in a "two crews" system where there wasn't resentment? Or where people didn't want to naturally migrate to the "future crew".
I like the idea of this on paper. I have a hard time believing it can work in practice. The closest I've seen are library teams that build some service (say a design system + components) that other teams utilize.
At my last job we had a version of our on-prem product where the company sold a super extension for a version that was supposed to go out of support. We had a small team (I think three people) whose only responsibility it was to ensure that version was supported, pipelines worked and we're ready to ship a big fix at needed. That was their responsibility but as long as that was covered they were allowed to work in their spare time on what they wanted as long as they saw value for the company in it. It was a good bargain. Everyone else was grateful the small team was doing the dirty work and the maintenance team was delighted they could use the remaining time working on things that had always bugged them, do research, etc. This was a few years ago and I forgot details but I vaguely remember that we got some really cool improvements and research from them. The people on that team also were really excellent and self-motivating which helped make this a success but also more expensive.
> 9. Build An Explainer Video Library
I am not sure this is great advice. I am sure it makes sense for certain things, like UI or animation. But, generally reading text is more efficient than video watching. It's also easy to seek to the important parts of a text than a video when you are in a hurry.
Some people like videos, some like text. Both are an investment in effort and payoff. Gen AI makes it easy to do both now.
Its interesting that the original HN discussion talked a whole lot about recorded meetings. Feels like another world just a few years later with AI transcript embedded in every video conference.
Popular in 2023 (452 points, 156 comments) https://news.ycombinator.com/item?id=37971795