I couldn't agree more. Writing code was always the easiest part of the job (sure, it has its hard moments where you have to solve complex problems, but that's the fun part). The hardest parts for me have always, always been:
- dealing with non-technical project managers
- dealing with "rockstar" developers that only want to work on greenfield projects
- maintaining legacy
- trying to anticipate potential design flaws or observability blindspots
- dealing with nasty bugs in production (and reproducing the bug!) and trying to get the right people in a room to solve them
All in all, the human aspect was always the hardest part, and as this article clearly states, is now even harder. You can't replace decades of crisis situation that might not have been documented, late nights spinning prod back up or using our human friendships to get devops guys to help us out with admin tasks! (Costs a few beers, instead of millions of tokens!)
On the other hand, LLMs actually can help with a lot of these “human aspect” problems, which to my mind boils down mostly to transferring information between humans is hard.
The focus on just the writing code aspect skips over that some of these models can be really good at reading code, faster than you can as a human, and pointing you in the right direction. Maintaining legacy code is hard for a number of reasons but one of those is that reading code is like trying to read someone else’s mind. If you don’t have the same mental view of the software they do, it’s going to be harder to understand what it’s doing. Lots of anticipating flaws comes from knowing best practices and from either direct experience, or having read other peoples experiences and extrapolating that to your code. Likewise finding bugs is often a lot of code reading combining with experience to identify things that look wrong and then testing that theory.
LLMs can help with all of this, and even better they don’t have to be exactly right so long as they don’t drive you down a non-existent rabbit hole. You can ask an LLM why some code might be exhibiting a behavior or where some behavior is implemented and the LLM can very quickly get you most of the way there. They don’t have to be right about how a shell script gets called to point you to the shell script being the location something happens. They don’t have to be right that the bottle neck in your code is at line 4583 to get you to a buried hot loop with bad performance.
Sure they’re not replacement for a dev with 10 years experience with the system and who was there when the specs were written. But you don’t always have that resource at your disposal. Sometimes you’re working on a system that was written decades ago for a world that doesn’t exist anymore, patched by teams that have since come and gone who also didn’t have that knowledge and held together by sheer force of will and developer grit. On a project like that, anything that can help you search that code base quickly and especially with fuzzy descriptions is absolutely a boon.
You're right, I certainly agree with your assessment that it can be an invaluable tool to search through codebases quickly! No team can process it that quickly, especially if, like you say, this was written by a team that no longer exists.
The same thing came up in the early 2000s when businesses tried offshoring software development. It turns out that translating business requirement to technical design and implementation is one of the hard parts of software development. In a pure waterfall model, this works ok, but you rarely know what you want until the end users see it. This is while agile is so important.
I hate to say it, but with over 30 years experience as a software developer in various roles, writing code has never been the hard part of delivering a successful project.
I'd have to more or less agree with a sibling comment, in the sense that writing the code isn't the hardest part (that's the agreement part) but it's not the easiest, either (that's the disagreement part).
I write a lot of code for highly secured black boxes that break a lot of networking rules: undirectional gateways and guards where two things must be guaranteed: one, no matter what, data can flow in one direction and one direction only, and two, no matter what, with the proviso that we cannot control customer operators, the only data flows permitted are those that have been reviewed by two people with independent roles.
The hardest part is understanding the space, how things need to be, what might work, what might go wrong, how a change here might break something there (and this in a system basically running in old-fashioned SELinux "strict" mode).
(There is only so much experimentation and instrumentation possible, because I need to make things work in that closed, tightly controlled appliance.)
The easiest thing is explaining this to others, at various levels: It’s about knowing what to include in the explanation, what to omit, and which analogies are useful, which are harmful. That all comes from repetition and retrospection.
Writing code is somewhere in the middle: once I understand the problem, the code is usually pretty easy, except, of course, when it isn't because oh, yeah, that.
Fortunately, I have a long legged hound. My entire team knows the value of our DWs, dog walks, or, as a colleague recently called it, Diagnostic Wandering.
Two hours a day in the woods, along the rivers, and much becomes clear. Not all, just much.
The really second hardest part is mentally working through the coding alternatives like an adversary, trying to figure which one is best, or, on the whole, given all the layers of defence, good enough.
I couldn't agree more. Writing code was always the easiest part of the job (sure, it has its hard moments where you have to solve complex problems, but that's the fun part). The hardest parts for me have always, always been:
- dealing with non-technical project managers
- dealing with "rockstar" developers that only want to work on greenfield projects
- maintaining legacy
- trying to anticipate potential design flaws or observability blindspots
- dealing with nasty bugs in production (and reproducing the bug!) and trying to get the right people in a room to solve them
All in all, the human aspect was always the hardest part, and as this article clearly states, is now even harder. You can't replace decades of crisis situation that might not have been documented, late nights spinning prod back up or using our human friendships to get devops guys to help us out with admin tasks! (Costs a few beers, instead of millions of tokens!)
On the other hand, LLMs actually can help with a lot of these “human aspect” problems, which to my mind boils down mostly to transferring information between humans is hard.
The focus on just the writing code aspect skips over that some of these models can be really good at reading code, faster than you can as a human, and pointing you in the right direction. Maintaining legacy code is hard for a number of reasons but one of those is that reading code is like trying to read someone else’s mind. If you don’t have the same mental view of the software they do, it’s going to be harder to understand what it’s doing. Lots of anticipating flaws comes from knowing best practices and from either direct experience, or having read other peoples experiences and extrapolating that to your code. Likewise finding bugs is often a lot of code reading combining with experience to identify things that look wrong and then testing that theory.
LLMs can help with all of this, and even better they don’t have to be exactly right so long as they don’t drive you down a non-existent rabbit hole. You can ask an LLM why some code might be exhibiting a behavior or where some behavior is implemented and the LLM can very quickly get you most of the way there. They don’t have to be right about how a shell script gets called to point you to the shell script being the location something happens. They don’t have to be right that the bottle neck in your code is at line 4583 to get you to a buried hot loop with bad performance.
Sure they’re not replacement for a dev with 10 years experience with the system and who was there when the specs were written. But you don’t always have that resource at your disposal. Sometimes you’re working on a system that was written decades ago for a world that doesn’t exist anymore, patched by teams that have since come and gone who also didn’t have that knowledge and held together by sheer force of will and developer grit. On a project like that, anything that can help you search that code base quickly and especially with fuzzy descriptions is absolutely a boon.
You're right, I certainly agree with your assessment that it can be an invaluable tool to search through codebases quickly! No team can process it that quickly, especially if, like you say, this was written by a team that no longer exists.
The same thing came up in the early 2000s when businesses tried offshoring software development. It turns out that translating business requirement to technical design and implementation is one of the hard parts of software development. In a pure waterfall model, this works ok, but you rarely know what you want until the end users see it. This is while agile is so important.
I hate to say it, but with over 30 years experience as a software developer in various roles, writing code has never been the hard part of delivering a successful project.
I'd have to more or less agree with a sibling comment, in the sense that writing the code isn't the hardest part (that's the agreement part) but it's not the easiest, either (that's the disagreement part).
I write a lot of code for highly secured black boxes that break a lot of networking rules: undirectional gateways and guards where two things must be guaranteed: one, no matter what, data can flow in one direction and one direction only, and two, no matter what, with the proviso that we cannot control customer operators, the only data flows permitted are those that have been reviewed by two people with independent roles.
The hardest part is understanding the space, how things need to be, what might work, what might go wrong, how a change here might break something there (and this in a system basically running in old-fashioned SELinux "strict" mode).
(There is only so much experimentation and instrumentation possible, because I need to make things work in that closed, tightly controlled appliance.)
The easiest thing is explaining this to others, at various levels: It’s about knowing what to include in the explanation, what to omit, and which analogies are useful, which are harmful. That all comes from repetition and retrospection.
Writing code is somewhere in the middle: once I understand the problem, the code is usually pretty easy, except, of course, when it isn't because oh, yeah, that.
Fortunately, I have a long legged hound. My entire team knows the value of our DWs, dog walks, or, as a colleague recently called it, Diagnostic Wandering.
Two hours a day in the woods, along the rivers, and much becomes clear. Not all, just much.
The really second hardest part is mentally working through the coding alternatives like an adversary, trying to figure which one is best, or, on the whole, given all the layers of defence, good enough.
Couldn't be more spot on. This is exactly why I started Dosu https://dosu.dev/ to reimagine on the "knowledge" side of software engineering.