tl;dr: Software projects exist on a continuum between the Lewis and Clark expedition, and laying down freeway. Knowing what kind of project you’re on can be the difference between success and failure.
In 1803 Thomas Jefferson commissioned the explorers Lewis and Clark set out to explore the American west, no one knew what they would find. The whole point of the venture was to figure out what was there. Gold? Farmlands? Trading partners? Rivers?
On the other hand, when that stretch of highway near your house was built, the decision to build happened years ago by some planner, the road was built to last for many years, and development proceeded methodically and carefully, drawing upon centuries of road-building experience.
Problems arise when the developers, managers, and product managers aren’t clear on what the mission is; employing the wrong techniques or wrong metrics of success. This shows up acutely between startups and large companies.
Startups are typically explorations. The founders have an inkling that there is value to be found in some direction. The developers write code quickly to explore the space, knowing that almost all of it will get thrown away. You keep scaling and maintainability in mind, but the top priorities are (1) code works (2) that can be easily changed and (3) most importantly, provides some data answering the questions “Is this something anyone wants?” or “Can this be built at all?”
Big companies are more like the highway crew. The sources of value have been located, there is no doubt that the traffic will come once the on-ramp opens. The developers work within a large body of existing code, careful not to break anything. Scalability and maintainability are top-of-mind.
Of course this is over-simplifying: both styles of mission exist within companies of any size. Big companies have innovation and R&D departments, classic Lewis and Clark explorations. And a startup that is preparing to take on their first big customer will want to be damn sure that their code can scale to handle the customer’s use!
But some startups try to act like big companies from the very start, crippling themselves. With no validation that anyone wants the service, they build a system completely containerized, with a CDN, load balancer, de-normalized noSQL scalable database, GraphQL API, and everything written from the ground up in Node. The better solution might be as simple as a Django or WordPress (PHP, gasp!) installation with a just a dash of custom code. They’re laying down concrete and guardrails when they should be bushwacking to the top of the nearest hill.
I find that within the developer community there tends to be a fettishization of the “highway building skills.” Who wouldn’t want to believe that their code will need to process billions of transactions? Why wouldn’t you want to use the same techniques as Google or Amazon?
But being a skilled explorer is just as important a skill, but harder to codify. Knowing how to keep your code nimble. Knowing how build that demo that works just good enough that the investors put in money and keep the lights on. Knowing how to find that open-source project that is kinda sorta like your goal but can be hacked into shape.
In short, there is a difference between bad code and exploratory code.
Finally, it’s important to realize every highway owes its existence to some explorer. Facebook began as some monolithic PHP code. Google began as student code from a PhD thesis. Slack began as a whipped-together internal tool of a game company. The beauty of exploring is never knowing what you’ll find!