INTRODUCTION:
With over two decades of hands on experience, Rob Williams has designed, built and supported numerous enterprise infrastructure environments across multiple cloud platforms that include AWS and Azure.
Here at Vintage, we sat down with Rob to discuss the meaning of the term “best practice”, learning to say no to people and tech and a fantastic insight into a Fortune 50 cloud environment challenges.
VINTAGE: When is a best practice NOT a best practice?
RW:
Everyone seems to chase “best practices” as though they are magical, but they seem to chase them blindly.
Every best practice is a solution, for which it is best because of the right context.
In the wrong context, every “best” practice can easily become a “worst” practice. Many “worst” practices are always “worst”, so no one uses them. But most “worst” practices are merely “best” practices being used in the wrong context, like a fish out of water. People use them because they work sometimes, but they may not understand when and why. Context is the answer.
One crucial role as an architect is to see underneath of the practice and look into what drives the specific practice to be useful.
To understand best practices, one needs to understand design patterns (Gang of Four book).
Design patterns are the textbook definition of best practices, and they are a few of the really good ones.
Each pattern has a context in which it applies.
The context has drivers, factors in the situation that motivate one to avoid certain issues or to choose to proceed in certain directions.
The pattern, or practice, has characteristics that interact with those drivers, factors, and issues. A best practice is only “best” when its characteristics align with the context.
Looking into how does the best practice work with other areas.
Computers are tools and are meaningless without people which ultimately is why most best practices fail if people don’t live these values.
The computer industry is NOT about computers; it is about people.
The computers care about nothing–they are merely tools without awareness.
Computers do not respect your status, your money, your degrees, or anything else. Computers only respond to expertise, to an understanding of the math and physics that govern their functionality.
But that expertise is wasted if it is not used to make the computers serve the people.
It is the people that give the computers purpose.
It is the people that make a mess of the computers by misusing them (such as by applying a “worst” practice).
It is important to get to know people in the team and identify their strengths and weaknesses and match that.
An agile approach attempts to optimize the software development process. It is almost entirely focused on the people, especially the specific people on the specific team with a specific customer. There is no single right way to develop (or operate) software and systems. But there DO exist “best” practices at the core of agile that support each other, complement each other, and cover for each other when used together as appropriate to the context. That “context” is the people.
VINTAGE: Handling overwhelm: learning to say NO to people & technology
RW: Back in 1992, I finished my military service and returned to civilian life.
I was quickly overwhelmed by all the computer industry job listings I found, because I had the skills to take on most of them.
I was applying to many of them, but not succeeding beyond that.
I was spread so thin across so many opportunities that I could not land any of them.
I spent my last $3,000 of credit to take a career marketing course, which I consider to be one of the stupidest and best decisions of my life.
I learned many things such as negotiating my compensation, writing good resumés, etc. But the most important lesson was handling overwhelm by learning to say “No”.
I gained the confidence and clarity to say “No” because I did not like something about an opportunity.
Working in an ancient programming language, “No”. Working on the other side of town, “No”. Accepting compensation that is too low, “No”. Working in an area of technology that I do not enjoy, “No”. In short, look at your own heart and mind and be confident to follow their guidance. Look for the reasons, big and small, that shape your values and preferences.
It can be related to technology and people. Do I even like that technology we are using?!
Lots of tools and technologies to choose from so it is important to select ones that compliment the team
What are my strengths and weaknesses and the teams?!
In terms of technology, Ruby is genius because it is so creative and powerful.
But it is an engineer’s nightmare because a single line of Ruby code can fundamentally alter the meaning of all Ruby code that follows.
It can be nearly impossible to reproduce a reliable Ruby execution environment.
This fact crippled Opscode Chef, for example, eventually driving them to create a “fat install” of Ruby (and perhaps to rewrite parts in Erlang).
PHP is another engineer’s nightmare, because it lacks an engineered design and specification.
It was homegrown, without the discipline and consistency necessary to encourage enterprise usage.
VINTAGE: Provide some insight into a Fortune 50 networking company cloud challenges
RW:Two-year contract, supporting database engines including Couchbase, MongoDB, and Oracle.
My team had an on-call rotation for production support. We tackled data migrations.
We advised development teams and architects. We cleaned up messes.
We had production impacting issues over and over with Couchbase, NOT because there was anything wrong with Couchbase itself.
Couchbase would ride through the issue clean as a whistle, beyond any reasonable basis for expecting it to do so.
Each time, my teammates and I would analyze the root causes. Most of the time, there was only one.
The network was too congested and unstable for ANY distributed system to function correctly.
Our own code manifested the problems most of all, so most of the production issues focused on our development teams. But the real culprit was the “flat network”.
One of the original architects messed up the system design by failing to “tier” the network.
Instead, all the components were directly connected together with no segmenting (partitioning) of the network. The result is like everyone working in a single large room, yelling over everyone else to be heard by their coworkers on the other side of the room.
The network congestion routinely manifested as tens of thousands of failed attempts for applications to connect to their database engines.
No wonder nothing was working correctly.
Yet, in two years of trying, my teammates and I could get no one to even begin to address the problem. Why? Because no one was in charge.
Development teams made their own technology decisions. No one dared to reign them in. And the development teams cared nothing about anything but “features”. Technical debt did not seem to be a concept to them. Architecture was something about which others worried, but not them. The technology did not fail. The people did.
Parts of this Fortune 50 network engineering company know how to properly build a cloud network and systems that depend upon it.
But other parts of such a huge company are too far removed from that core to follow the company’s strength.
………
VINTAGE: Finally, what books, blogs, podcasts do you follow?
RW:
Bruce Eckel, “Thinking in Java” and others
Tavis Rudd, “Throw away your templates”
Bill Poulos, Profits Run, options trading
VINTAGE:
Thanks for taking the time to speak to us today Rob.
Interesting insight into best practices and cloud challenges.
