As a part of my general interest in the big picture of systems, a few months back I decided to start sniffing around for ways I could cultivate a systems design mindset. I really believe that an excellent launch pad for expertise in any given area is to have a good high-level overview from the start. Certifications and books are my favourite way to do this. They can’t replace real-world experience. However, how can you get experience when you are no better than a gorilla smacking rocks together in terms of your ability to communicate your designs?
So my search led me to DDIA (Designing Data-Intensive Applications), apparently the seminal book on the “big picture”. The book goes into the idea of the trade-off rather than a prescriptive “do X if Y”. You could build a system using XYZ design patterns, principles, and functional requirements but you inevitably have to lose in some area. The question is of your (or your organisation’s) value system. Do you value response time? Correctly ordered events? Raw throughput? All of these are very achievable in today’s technology environment. However, the limitation imposed by any given trade-off usually doesn’t boil down to what tool or what programming language you use, but often the base limitations of mathematics, distributed systems in general, or the amazing human ability to fuck things up.
In general, an excellent book I always recommend to people who don’t just want to stick to implementation. A must-read for any good industry professional.
Admittedly, I haven’t finished the book — namely because I had the first edition and knew a second was coming out very shortly. So I skimmed through the first edition, diving into any particular topic that I found interesting. I don’t think that’s the wrong way to use this book. I feel like it’s a reference guide rather than something to be completed.
However, I do plan on finishing it end to end once I get the second edition in my hands.