High-Level vs Low-Level
What does it mean when someone is describing something at a high-level vs a low-level? It depends on the context (what’s being described). In software, the lower you are, the closer you are to the hardware of a single machine. Conversely, at the highest level, you have a full picture of the entire system, end-to-end — which could potentially span the entire planet (or more).
In such a planet-spanning system of thousands of machines communicating with each other, its human-readable system design diagram wouldn’t show every single machine, it would likely show how regional clusters of machines communicate with each other. An example of this is the following cross-region Pub/Sub messaging pattern:
Note: In AWS, these Pub/Sub Topics and Queues would likely be SNS (Simple Notification Service) and SQS (Simple Queue Service), respectively. In either case, the above diagram is very high-level.
Even a system design diagram for one of these regional clusters (like the one below) would be considered high-level (in an absolute sense). However, relative to the diagram above, it would be a lower level diagram (or description). It’s important to note that a high-level diagram in one conversation could be considered low-level in another conversation. It’s all relative. Below, we dive a little deeper and imagine how one of our regional clusters might consume and process these cross-region messages:
Note: We’re simply storing the messages in a datastore. In reality, a system that requires regional clusters would likely be much more complex than this, with its own internal Pub/Sub topic queues, caching services, sharded database tables, etc…
Taking a step lower might be a diagram (or description) depicting the interaction of different components within a single application running on a machine. Below is an example of an Object-Oriented Design diagram depicting a basic single-purpose event processor.
Note: This example of an abstract class with concrete subclasses is a very basic example of object-oriented design, but enough for the scope of this blog post. Also remember: prefer composition over inheritance as the former is more malleable (open to change). An actual diagram of the different components in an application in the real world would typically be much more complicated than the one above. Final notes: one could argue that event messages like this would likely flow through an asynchronous pipeline without responses or acknowledgement of receipt. The classes in the class hierarchy would likely have more fields as well.
Can we go even lower?
Most developers don’t need to go any lower because high-level programming languages like Python or Java are tools with APIs that provide an interface that hide all the nitty-gritty interactions with the Operating System’s Kernel and system hardware. Python is built on top of (written in) C, which is a relatively lower level programming language. Taking a step lower, those who are familiar with Assembly languages would consider C to be a high-level programming language. Assembly languages are the lowest level of programming that consists of human-readable code. At this level, the code that’s written is specific to the hardware it’s running on. Lastly, the absolute lowest level is machine code — zeros and ones.
A good developer writing in a high-level programming language like Java or Python is still needs to be aware of the interactions that take place in hardware — specifically in memory, in persistent storage, and even in CPU cycles through the looping that takes place in their code. Some might argue that this awareness is the distinction between a software developer and a software engineer, but in either case, writing code is developing software. This may sound easy enough, but it becomes challenging with distributed systems handling internet-scale traffic and data.
It’s important to note that there isn’t a finite number of “levels” between high-level and low-level. There are system configurations that would easily fit in-between the levels mentioned here. For example, a diagram depicting the communication between multiple Docker containers in a single machine would fit nicely between Single Region and Single Application. Alternatively, the level depicting Object-Oriented Design could be replaced with Functional Design, or some other programming paradigm. The incredible flexibility of software and the systems it runs on makes writing a post like this difficult. The difficulty lies in breaking down an abstract topic (like High Level vs Low Level) into concrete terms with clearly defined boundaries. That’s why such posts need to make heavy use of words like might, likely, or typically. Although there are plenty of absolutes with clearly defined boundaries in this field, there’s just as much (if not more) abstract concepts that are not only relative to each other, but actually overlap each other! An example of this flexibility is how object-oriented features can be built into a language like C, which doesn’t offer that functionality natively. This doesn’t mean that “anything goes” — each solution depends on the individual use case and technology (including software) is the perfect tool for infinite use cases.
Sam Malayek works in Vancouver for Amazon Web Services, and uses this space to fill in a few gaps. Opinions are his own.