Learn System Design
A bi-weekly podcast hosted by a senior engineer named Ben Kitchell that takes a deep dive into learning about technical system design by learning together. Each episode we will explore the inner workings of what makes these systems so complex and fascinating while building on our knowledge of how they came together.
All music written and performed by the mysterious Aimless Orbiter. You can find more info about him and his music at https://soundcloud.com/aimlessorbitermusic
Learn System Design
5. Crafting Resilient Architectures with Messaging Brokers
Discover the secret sauce that makes software systems scalable and robust as we dissect the architecture that powers modern applications. You're about to get a masterclass in software design, where we unravel the complex world of monolithic and microservice architectures. Say goodbye to confusion around when to stick with the simplicity of a monolith and when to break free into the modular world of microservices. We'll walk you through the need for message brokers and queues in this digital maze, ensuring your journey in software scalability is as smooth as possible.
Then, fasten your seatbelts as we delve into the messaging protocols that keep the world connected. MQTT and AMQP are more than just acronyms; they're the backbone of reliable communication in distributed systems. We'll demystify MQTT's retained messages and 'Last Will' features, and break down AMQP's advanced message handling that's critical in sectors like banking. This episode is jam-packed with insights that promise to elevate your understanding of the intricate dance of message exchange types and delivery processes. Tune in and equip yourself with the knowledge to architect resilient and flexible software systems.
Show Notes:
You can follow Diego's language podcast here: https://diversilingua.com/
Dedicated to the memory of Crystal Rose.
Email me at LearnSystemDesignPod@gmail.com
Join the free Discord
Consider supporting us on Patreon
Special thanks to Aimless Orbiter for the wonderful music.
Please consider giving us a rating on ITunes or wherever you listen to new episodes.
Hello everyone, welcome to episode number five of the Learn System Design podcast. A few of you did reach out about the sound mix over the last couple of episodes, so I want to let you know that episodes three and four have been re-uploaded with a better mix. Special thanks to Pallavi and Diego, who both reached out and mentioned that the music I was playing is honestly not just a little loud but also distracting. So for all future episodes, the only music will be the intro and the outro music, none during the conversation portions. Thank you again to Pallavi, diego and everyone else who reached out for this feedback. Please feel free to reach out on any ways that I could improve this podcast and you have a better listening experience. Diego in particular actually dropped his podcast for learning a new language, and so I'll also be including that in the show notes below if you want to check that out.
Speaker 1:But in today's episode we're going to be tackling all things message brokers and message queues. I'll be covering a few topics this episode that sort of correlate with that, but it will all add up to explain how a message broker works, when to implement it and how to use it To do that. I want to first touch on microservices and message queues, thank you. Briefly, I want to explain an important concept that needs to be clear before we talk about one of the biggest use cases of using a message queue or even a message broker. Using a message queue or even a message broker. To be clear, a message queue is a queue that lives inside a broker and you can have multiple message queues inside of a broker. But we'll get into all that. For now, the concept I want to talk about is a monolithic application versus a microservice application. These are both architectures and there's a lot of other architectures that we'll dive into in a future episode, things like event-driven architecture, domain-driven, mvc, all of these sorts of things but for this episode, I just want to talk about monoliths and microservice architecture and their paradigms today and how they're used.
Speaker 1:Make no mistake, most applications start off as a monolith, and what I mean when I say a monolith is just that all of the code for your application is in one place, usually some sort of repository, but you know it exists all in a single node. If you think about like a simple web application with a front end and a back end, that code for the front end, whether it's in React or Vue and that code for the back end, whether it's in Django or Node or Nest or whatever makes you happy. All of these live in a single place, all the codes right beside each other within the same repository. And this is great because if you need to change something, if you need to see, you know what the front end is expecting on the back end and what the request needs to look like. It's very simple you can see all of the code in a single place. It's great for quickly developing things and, especially if you have a small team, being able to see what all the code's doing in your entire application without feeling siloed and what I mean by siloed is blocked off from what other teams are doing and what other code looks like. Having to rely on someone telling you what a request looks like or calling that, rather than understanding what the request will look like. And so that's great.
Speaker 1:Until it comes time to actually scale at a reasonable rate. If your backend needs more resources, there's no direct way to give only your backend more memory. Instead, you have to scale vertically, give your entire application more memory and hope that a different part of the code doesn't steal it. That you know I'm giving my application more memory and hopefully the backend gets as much as it needs to. The second biggest hurdle you're going to hit is code quality, because all of your developers are working out of a single place, a single repo.
Speaker 1:Once monoliths get too large, it is nearly impossible to make a change and not have it affect another piece of code in your repository in an unknown way. So why then do most applications start off as a monolith? The answer is simplicity, right, they are much easier to work on with smaller teams, as I've mentioned, but they're also so much easier to deploy. You're not having to deploy you know 10 microservices or n number of microservices just to get everything to work. You deploy one thing that you know works because you've ran it on a single node before. The bottom line is, when a monolith is small, despite the obvious oxymoron here, it is much easier to work with for your dev team at every level and, much like with the world of fashion or culture, software architecture is ever-changing. It's not immune from trends, and one of the hottest trends of the last 10 years, large in part thanks to companies like Netflix, google, meta, have been what's called a microservice architecture.
Speaker 1:A microservice architecture is simply a paradigm that takes the idea of a small monolith and implies it to the entire application, using sort of like a chunk style approach. So let's think about that small web application that I talked about before. You have your front-end code and now, in a microservice architecture, that's its own service right that can be deployed on its own. Then you have a separate service for your back-end, which architecture? That's its own service right that can be deployed on its own. Then you have a separate service for your backend, which, again deployed on its own. But, most importantly, these applications have their own resources and can be scaled independently, horizontally, as quickly as needed. So I'll restate that. So if you have the need for more back-end services, you can scale that horizontally, have duplicates that handle it without having to worry about whether or not the front-end needs more resources, or you can scale them both at the same time. That also works. It gives you more independence on controlling the scaling, and so one of the biggest drawbacks of microservices is getting them to work together as expected.
Speaker 1:It's a bit more complex than, say, a monolithic system because, again, a monolith everything is there, it can communicate all in the same application. It can communicate all in the same application, whereas a microservice, you have to either communicate via a message request or on the local network, or through a message broker, which we will talk about in just a bit. For now, I want to rewind for a minute and talk about what a message queue actually is A queue which, if you need a refresher, is just a data structure in which the flow of the elements serve as a first in, first out paradigm FIFO for short. If you imagine yourself at a bank, the first person in line usually is the first to be taken care of and then the first out the door. The same applies for a queue in software engineering. A message queue is the same, but the elements in this case are messages, and these messages get processed by a service. The queue takes a message from one service, in this case a producer, and then it's piped into a single consumer, which of course is another service of some kind, and then it's removed from the queue.
Speaker 1:So imagine for a moment uploading a picture to your Instagram or your Facebook or any social media site. You don't want to have to wait for the picture to be uploaded to the object storage and then written to the database and then returned and says okay, your picture is all good, right, like that takes a long time for the consumer and if that happened, people would like be like this. This service is extremely slow, right, and that's how it would work with a normal request structure, whereas something with like a message queue, all you have to do is, once it's on that queue, return a 200 to the, to the user. They just understand okay, my picture's been uploaded, I don't need to worry about what, what else is happening. And then on the back end, you take it, you put it in the object storage, you put it in the database, etc. Etc. And then, once one of their friends logs online, you just pull it from the database, fetch it from the object storage and show it. And so that's the difference, right, is, the user doesn't have to wait for all of this processing to happen. It needs to happen asynchronously from the user, because, genuinely, the user doesn't need to know how long it takes to to save the picture somewhere or anything like that. What they want to know is that their picture is safe and that we're doing something to it.
Speaker 1:As far as popular message queues, they come in all shapes and sizes. There's things like IBM MQ, which I think might be the most popular one for like hardware solutions and for this I kind of want to just focus on like external cloud-based solutions, and so like a few of those is like. Amazon, of course, has their SQS system and that's a proprietary solution offered by Amazon. Sqs is a great solution if you need something hands-off. It scales well and cost isn't really a concern because it can get costly sometimes. On the other side of that coin, if you need something more open source, there's a solution called Celery. Celery offers the ability to actually bring your own messaging broker and then Celery, a message queue, just lives inside of that broker.
Speaker 1:Message brokers can be thought of as an application that sits between any two services that are trying to communicate with each other. It takes a message sent by the producer or the sender or any service, and transforms that message into something the consumer or the receiver basically another service can actually use. Basically, it changes the protocol of the message so that it can be consumed properly. Let's take a pause for a second and actually talk about what a protocol is and what sort of lives in a message broker sort of the things I'm about to dive into and it can get a little messy and a little complicated whenever I throw a bunch of technical jargon that's all nested within each other. So instead I'm just going to sort of lay it out and then we can dive into it a little more.
Speaker 1:In effect, message brokers consist of two types of things an exchange and a message queue. And you can have multiples of exchanges or multiples of message queues, but you'll always have at least one. In a general sense, we've already touched on what a message queue is and how it fits into a message broker. And on the other end of that, usually what comes first in the order of a message broker is what's called the exchange into that. Usually what comes first in the order of a message broker is what's called the exchange. And the exchange is basically some code that handles a message in a certain way and how it handles it is called a protocol. So just to sort of clarify a message comes into a broker. That broker handles the message in a certain way, aka a protocol. That exchange will tell the message where to go, so which queue to go to, and then, as it goes into the queue, whenever the consumer of that message is ready, it will take it off of that queue from the message broker, and we've effectively moved through the whole thing.
Speaker 1:The first and easiest to remember protocol is called STOMP, so STOMP stands for Simple Text Oriented Messaging Protocol. Its appeal is the fact that it's very simple to use, especially for scripting languages. It works very similar to HTTP requests and can even be used via TCP and WebSockets, and so what this means for you is that you can use it in a browser for real-time chat clients. Another benefit of Stomp is that it is the only one of these three protocols that I'm going to talk about that are actually human-readable instead of binary. If you're familiar with JSON, if you've ever opened up a JSON file to read all the key value pairs, it's the same thing. In the fact that it's human readable not that Stomp is JSON-like, but because it's human readable, you actually sacrifice on the size of a message and the time spent sending that message. That's why you know stop is great for things like web clients where, like you, stay connected to the server, but something where you might have a disconnect it's not as great. Genuinely, because you risk having a larger message that can be broken apart and thus it fails.
Speaker 1:The next protocol I want to talk about is known as MQTT. It's most widely used for embedded or low-powered devices where network issues are possible. Mqtt stands for Message Queue Telemetry Transport. It focuses on a single pattern of publisher and subscriber. One of the biggest benefits of MQTT is the ability to give it a quality of service level, so a QoS level can be marked as 0, 1, or 2. Qos2 gives it the highest level of a guarantee that a message will be received while also using the most amount of network bandwidth, whereas something with a QoS of zero uses the least amount of network bandwidth and therefore has the least likelihood the message will actually be delivered successfully.
Speaker 1:The other benefits of MQTT are retained and LWT messages. Retained messages are pretty much exactly what they sound like. It's a message that is retained in a broker that is sent to any consumer that comes online after being down. You can think of retained messages as the last important message since you've been gone. The LWT or Last Will and Testament message is a little bit different, but also extremely powerful. While the message itself is the same as any other MQTT message, it is set by a consumer instead of the producer and only sent out of the event of that consumer disconnecting in a non-graceful way. So something like a network outage or a 500 error. To compare this to a real life situation and explain the name a little bit, imagine you and everyone you love is a consumer and in this scenario you only communicate through letters sent through your lawyer. I know it's a strange concept, but just follow me here. Each of you gives a last will and testament message to the lawyer and he is not to give it out until you pass away. When any of you dies, the other consumers get this LWT letter that you have written. It's the same concept with the MQTT protocol One of your consumer disconnects in some undesirable way and every other consumer gets your LWT message. This helps if a service needs to know that another service is down.
Speaker 1:The last protocol I want to talk about is called the AMQP or Advanced Message Queuing Protocol. Amqp, much like MQTT, uses a publisher and subscriber architecture where some application publishes a message and one or more applications can subscribe to that message. The message broker in this case is what applies this protocol when handling the messages? Amqp, which is mostly used in like baking systems, uses exchanges, routing keys, bindings and queues to help facilitate this whole process. When a producer sends a message to the message broker. The message broker the message goes through one of these types of exchanges that we talked about earlier. These exchanges forward the message to a specific queue and finally out to any consumer that is subscribed to the topic of that queue. So I briefly talked about exchanges earlier. Let's talk for a minute about the different types of exchanges that can exist in a broker and, honestly, why they're important. The four key exchange types are called direct, topic, fan out and header. Before I dive into these exchange types, I need to define a few things that are going to make it easier for you to follow here.
Speaker 1:Let's start with a binding, sometimes called a route. A binding in a message broker is just a link that matches a specific queue to a specific exchange. This link is submitted using a binding key, which is just a specific way of identifying that binding. It might be helpful to picture a binding as a road and a binding key as the name of the road. Routing keys, which sometimes are confused for binding keys, but they are not the same thing are a message property, depending on the type of exchange. It is used sometimes to verify which queue a message should go down. It is used sometimes to verify which queue a message should go down. If we keep the analogy before that we had, where a binding is a road, the binding key is the name of the road. We can consider the message a package that needs to be delivered to a certain place and that routing key is the address on the package itself.
Speaker 1:So now, with our newfound knowledge of how brokers handle queues and map the messages from exchange, let's dive into the actual types of exchanges and how they work. First up is a direct exchange. They're the simplest type and it simply looks at a routing key on a message, compares that to the binding key on a queue and sends it to the queue only if it's an exact match. In the event where there are no binding keys that match, the message is simply discarded and it doesn't leave the message broker. Next up is called a topic exchange. These are a little more powerful and better for use cases where you have messages that should go to more than one queue and eventually to more than one consumer. Right, topic exchanges introduce a sort of new concept. We haven't heard. It's called a binding pattern.
Speaker 1:Binding patterns are the secret sauce to allowing messages to be sent to multiple queues. It forces all routing keys to be specifically separated by single dots ABC instead of A, space B, space C, and instead of having a specific binding key that needs to match, you can actually use an asterisk or pound sign in your binding key. That needs to match. You can actually use an asterisk or pound sign in your binding key to help catch potential candidates for a queue. The two special characters that enable this are an asterisk and a pound sign.
Speaker 1:The asterisk or wildcard is used to say there will be a single word here and I don't care what that word is. If we use the example from before abc, we have a binding pattern that is asteriskbc. We can have any word at the front, regardless of length, as long as it ends with dot b, dot c, it is valid. However, if anything follows that c, it is invalid. So, for example, we can have ben dot b, dot c. That's valid. Cat dot b, dot c. That's valid, but benbcanything is invalid because it needs to be specifically wordbc, because that asterisk only replaces the word. The pound sign, sometimes called a hashtag, covers this scenario where the asterisk doesn't. Its rules are basically any number of words or letters can go here, from zero words to an infinite amount of words. Imagine again this scenario of abc. If we add apound sign, then abc is valid, abcd is valid, dot c is valid, a dot b, dot c, dot, d is valid and everything all the way to a, dot b, dot c, dot, d, dot. You know infinite number of letters or words. However, it always has to begin with a, always.
Speaker 1:Next, let's talk about fan out exchanges. They are perfect if you want your message to go to a lot of queues without a lot of overhead on patterns and keys. With fan out exchanges, every time the exchange gets a message it will send it to every single queue that is bound, regardless of keys. In fact, if you specify a key on these, you're just wasting your time because it doesn't even look at it. Fanout is a great strategy when you have consumers that might care about any message that comes through and can handle any message in their own way. And, to top it all off, there is no limit to the amount of queues that you can bind to a fanout exchange. A real world example here might be some sort of breaking news alert. It doesn't matter what the news is, doesn't matter if it's sports or an emergency, every news organization is going to want to know about it and display it in their own way, and that's when fan out exchanges are really helpful.
Speaker 1:The final exchange I want to talk about is called the header exchange. If you're not familiar with headers in HTTP requests, they're a simple way of giving information to the server describing the encoding or the authorization or whatever you want. It describes something about the request you're sending. You can think of them as just little helpful bits of information describing what you're sending. You can think of them as just little helpful bits of information describing what you're sending. Headers in an exchange work the same way. They are specialized keys that are used to specify which queue the message should be routed to. Header exchanges are very similar to topic exchanges, but instead of using binding keys, we just use headers on a message. I do want to make a special like sort of comment here and say that all message brokers use exchanges and queues. It's specifically the protocols within them is how they're used. So while these exchanges were mostly covered under the AMQP strategy, any protocol can use these exchanges effectively. In our discussion about header exchanges, I briefly talked about how HTTP requests and how message brokers both use headers and how they're sort of similar. I think it would be useful to take some time to talk about when to use a message broker instead of HTTP requests between your microservices to help them communicate.
Speaker 1:When you are breaking out a monolithic system into a microservice architecture, one of the first and easiest things to do is have your services communicate through HTTP requests. Some teams opt into having requests go through a normal everyday process You're just firing it off across the web Whereas some teams try to utilize more of a local network. They both have their pros and cons, where working on a local network is great because it means all of your nodes are together and your requests are really fast, but it also means all of your nodes have to be on the same region of the same cloud and share that same local network. However, eventually, as with most things, the solution of just using http requests across your system starts to prove very restricting. It makes it so your services have to be tightly coupled with each other and makes them very hard to separate. It also makes it very hard to handle that fault tolerance that we talked about in the earlier episode, which, in case you don't remember, fault tolerance just means if one piece of your system goes down, then the rest of your application shouldn't be rendered unusable.
Speaker 1:Imagine a scenario where you're working on a simple social media app. Now imagine you have a few different services, but your specific services for sending a push notification goes down. One of your user tries to post a meme and everything is great until it sends a request to your notification service and it waits for that response saying, hey, we sent that out. Eventually, that call is going to time out and your user is going to have a bad experience. Now let's imagine that exact same system with a simple PubSub message broker. Your user posts their hilarious meme, the message goes to the message broker. The broker sends back a 200 saying great, everything's good on our end, and then processes that message into the exchange and it sends it through all the proper queues that it needs to. All the consumers bound to these queues are alive, except for the notification service. Right, what happens in this scenario? And feel free to pause and sort of challenge yourself and and guess before I take this answer Ready, Okay, so remember when I said the message goes into the broker. Well, like I said, the user at that point gets a success because our backend has acknowledged that we have that message. So then the services are all still alive, except for the one. They take the message and then process them as they need to. Maybe one service adds it to the user's feed, one adds it to the S3 bucket, but the notification service in particular, once it comes back online and depending, of course, on our protocol, either pulls the message from the queue for all new messages or grabs the latest one that it needs to process and sends out that notification correctly.
Speaker 1:A much less lengthy way of thinking about this is if you're using a microservice architecture. When building a system or improving a system, are there parts of that system that should be asynchronous? Ie, if a service goes down, will your entire application hang up waiting for the response? If the answer is ever yes, then it's probably a good idea to set up a message broker and save yourself a lot of heartbreak in the future. And with that we've taken a huge step into understanding the enigmas of microservice architectures and message brokers, more specifically, how message brokers work, why they are really useful and, when it comes to scaling and fault tolerance, why they're one of the best tools for the job.
Speaker 1:Next episode we're going to be jumping into what I believe is one of the most important parts of building out a microservice architecture, and that's load balancing. So if you haven't already, go ahead, subscribe to the podcast, because I'm very excited to talk about them and talk about the algorithms that go along with them. I really enjoyed everyone's feedback on the last episode. I hope these newer episodes improve in sound quality and are a lot easier and less distracting to hear.
Speaker 1:If you'd like to suggest any new topics or even be a guest on the podcast, feel free to drop me an email at LearnSystemDesignPod at gmailcom. Remember to include your name if you'd like a shout out. If you would like to support the podcast, help me pay my bills, help me pay off my student loans or even just help me hire someone to do more research, jump over to our Patreon. Consider becoming a member. All podcasts are inspired by Crystal Rose. All music is written and performed by the wonderful Aimless Orbiter. You can check out more of his music at soundcloudcom. Slash aimlessorbitermusic. And with all that being said, this has been and I'm scaling down Bye.
