In the last few months, we have seen a great deal of growth in the number of people moving from other systems to KumoMTA. This is partly the reason for the lack of blog posts lately - we have been a little busy :)
In other systems that have single-threaded queues, there is a real danger of mail getting "piled up," where 100,000 newsletter emails are blocking that one critical alert email that you need to get through immediately. In those systems, a way to prioritize critical email is essential. The older open-source standbys (Postfix, Exim, etc.) are all single-threaded, single-queue architecture, which is perfectly fine for the workloads they were designed to manage.
Some commercial MTAs offer multiple pathways or "virtual" MTA paths, but each of those is a single-threaded queue as well. For instance, if ABC company has messages flowing for marketing and also for alerts, they could potentially have two separate queues, but each of those is still single-threaded, so you ease the problem but cannot eliminate it. This is why they developed the "priority flag" system. If, for instance, a critical alert needs to be delivered immediately, but there are also 50,000 other alerts in the "alert" queue, you need a way to put the immediate one at the front of the line. Hence, the priority flag.
So, how do we set this in KumoMTA? We don't. We don't have one because it is not necessary.
KumoMTA will dynamically create a queue as needed for a particular message or set of messages based on data in the message stream itself - just a little more KumoMTA magic smart engineering. The initial scheduled queue will be created based on a combination of campaign, tenant, and domain. These combine to produce the name of the queue in the form of campaign:tenant@domain. When the message is actually ready for delivery, a path building process modifies that even more based on the egress source and mx record to create a more complex ready queue.
Let's say for instance that ABC Company is sending 3 newsletters and 2 alerts to customers at gmail and yahoo. KumoMTA might dynamically create queues like this:- newsletter:abc@gmail.com. x 2 messages
- newsletter:abc@yahoo.com x 1 message
- alerts:abc@gmail.com x 1 message
- alerts:abc@yahoo.com x 1 message
So the message stream has fanned out to 4 different queues with fewer messages to process in each. When you put this into a real-world situation, it fans out much more and can result in thousands of parallel queues. Now, that single alert message may only have to wait for 3 or 4 others, not 300,000 others. It makes the concept of a priority flag pointless.
Recently, one of our users ran some internal performance testing and shared numbers in the 20 Million per hour range. That sounds impressive on its own, but the more interesting number was that there were about 200,000 parallel queues, each with only a few messages. Injecting a single new alert message requires no extra flags or processing and it is up for its first delivery attempt in less than 200 microseconds (I did the math).
Investing in better queuing technology makes a priority flag seem pointless.
Let us know if this helped you. We would love to tell your story.
------------------------------------------------------------------
KumoMTA is the first open-source MTA designed from the ground up for the world's largest commercial senders. We are fueled by Professional Services and Sponsorship revenue.
Join the Discord | Review the Docs | Read the Blog | Grab the Code | Support the team