Azure Service Bus - Dead Letter Queue
We have not received that email said the sales person, here are the details. Can you please check?
That's a support ticket I received some time ago. I check the logs and I can see a record for that e-mail and that it was sent or was it not?
Context: The Messaging mechanism in the platform I'm working on is pretty simple, every time an e-mail needs to be sent, a message with the right payload is sent to a queue in Azure Service Bus and then a function app is waiting to pick and handle the message.
When a message is processed there is always the possibility that processing will fail and then the message is placed in a special queue, the Dead Letter Queue or DLQ.
The purpose of the dead-letter queue is to hold messages that can't be delivered to any receiver, or messages that couldn't be processed. Messages can then be removed from the DLQ and inspected. An application might, with help of an operator, correct issues and resubmit the message, log the fact that there was an error, and take corrective action. (from https://docs.microsoft.com/en-us/azure/service-bus-messaging/service-bus-dead-letter-queues)
By default, a message will be processed 10 times before it is sent in the DLQ, this is configuration via the MaxDeliveryCount settings. Unless explicitly set a DLQ will be available for every queue.
There are a couple of reasons why a message cannot be processed, such as an invalid payload or a dependent service not being available which was the case for me.
Service Bus Explorer (SBE) is a popular and open source tool for interacting with the Service Bus and there is also the option to interact via the portal.
outgoing-mail is the queue I am interested in, so I select and click the Dead Letter button, click ok and now I can review all the messages in the DLQ. Important to not that the operation will peek in the queue and won't remove any messages.
Once the failed messages are available, there is the option to delete them or re-submit for process. As I also wanted to debug this I opened the function app responsible for handling messages left in the outgoing-mail queue and switched to the Monitor section and then I've re-submitted the message.
As you can see this time the message was handled successfully and the sales team was able to receive the message. Of course this issue revealed an error in our process of handling messages and also in the logging mechanism that we should address, but that's a work in progress.
Thank you for reading.