SQS DeadLetterQueue redrive

Dead-letter queues

If a message can’t be consumed successfully, you can send it to a dead-letter queue (DLQ). Dead-letter queues let you isolate problematic messages to determine why they are failing.

When you designate a queue to be a source queue, a DLQ is not created automatically. You must first create a queue to designate as the DLQ. The DLQ queue type (standard or FIFO) must match the source queues. You can associate the same DLQ with more than one source queue.

The Maximum receives value determines when a message will be sent to the DLQ. If the ReceiveCount for a message exceeds the maximum receive count for the queue, Amazon SQS moves the message to the associated DLQ (with its original message ID).

You must use the same AWS account to create the DLQ and the source queues that send messages to the DLQ. Also, the DLQ must reside in the same region as the source queues that use the DLQ.

Maximum receives value, which defines how many times a message can be received before being sent to the dead-letter queue (valid range: 1 to 1,000).

DeadLetterQueue redrive -

Normal message delivery

AWS_REGION = os.getenv("AWS_REGION", "us-east-1")
# SQS_QUEUE_URL = os.getenv("SQS_QUEUE_URL")
SQS_QUEUE_URL = "https://sqs.us-east-1.amazonaws.com/4XXX880XXXX45/ingestions-queue-dead-letter"

def pretty_print_message(message: Dict[str, Any]) -> None:
    """Print important message fields in a readable format."""
    print("=" * 80)
    print(f"MessageId: {message.get('MessageId')}")
    print(f"ReceiptHandle: {message.get('ReceiptHandle')}")
    print(f"Attributes: {message.get('Attributes', {})}")

    body = message.get("Body", "")
    try:
        parsed = json.loads(body)
        print("Body (json):")
        print(json.dumps(parsed, indent=2, ensure_ascii=False))
    except json.JSONDecodeError:
        print("Body (text):")
        print(body)


def main() -> int:
    if not SQS_QUEUE_URL:
        print("ERROR: SQS_QUEUE_URL is not set.", file=sys.stderr)
        return 1

    sqs = boto3.client("sqs", region_name=AWS_REGION)

    print(f"Starting SQS consumer in region '{AWS_REGION}'")
    print(f"Queue: {SQS_QUEUE_URL}")
    print("Press Ctrl+C to stop.")

    try:
        while True:
            response = sqs.receive_message(
                QueueUrl=SQS_QUEUE_URL,
                MaxNumberOfMessages=10,
                WaitTimeSeconds=20,         # long polling
                VisibilityTimeout=5,       # time to process before message reappears
                MessageAttributeNames=["All"],
                AttributeNames=["All"],
            )

            messages = response.get("Messages", [])
            if not messages:
                continue

            for msg in messages:
                pretty_print_message(msg)

                # Delete after successful processing to avoid re-delivery
                sqs.delete_message(
                    QueueUrl=SQS_QUEUE_URL,
                    ReceiptHandle=msg["ReceiptHandle"],
                )
                print("Deleted message from queue.")

            # small sleep to avoid busy loop in edge cases
            time.sleep(0.1)

    except KeyboardInterrupt:
        print("\nStopped by user.")
        return 0
    except (BotoCoreError, ClientError) as e:
        print(f"AWS error: {e}", file=sys.stderr)
        return 2


if __name__ == "__main__":
    raise SystemExit(main())

First let’s explore how normal message delivery happy flow looks like. Here is the output of our Consumer that outputs all metainformation:

Queue: https://sqs.us-east-1.amazonaws.com/4XXX880XXXX45/ingestions-queue-dead-letter
================================================================================
MessageId: 6810de9a-06e2-4d47-bcde-1c2b59c6cb09
ReceiptHandle: AQEBzL4l2iRXfR+NFpiwhs0ybpEB9ozxoGsfdzeWPdocSCcEa0gqoMk9gZg/5ehclCA/3SdZ1xfgCWIMXbwQJppQ9yU+F872eKGAfBi/RBrL8hMg4K3KojNHq2rO0+xEzCuHvwQmQQIA3t/fz7wcifvX0VGBgxSpbnvk63E9hk6P15yf/gCjlIMpZf06sU+hOyRdqXJh43JvTOrmCixzBUB/lExkTdMauWIzU0cJwpO+B2xd5fa0Kv2AAnfaq3DDZxdz7jFF2r/KAuZ79HnGotfdJoss3pUdpi9I9ffXDyx0SVzd2fHTCD60u4Ye6PfO/cjjvPbqSxFy0jms1b4z52ixdJzzv0qgOLHqkXJpihOVGjEFFW6i3DP8xOxKBLW5edzrcYH9apBRN0tCoET/MXuQag==
Attributes: {'SenderId': 'AROAWVOQOP3URX5SNJ2LD:xxx.yyy@xxx.yyy', 'ApproximateFirstReceiveTimestamp': '1779804004772', 'ApproximateReceiveCount': '1', 'SentTimestamp': '1779804004766'}
Body (text):
test2
Deleted message from queue.

We will put attention to several attributes:

• MessageId: 6810de9a-06e2-4d47-bcde-1c2b59c6cb09
• ‘ApproximateReceiveCount’: ‘1’,
• ‘SentTimestamp’: ‘1779804004766’
• Body (text):

Comparing them with next experiments will provide the confidence in what happens in SQS under the hood.

We in

Starting SQS consumer in region 'us-east-1'
Queue: https://sqs.us-east-1.amazonaws.com/4XXX880XXXX45/ingestions-queue-dead-letter
Press Ctrl+C to stop.
================================================================================
MessageId: 497e9e73-1c01-46e9-bf7c-ef721a223d35
ReceiptHandle: AQEBqjXGk/CiC8SZGSEwDpTFLvFfBUr52nr9Z3iWJojv9L8+NlKi4wHc58VF7Vrqf6W5D2N06JNI5df/9uIVNEA/xuBCDdPA+3apBPUXBfbtmpX7J2AFLMa9AOEPweQLYi70WV75GfOPz6As3FJa75yMq4K1t6Q9Df71bNh5Zx4OS9PiapLYPaXmq4uGZe/4v7iSsWpxbTG7Hsgc4ii5s4930y+QMjxxkWEe979bnAE20DCj7zZCQ3T4v84VWy6hwVkk/BFFVsNoNlR26/u6J4gKkCn1qzeY9fZ9CPkA8lQeFgWymzzguMTFqYvLUFA4/kFjksdhia2BSIKkTJ+jAXAvyTT1kIwyA313BpGnCPY25Sbt5kprWQUWOgB9KcShaGaNteSox6aZqBTcCiNO+KHesg==
Attributes: {'SenderId': 'AROAWVOQOP3URX5SNJ2LD:xxx.yyy@xxx.yyy', 'ApproximateFirstReceiveTimestamp': '1779804763934', 'ApproximateReceiveCount': '1', 'SentTimestamp': '1779804763926'}
Body (text):
poisoned message
================================================================================
MessageId: 497e9e73-1c01-46e9-bf7c-ef721a223d35
ReceiptHandle: AQEBmRSRUHzIIxIbzTTC3KRjRA79ZoYhKREHmprz33T5cXursicHhGa+stEH9sBb6ZSXBlBf1r68Dy2zYSi0WlB5dM/DKY9+LVP9vb7LkDsa4IJlEc/fiYvBx8WCq3EsXAGBEKyje91CpJMg1HnI3F7fFi3+m1ejSk9WqnFKXnoXkctb2zrYvhVWzzI6y6NhdiEW5ewpru/6cLXjkkoF0RV/M//+CsjAR6La8gHyvVO4lnJw0UuAPzSfBAazIi/UOq95iJFq3UybdWluIsE8rQEA8RXjWAfjaj/T/tq3nE2bqV2Gy1uV39KmGIwcLBUw5gamuhqey5LerAjEDZe2qYIFIKA280sOva8MMZc1QZ6H1g+phsaG+ogUWJoKiDn9jyt+Igu8eUfTW6n67wq6JYrV3g==
Attributes: {'SenderId': 'AROAWVOQOP3URX5SNJ2LD:xxx.yyy@xxx.yyy', 'ApproximateFirstReceiveTimestamp': '1779804763934', 'ApproximateReceiveCount': '2', 'SentTimestamp': '1779804763926'}
Body (text):
poisoned message
================================================================================
MessageId: 497e9e73-1c01-46e9-bf7c-ef721a223d35
ReceiptHandle: AQEBBuZW4cwZ09unjLvRI5YS/DdfC++6yjS6GLShnwfZx6IfqorZ6ZiP7hFOoUha+e3+LaqFOCZpD5R93uUTWb2hMARxrRpfWJEP3GyaKbh6p128+RRvtr7GgC/n7vJpHSz901hNKyztWf8kL5kfaIu3ocfLYBoPCc0ca0wX3V3vL8MlUXoYXuhK4t0MyakCUHUM2MoJiKDcHvGtTnEL08CnD2WxDOR6SJ//lJlmH7JuOh7rLP25hhUKVYAkwHRwq3xLoEUce/IppMNRgGzQPTSApl1tBSwDWu7KoK4PiYmUOdJaUvcujW/Axaus3LbytU6oe1nVFdQpkYo42uI4Ebsqo0ac14CiDmqE/tWHcNSbGJj74h7axquI1/p0UOKHNxOJj6fhtD0LampxdbRjGI2isbyV1xzRjrn0140yGKks4Ps=
Attributes: {'SenderId': 'AROAWVOQOP3URX5SNJ2LD:xxx.yyy@xxx.yyy', 'ApproximateFirstReceiveTimestamp': '1779804763934', 'ApproximateReceiveCount': '3', 'SentTimestamp': '1779804763926'}
Body (text):
poisoned message
================================================================================
MessageId: 497e9e73-1c01-46e9-bf7c-ef721a223d35
ReceiptHandle: AQEBNj4rucG5xDy/b3wuhqs5qOX88sYSaTk81BjKkuLp2lLDXW01NtKjv26rvC2D1KEAl+MW3mIzSkjf6xP9rulfCucqnNVc5KKySKjDckH9LslbxDVZ2W4uggO7EGVYsc9M0kDq49h05rPj8l+jsxh0AWUrs67fM1mhCI0avXMj7C3yxIRuX7jcmZBsd5FP9ZRmhK0QjB/YMy8Q/+QjnlvTL2IydTOHCWYoCWxxvBoAZQDzHFNFUbg/VVOnrP/bpVabRK+AaFifh+x0c6rHdDzFcV0MGU3pm7Mstrz+0Z2UVAH4WBkmn3nzPYSuBekpUp3PHVkZT8XVUPW/UddLS+O3bkkCrOi3j23hRR0182Dn6loJWV1XqkH1Ps9jsuUNQ3b2B3a+MluIUbgD8Sm5DnxS4jnDe+CKUzBJvX8KybiQh4Q=
Attributes: {'SenderId': 'AROAWVOQOP3URX5SNJ2LD:xxx.yyy@xxx.yyy', 'ApproximateFirstReceiveTimestamp': '1779804763934', 'ApproximateReceiveCount': '4', 'SentTimestamp': '1779804763926'}
Body (text):
poisoned message
================================================================================
MessageId: 497e9e73-1c01-46e9-bf7c-ef721a223d35
ReceiptHandle: AQEBYOqyD3vtWSkLLezNvgAhDuqwnM2mVrcVbPpu/wtlGXzzm0ZrMPj0oHVcn9m/TvB6Q7TkRSzQczUeeJUZKb+2lC33+En3Ywc1DAEcdJ0lv2T7LCd3RTfeFSLzXl+jWq8IZAKTNdBlcwZgVxDgTQw9taF8ktSKaJxRzksoSvrJGMbx/qm5J0A5gR0/9ssgGHHzv+PgtRqlR2OHpfFMoycuVRDHdFMBC1WeXKecGiq/oHh9agIKImwp3FUOg+cxM0/XC57iqFN3Xfbds5CGyneYjXro5NpAUL7kGgE29mf2cWac4G906mRkeX0WKHwivk1cLga5Jb2DgkKv0g/15w/1OSEDskyS1eXs/gqkHue+xPhqmGAiqpLIJgEH6J3ZGTzEP9eBC3tJdL0ITCDAUKv3MYesdIzG/ie6ieuZfi4Ddkw=
Attributes: {'SenderId': 'AROAWVOQOP3URX5SNJ2LD:xxx.yyy@xxx.yyy', 'ApproximateFirstReceiveTimestamp': '1779804763934', 'ApproximateReceiveCount': '5', 'SentTimestamp': '1779804763926'}
Body (text):
poisoned message

So once messages are in DLQ they are available for analysis. Usually once engineer identify the root cause or data inconsistency, fixes and re-deployes code version those messages are ready for reprocessing.

Same application can have dlq-processing logic, or a dedicated service can perform this task

But here comes the same problem as with main Consumer - if there is some issue with messages processing then they will return back to DLQ and will be retried again and again.

Usually such anomalies are steal data that is not present in the state of system (some messages that were already deleted, some business transactions that were rollbacked) but received outdated state computation in message. So the business logic should handle it with Ids, timestamp and other states comparisson, also such queues are auto purges within some moment of time.

But want is interesenting that now this queue can be redrived to the same source SQS or even to other SQS, both with settings rate limmiting of requests per second if needed.

And this is very useful - the anomaly messages are isolated and are not locking the main queue. We can explore them in isolation to find the discrepancies. And once thos discrepencies are found and application logic is updated and released to handle corner case, we can redrive this messages into main SQS.

================================================================================
MessageId: b097edbf-1d12-4be9-aaf8-1323c2899d40
ReceiptHandle: AQEBS638kWZAPjIzk65GQuuDqBf7kmeXk5g+42hO2HN2U4TDIuzeyzAEGVRWP8YH0OvT1N1tijFjPIWdMtN5p/5G3T69FJdpEaI/yGnw3jjdwm+GxUySK5vQfQzmHT055ydHLMB1UJ5gh+qAWn0LP8kKfjtKXX5dZDXNFCKqflF3GsmLkUFLEGXFmQrvNu0KmYrpG0bXtHgWc+DOm7EOAb1nGuLOZl+f7eW9XHnbJQe+/BfzbZXTgj5xiryqzSN++MSiAaAg+QEUJJD7GK2HHReoaQmHRi4ZmGUFAEhAr8BfXcut2OxuDMAkgYumdkFuah7fxEHk99YWvO0nXl7EwafjA1HSUe9I7/283oZy41f1jxebIenRV3OqlvYGG46MlrMJYgnMoceSTd2qEmTJWmMl0A==
Attributes: {'SenderId': 'AROAWVOQOP3URX5SNJ2LD:xxx.yyy@xxx.yyy', 'ApproximateFirstReceiveTimestamp': '1779804833157', 'ApproximateReceiveCount': '1', 'SentTimestamp': '1779804833155'}
Body (text):
poisoned message
================================================================================
MessageId: b097edbf-1d12-4be9-aaf8-1323c2899d40
ReceiptHandle: AQEBlh1KG0/eurRWn3w7fCTvQgQR5bSZlb3vc9N128dSjn+sZLUpRT4ue6944TKoJg/KUyb3XAoQscwfkFQfokf7ahdWSFNVplTkmArk5IwmZkvlwOSu01TCQznaYiE21NFGn3ZWSurHB3JXVYUe3Gs8VbagE1/MruvYb2LGDVLx0St8zfoIsqA3M42ej5AF1N/zXJKoA2gD/vQ7neQ0DAm+iBeYtYcafApXD//zGevfyCFy6e2mMnc8O5rlChyNbH/OQck10XD3e3Mb0VcNs1RxPVRB4ICkSyIO+z5d8LG3xIzkKLKoI/s2sOwUGGCJWdC/x24+MCAwhQVGviceVtvq5JRm5GEz6Op5j/kvnOqgOIDUF9f/x7MMWMu8YmZJaMTAyxh9r5uelaERpNMkASBLAA==
Attributes: {'SenderId': 'AROAWVOQOP3URX5SNJ2LD:xxx.yyy@xxx.yyy', 'ApproximateFirstReceiveTimestamp': '1779804833157', 'ApproximateReceiveCount': '2', 'SentTimestamp': '1779804833155'}
Body (text):
poisoned message
================================================================================
MessageId: b097edbf-1d12-4be9-aaf8-1323c2899d40
ReceiptHandle: AQEBvQA44Yqxl+rCaFXOXVJMMyI97eE/ZOo4DcouRSr39eREdevvg0SkLbfyOoaivAzyW+YxtiRTs6dYivfXld3Q39yg+88oO8SneehoqHDg5GgLuxo0pBlkGCbcVpF/9VfD7Di3JLJQinPjy4B+L3ozg1WxJw0DBMsX1dOCutWRowIvPiT2E+j2mDReZwD+d47o+iFf+1TEYAStQU/7349VLFL4pzc0k9HqcfPqcWZZvOqPAurzmXcCEyAPNUoxnBKNCpUdvQ2yE7Vo9KPptNjUxnhwVcsDMafQkf1G4jRbl/hHQMsAlu5ZRlxO0LkszIE926ix9YpW3RA/isnkDJH4vM7CLRy4kPEDujO/OhOpOfc+ql1uwTgfE6bE846vqonckDh2Z5eRezr40M2ICNbofSSAyv92esvt7U4gJ6uXVzs=
Attributes: {'SenderId': 'AROAWVOQOP3URX5SNJ2LD:xxx.yyy@xxx.yyy', 'ApproximateFirstReceiveTimestamp': '1779804833157', 'ApproximateReceiveCount': '3', 'SentTimestamp': '1779804833155'}
Body (text):
poisoned message
================================================================================
MessageId: b097edbf-1d12-4be9-aaf8-1323c2899d40
ReceiptHandle: AQEBxRxwaIJLZouA/2u+tZmR2bUvy0x5Olyl65luxFbhWmcrFyI9Y3K3c4h/9ZA2HE2uivU119O17kPIhucNg72F7b8lYnwpQYinjOvEg+bMtAVXmkT8TosXtCVE454Kz67D0Nil1SLxzCnaB7UXE6F8IccaGJkC07UighCJV4yMtkcXXIy88zsfmRFNaXLkdgvrghNNeKaOWC5w69MpSceBpFJXxnmD4//xYFd+St6TR3Mu6roXLCWW6qTIH9mk/5Uj/QKuaVWFpg3L/ITX8UIT7qgmTF/c8xRl4Ktz4XaUP0QKxQ9Mm5s92AwR56xXFl42OH43dRwBN3DRgvQjkgfyYSGR3LSNEqBcWvLfLCrjTRjZ5WEMs1warGdV+9G7sIr4fA2oZZwEpWQHNRQjVz7IGzj9ZIOrx+WdKU4q9ASZKQg=
Attributes: {'SenderId': 'AROAWVOQOP3URX5SNJ2LD:xxx.yyy@xxx.yyy', 'ApproximateFirstReceiveTimestamp': '1779804833157', 'ApproximateReceiveCount': '4', 'SentTimestamp': '1779804833155'}
Body (text):
poisoned message
================================================================================
MessageId: b097edbf-1d12-4be9-aaf8-1323c2899d40
ReceiptHandle: AQEB93HNQRO6nafUwLUPrVuM1AS6f0c/PKDvVpNxzug/sHyLLd2xGyPuZWbFLy2gX+Lb10Fv8nfG7NpYdDU6xln3c6qrov7KpsoP0I7ucRLmNbNIAhC2qGrNpB3bnvvnT18RUBFsB8j3LR1Gxtzp/Joy1HSfWsEonvNLnKAwn3ps2EGwJ3UkDRTC0gomslEYwc9//SQb/QaELUcew1opja4mlrYfDgSXZ4CGMq9wWbOhO+qrqzr4hoDIeQyeZprsMKKPBtPYoBMCRL1RtfdvxSGTdzx8clpJYvKsYI5n7qgtAzFv2ipRAoa/A19TcOHTQcHTNjmEdH2YHPQlH231UPK268aLx2i009bMdxGl7PfxfTeRfggBLwH/pEpUkN9zSjXBHAnfjiYJzTvdehtyNyh5mu9+LnFnsw4A9VSEMDSbV/g=
Attributes: {'SenderId': 'AROAWVOQOP3URX5SNJ2LD:xxx.yyy@xxx.yyy', 'ApproximateFirstReceiveTimestamp': '1779804833157', 'ApproximateReceiveCount': '5', 'SentTimestamp': '1779804833155'}
Body (text):
poisoned message

Conclusion

Links:

Let’s update the script not to commit the ACK to SQS - we simulate that message was consumed, but during processing something failed so it stays in the SQS for retry.

    try:
        while True:
            response = sqs.receive_message(
                QueueUrl=SQS_QUEUE_URL,
                MaxNumberOfMessages=10,
                WaitTimeSeconds=20,         # long polling
                VisibilityTimeout=5,       # time to process before message reappears
                MessageAttributeNames=["All"],
                AttributeNames=["All"],
            )

            messages = response.get("Messages", [])
            if not messages:
                continue

            for msg in messages:
                pretty_print_message(msg)

                # Delete after successful processing to avoid re-delivery
                # sqs.delete_message(
                #     QueueUrl=SQS_QUEUE_URL,
                #     ReceiptHandle=msg["ReceiptHandle"],
                # )
                # print("Deleted message from queue.")

            # small sleep to avoid busy loop in edge cases
            time.sleep(0.1)