The Short Answer
TL;DR
Executive Summary
The Ambiguous Timeout Is the Whole Problem
What duplicatecheck Actually Does, and What It Does Not Do
The Seven Layers That Generate Duplicates
Two-Level Idempotency Keys
The Claim-Before-Send State Machine
Python: Celery, acks_late and the Duplicate Generator
Node.js: BullMQ, Stalled Jobs and Redis Claims
PHP and Laravel: ShouldBeUnique Is Not Idempotency
cURL and the Raw HTTP Contract
Batch Sends: One transactionId, Many Recipients
Cross-Channel Deduplication Across SMS and WhatsApp
OTP Is a Special Case
Reconciling With Delivery Reports
Duplicates You Cannot Prevent
Exactly-Once Is a Myth, Effectively-Once Is Achievable
Ten Steps to an Idempotent Send Path
Decision Matrix: Which Guard Belongs Where
Pre-Production Checklist
Ten Mistakes That Cause Duplicate Sends
FAQs

The Short Answer

A duplicate SMS is almost never caused by the gateway sending twice. It is caused by your application submitting twice, because a network timeout told you nothing about whether the first submission was accepted. The fix is to make the send path idempotent at the application layer: derive a deterministic business key from the thing that caused the message to exist, claim that key in a durable store with a uniqueness constraint before you open the HTTP connection, and treat the claim, not the HTTP response, as the record of truth. The gateway’s duplicatecheck parameter is not a substitute for this, because it de-duplicates the recipient list inside a single request rather than de-duplicating repeated requests over time.

TL;DR

A client-side timeout on POST https://unify.smsgateway.center/SMSApi/send is ambiguous. The message may have been accepted, billed and delivered. Retrying without a guard bills you twice and delivers twice.
Billing fires at submission, not delivery. SMSGatewayCenter states plainly that “Credits are non-refundable once SMS is successfully submitted to the operator.” A duplicate is not just a customer experience problem, it is a line item.
duplicatecheck is documented as “Enable to remove duplicate mobile numbers. Default is true.” That is within-request recipient deduplication. Do not model it as a cross-request retry guard.
Use a two-level key: a stable business key that identifies the logical message, and a per-attempt key that identifies the HTTP attempt. The business key is what you enforce uniqueness on.
Claim before send. Insert the business key into a table with a unique index, or SET NX it in Redis, and only then make the API call. Never make the call and hope to write state afterwards.
Celery acks_late and BullMQ stalled-job recovery are at-least-once by design. Enabling them without an idempotency guard converts a worker crash into a guaranteed duplicate message.
The Send SMS response carries a single top-level transactionId and an invalidMobile field. Per-recipient correlation comes from the delivery report API, not from the send response.
Cross-channel fallback between SMSApi/send and WAApi/send needs a channel-agnostic business key, otherwise a fallback becomes a double-notify.
Exactly-once delivery does not exist. At-least-once transport plus an idempotent receiver is what production systems actually build.

Executive Summary

Every messaging pipeline that grows past a few thousand sends a day eventually ships a duplicate. It usually happens on the worst possible day: a partial network incident, a slow upstream, a queue that redelivers a batch, and suddenly a cohort of customers gets the same OTP twice, or the same payment reminder three times. The support tickets are annoying. The regulatory exposure in India, where every commercial message is bound to a registered template and a consent record, is worse. The bill is worse still, because the platform charges on submission.

The reflex fix is to add a retry cap or a sleep. Both are wrong, because they attack frequency rather than correctness. A pipeline that sends a duplicate 1 percent of the time is not fixed by making it 0.1 percent. It is fixed by making duplicate submission structurally impossible for a given logical message, which means the send path needs an identity, and that identity needs to be enforced by something with a uniqueness guarantee.

This article builds that. It starts from the ambiguous timeout, which is the root cause almost every team misdiagnoses. It corrects a widespread misreading of the duplicatecheck parameter. It then walks the seven distinct layers of a real messaging stack that can each independently emit a duplicate, and gives a concrete guard for each. There is runnable code for Python with Celery, Node.js with BullMQ, and PHP with Laravel, plus the raw cURL contract. It covers batch semantics, cross-channel deduplication when you fall back from SMS to WhatsApp, the special constraints of OTP, reconciliation against delivery reports, and the small class of duplicates that originate below your application and cannot be prevented at all, only detected.

This is a companion to the SMS API retry strategy guide, which covers when to retry. This article covers how to retry without duplicating. Read both. A retry policy without an idempotency guard is a duplicate generator with a schedule.

The Ambiguous Timeout Is the Whole Problem

Consider the simplest possible send. Your application posts to https://unify.smsgateway.center/SMSApi/send with a 10 second timeout. At 10.0 seconds the client raises a timeout and your code catches it.

What do you actually know at that moment?

You know your process stopped waiting. That is all. You do not know whether the TCP connection was established, whether the request body was fully written, whether the platform parsed it, whether it debited your wallet, whether it handed the message to the operator, or whether the operator has already delivered it to the handset. The response that would have told you all of that is either still in flight, or was written to a socket you have already closed.

This is the two generals problem in its most expensive form. There is a taxonomy that matters here, and most HTTP clients obscure it:

Failure	What it means	Safe to retry blind?
DNS resolution failure	Nothing left your process	Yes
TCP connect timeout	No connection established, no bytes sent	Yes
TLS handshake failure	Connection established, no application data sent	Yes
Request write error before body flush	Partial or no request received	Almost always, but treat as ambiguous
Read timeout after full request sent	Unknown. Request may be fully processed	No
Connection reset after request sent	Unknown	No
HTTP 5xx from the platform	Platform received it and errored. May or may not have debited	No
HTTP 4xx with a parseable error body	Definitively rejected, nothing was sent	Retry only after fixing the request

The critical row is the read timeout. Python’s requests collapses connect and read timeouts under a single Timeout exception unless you inspect the subclass, which is precisely why so many Python pipelines double-send. httpx separates them into ConnectTimeout and ReadTimeout, and that distinction is worth switching clients for. In Node, fetch with AbortSignal.timeout gives you a TimeoutError that is inherently ambiguous, because the abort fires the same way whether the request was written or not.

The economics make this asymmetric. Consider a transactional send at the published 100,000 unit slab rate of INR 0.14 per SMS documented on the transactional SMS pricing page. One duplicate costs 0.14 rupees, which is nothing. One million sends with a 0.5 percent ambiguous-timeout rate and a blind retry costs 5,000 duplicate submissions, or 700 rupees, which is still not much. The cost is not the money. The cost is that 5,000 customers received a duplicate message, and if those were OTPs, an unknown fraction of them entered the wrong code, failed authentication, and either churned or called support. If they were promotional, you have 5,000 people with a concrete reason to reply STOP. The cost mechanics of over-sending are covered in more depth in the bulk SMS pricing breakdown, but the reputational cost dominates the credit cost by orders of magnitude.

So the rule is simple and absolute: an ambiguous outcome must never be resolved by re-submitting an unguarded request. It is resolved by re-submitting a request that carries an identity the system has already claimed, or by not re-submitting at all and reconciling later.

Diagram about The Ambiguous Timeout Is the Whole Problem

What duplicatecheck Actually Does, and What It Does Not Do

This section corrects a misconception that is common enough to be worth stating bluntly, and it is the single most useful paragraph in this article.

The Send SMS API accepts an optional parameter named duplicatecheck. The Send SMS API documentation describes it in the optional parameters table as:

duplicatecheck | true|false | Enable to remove duplicate mobile numbers. Default is true.

Read that carefully. It removes duplicate mobile numbers. The unit of deduplication is a recipient inside a single request, not a message across requests.

That means duplicatecheck=true protects you from exactly one thing: submitting a comma-separated recipient list where the same number appears more than once. If your CRM export contains 919999999991,919999999992,919999999991, the platform will collapse that to two recipients and you will be billed for two, not three. That is genuinely useful, and it is on by default, which is the right default.

What it does not do, and what you must not assume it does:

It does not remember a request you made ten seconds ago and suppress an identical one now.
It does not key on message content, so two different messages to the same number in one request are both sent.
It does not span requests, processes, workers, or channels.
It does not give you a stable identifier you can use to ask “did this already go out?”

The published documentation does not specify any cross-request deduplication window for this parameter, and no such window is documented on the site. Treat duplicatecheck as within-request recipient hygiene with no documented cross-request memory. If your architecture depends on a time-windowed duplicate suppression behaviour, confirm the exact semantics with support before you rely on it in production, because an undocumented behaviour is not a contract, and building your correctness story on one is how you end up with a Sev 1 after a platform upgrade.

There is a second, smaller trap in the same parameter. The documentation is internally inconsistent about capitalisation. The parameter table and the PHP and JSON samples use lowercase duplicatecheck, while the Python, cURL, Ruby, Go, Node.js, Swift and Google Apps Script samples on the same page use camelCase duplicateCheck. Form-encoded parameter names are frequently case-sensitive server-side. Since the default is true in either case, a silently ignored misspelled parameter is invisible until the day you explicitly set it to false and it does not take effect. If you are setting this parameter deliberately in either direction, verify with a two-identical-recipient test send against a number you control, and use whichever casing you observe working. Do not assume.

Here is the correct mental model, expressed as a table:

Guard	Scope	Where it lives	What it stops
`duplicatecheck=true`	One HTTP request	Platform	The same number appearing twice in one recipient list
Business key uniqueness	All time, all workers	Your database	The same logical message being submitted twice ever
Per-attempt claim TTL	Seconds to minutes	Redis or equivalent	Two workers racing on the same message right now
Queue job uniqueness	Queue lifetime	Celery, BullMQ, Laravel	The same job being enqueued twice while pending
Rate limit per recipient	Rolling window	Your application	A user hammering a resend button
DLR reconciliation	After the fact	Your database	Nothing. It detects, it does not prevent

Only the second row is idempotency. Everything else is defence in depth.

The Seven Layers That Generate Duplicates

A duplicate message is a symptom. There are seven distinct places it can originate, and they need different fixes. Diagnosing the wrong layer is why teams add retry caps that do not help.

Layer 1: The user

A customer taps “Resend OTP” four times in six seconds. Every tap is a legitimate, distinct API call from your application’s point of view. No idempotency key helps here, because these are genuinely different requests with different intent.

Fix: a per-recipient rate limit with a visible countdown in the UI, plus a server-side floor. The OTP endpoint already returns a retryAfter field as a millisecond epoch on generate; surface it and enforce it server-side rather than trusting the client to respect its own countdown.

Layer 2: The application entry point

Two HTTP requests arrive at your own API for the same business event, because an upstream service retried, a webhook fired twice, or a form was double-submitted.

Fix: derive the business key from the upstream event identifier, not from a random UUID generated at request time. A random UUID per request means two retries of the same event get two different keys, which is the same as having no key.

Layer 3: The HTTP client library

This one is invisible and catches almost everyone. urllib3.Retry in Python, when configured with allowed_methods including POST, will silently re-issue the request on a connection error or a 5xx. Your code sees one call. The platform sees two. Some HTTP client wrappers and service meshes do the same thing at the network layer.

Fix: set max_retries=0 on the adapter for the send path specifically. Transport-layer retries are safe for idempotent reads such as the delivery report endpoint. They are never safe for a billable write. If you run a service mesh, disable retries for this route in the mesh config too.

import httpx

# Send path: no transport-level retries, split timeouts.
send_client = httpx.Client(
    timeout=httpx.Timeout(connect=5.0, read=15.0, write=10.0, pool=5.0),
    transport=httpx.HTTPTransport(retries=0),
)

# Read path: transport retries are fine here.
report_client = httpx.Client(
    timeout=httpx.Timeout(connect=5.0, read=30.0, write=10.0, pool=5.0),
    transport=httpx.HTTPTransport(retries=3),
)

Note the write timeout being separate. A write timeout that fires after the body is partially flushed is ambiguous in exactly the same way a read timeout is. Only ConnectTimeout is unambiguously safe.

Layer 4: The queue

Celery with acks_late=True redelivers a task whose worker died mid-execution. BullMQ moves a job back to wait when its lock expires and the stalled-job checker picks it up. SQS redelivers after the visibility timeout. All of these are correct at-least-once behaviour and all of them will hand the same message to a second worker.

Fix: the claim-before-send pattern below. Do not disable at-least-once redelivery to avoid duplicates, because the alternative is at-most-once, which silently drops messages, and a dropped OTP is worse than a duplicate one.

Layer 5: The ambiguous timeout

Covered above. This is the layer everyone thinks is the only layer.

Fix: claim before send, plus a reconciliation pass that resolves unknown-outcome records against the delivery report API rather than by re-sending.

Layer 6: The batch boundary

You submit 5,000 numbers in one request. The response times out. You have no idea whether zero, some, or all of them were accepted. If you re-submit the batch you may double-send all 5,000.

Fix: treat a batch as a set of individually keyed messages, not as one atomic unit. Details in the batch section below.

Layer 7: The operator

Below your application entirely. If a handset does not acknowledge receipt over the air, the operator’s SMSC may retransmit, and the subscriber sees the message twice. This is documented behaviour across the industry and it is not something an API parameter can fix.

Fix: none available. This is why the last section of this article distinguishes between duplicates you prevent and duplicates you detect and absorb.

Layer	Duplicate mechanism	Guard	Prevents or detects
1. User	Repeated resend taps	Per-recipient rate limit plus `retryAfter`	Prevents
2. App entry	Upstream event replayed	Business key from upstream ID	Prevents
3. HTTP client	Transport-level auto-retry on POST	`retries=0` on the send adapter	Prevents
4. Queue	at-least-once redelivery	Claim before send, unique index	Prevents
5. Network	Ambiguous read timeout	Claim before send plus reconciliation	Prevents
6. Batch	Whole-batch re-submission	Per-recipient keys, chunked sends	Prevents
7. Operator	SMSC retransmission over the air	None	Detect only

Two-Level Idempotency Keys

The mistake that defeats most idempotency implementations is using one key for two jobs. You need two, and they have different lifetimes.

Level 1: the business key

This identifies the logical message and it must be derivable from the business event without any randomness. If the same event is processed a hundred times, the business key must come out identical a hundred times.

A good business key is a composite of the things that make the message unique in your domain:

{tenant_id}:{event_type}:{event_id}:{recipient_msisdn}:{template_id}

For an order shipment notification that would look like:

acme:order_shipped:ORD-88213:919999999999:1207161234567890123

Note what is not in there: no timestamp, no UUID, no attempt counter, no random salt. Any of those would make the key unique per attempt, which is precisely the failure mode.

Note also what is in there: the template ID. Two messages to the same customer about the same order, one an order-shipped notice and one an out-for-delivery notice, are different logical messages and must produce different keys. Including the registered DLT template ID is a cheap way to get that separation for free, and it aligns your deduplication boundary with your compliance boundary. If you are not yet registering templates, the DLT registration walkthrough explains what you need before any of this applies to Indian traffic.

Store it hashed if the raw key is long or contains a phone number you would rather not index in plaintext:

import hashlib

def business_key(tenant, event_type, event_id, msisdn, template_id) -> str:
    raw = f"{tenant}:{event_type}:{event_id}:{msisdn}:{template_id}"
    return hashlib.sha256(raw.encode("utf-8")).hexdigest()

A SHA-256 hex digest is 64 characters, fits comfortably in a VARCHAR(64) with a unique B-tree index, and does not leak the MSISDN into your logs or your index dumps.

Level 2: the attempt key

This identifies one HTTP attempt at delivering the logical message. It is random, it is new on every attempt, and its only job is correlation in your logs and traces. It never controls whether a send happens.

import uuid
attempt_key = uuid.uuid4().hex

Log both on every attempt. When you are debugging a duplicate at 2am, the question you need to answer instantly is “did the same business key produce two attempts, and if so, did the first one already claim?” That is only answerable if both identifiers are on every log line.

Why the platform’s transactionId is not your idempotency key

The Send SMS API returns a transactionId in the success response:

{
  "status": "success",
  "mobile": "919999999999",
  "invalidMobile": "",
  "transactionId": "6305583318236810379",
  "statusCode": "200",
  "reason": "success"
}

That identifier is excellent for reconciliation and it is what you correlate delivery reports against. It is useless as an idempotency key, for one structural reason: you only receive it if the request succeeded. The exact scenario you are guarding against is the one where you never see the response. A key you cannot obtain during a failure cannot protect you during a failure.

This is a general principle worth internalising. An idempotency key must be generated by the client, before the request, deterministically. Any identifier that originates server-side is a receipt, not a key.

Identifier	Generated by	Available before send	Stable across retries	Use for
Business key	Client	Yes	Yes	Deduplication, the only correctness guard
Attempt key	Client	Yes	No	Tracing and log correlation
`transactionId`	Platform	No	No	DLR correlation, billing reconciliation
`uuId` in DLR reports	Platform	No	No	Per-recipient report correlation
Queue job ID	Queue	Yes	Sometimes	Queue-level dedup only, never the sole guard

The Claim-Before-Send State Machine

Here is the core pattern. It is six states and two rules.

States: pending → claimed → submitted → delivered | failed_terminal, with unknown as a distinct terminal-pending state.

Rule one: claim before you send. The claim is a durable write with a uniqueness guarantee that happens before the HTTP request opens. If the claim fails because the key already exists, you do not send, full stop.

Rule two: an ambiguous outcome moves to unknown, never back to pending. A record in unknown is resolved by reconciliation against delivery reports, not by re-sending.

The second rule is the one teams get wrong. The instinct after a timeout is to put the message back on the queue. That instinct is the bug. A record whose outcome you do not know must not re-enter the send path until something external tells you it was not delivered.

The schema

CREATE TABLE outbound_message (
    id                BIGSERIAL PRIMARY KEY,
    business_key      VARCHAR(64)  NOT NULL,
    tenant_id         VARCHAR(64)  NOT NULL,
    channel           VARCHAR(16)  NOT NULL,      -- sms | whatsapp | voice | rcs
    recipient         VARCHAR(20)  NOT NULL,
    template_id       VARCHAR(32),
    body_hash         CHAR(64)     NOT NULL,
    state             VARCHAR(20)  NOT NULL DEFAULT 'pending',
    attempt_count     INT          NOT NULL DEFAULT 0,
    last_attempt_key  CHAR(32),
    transaction_id    VARCHAR(64),
    provider_status   VARCHAR(32),
    claimed_at        TIMESTAMPTZ,
    submitted_at      TIMESTAMPTZ,
    resolved_at       TIMESTAMPTZ,
    created_at        TIMESTAMPTZ  NOT NULL DEFAULT now(),

    CONSTRAINT uq_outbound_business_key UNIQUE (business_key)
);

CREATE INDEX ix_outbound_unknown
    ON outbound_message (state, submitted_at)
    WHERE state = 'unknown';

CREATE INDEX ix_outbound_txn
    ON outbound_message (transaction_id)
    WHERE transaction_id IS NOT NULL;

The unique constraint on business_key is doing all the work. Everything else is observability. Note the partial indexes: the unknown reconciliation sweep and the DLR correlation lookup are the only two queries that need to be fast at scale, and partial indexes keep them small.

Deliberately absent: a channel component inside business_key. That omission is intentional and the cross-channel section explains why.

Why a unique index and not a SELECT-then-INSERT

Because SELECT followed by INSERT is a race, and under exactly the conditions that produce duplicates, that race is not rare. Two workers that both pick up a redelivered job will both SELECT and find nothing, and both will INSERT. The database’s unique index is the only thing in your stack that can adjudicate that atomically. Let it.

from sqlalchemy.exc import IntegrityError

def claim(session, key: str, **fields) -> bool:
    """Return True if this caller owns the send, False if someone else already claimed it."""
    row = OutboundMessage(business_key=key, state="claimed", claimed_at=utcnow(), **fields)
    session.add(row)
    try:
        session.commit()
        return True
    except IntegrityError:
        session.rollback()
        return False

That is the whole guard. IntegrityError on the unique constraint means somebody else got there first, and the correct response is to return without sending, not to raise.

Redis as a fast-path claim

If your send volume makes a synchronous database write on the hot path uncomfortable, Redis SET key value NX EX ttl gives you the same atomicity with a fraction of the latency. It is a fast path, not a replacement, because Redis without AOF persistence can lose the claim on a restart.

def claim_fast(redis, key: str, attempt_key: str, ttl_seconds: int = 900) -> bool:
    return bool(redis.set(f"idem:{key}", attempt_key, nx=True, ex=ttl_seconds))

Pick the TTL deliberately. It must be longer than your worst-case end-to-end send latency including all retries, otherwise the claim expires while the first attempt is still in flight and a second worker sends. Fifteen minutes is a reasonable default for a path with a 15 second read timeout and three backoff attempts. If it is too short you get duplicates; if it is too long a genuinely failed send is blocked from legitimate reprocessing. When in doubt, err long and let the reconciliation sweep handle the stragglers.

The correct architecture uses both: Redis for the hot-path claim, the database unique index as the durable backstop that survives a Redis flush.

Diagram about The Claim-Before-Send State Machine

The state transition table

From	Event	To	Notes
(none)	Claim succeeds	`claimed`	Only now may you open the connection
(none)	Claim fails on unique violation	(no row created)	Return silently, do not send, do not error
`claimed`	HTTP 200 with `status: success`	`submitted`	Persist `transactionId`
`claimed`	HTTP 4xx with parseable error	`failed_terminal`	Do not retry without changing the request
`claimed`	Connect timeout or DNS failure	`pending`	Nothing was sent, safe to re-drive
`claimed`	Read timeout or connection reset	`unknown`	Never back to `pending`
`claimed`	HTTP 5xx	`unknown`	Platform received it, outcome unclear
`submitted`	DLR reports delivered	`delivered`	Terminal
`submitted`	DLR reports permanent failure	`failed_terminal`	Terminal. See the error code reference
`unknown`	Reconciliation finds a matching DLR	`submitted` or `delivered`	Resolved without re-sending
`unknown`	Reconciliation finds nothing after the SLA window	`pending`	Only now is a re-drive safe

That last row is where the judgement lives. How long you wait before declaring an unknown record genuinely un-submitted is a business decision, and it differs by message class. For an OTP, waiting is worse than a duplicate, so the window is short, perhaps 60 seconds, and you accept the duplicate risk. For a marketing message, waiting is free, so the window can be an hour. Encode this as a per-message-class policy, not a global constant.

Python: Celery, acks_late and the Duplicate Generator

Celery is where most Python messaging pipelines acquire their duplicate problem, and the irony is that it happens while trying to fix a reliability problem.

The Celery configuration documentation is the authority on these settings, and the default behaviour acknowledges a task the moment a worker picks it up. If the worker is killed mid-task, the message is gone and the SMS never goes out. That is at-most-once, and it silently drops OTPs, so teams set acks_late=True. Now the task is acknowledged only after it returns, and a worker killed mid-task causes the broker to redeliver to another worker. That is at-least-once, which is correct, and it means the send code will run twice for the same task, by design.

Stated plainly: acks_late=True without an idempotency guard is not a reliability improvement. It is a duplicate-message generator with better uptime.

# celery_app.py
from celery import Celery

app = Celery("messaging", broker="redis://localhost:6379/0")

app.conf.update(
    task_acks_late=True,                # at-least-once, correct
    task_reject_on_worker_lost=True,    # redeliver on SIGKILL, correct
    worker_prefetch_multiplier=1,       # do not hoard tasks in a dying worker
    task_acks_on_failure_or_timeout=False,
    broker_transport_options={"visibility_timeout": 3600},
)

Note visibility_timeout. If it is shorter than your task’s worst-case runtime, Redis redelivers a task that is still running, and you get two workers on the same message with no crash involved at all. Set it above your longest plausible task duration including all in-task retries.

And now the task itself, with the guard:

# tasks.py
import hashlib
import logging
import uuid

import httpx
from celery import Task
from celery.exceptions import Ignore

from .celery_app import app
from .db import session_scope
from .models import OutboundMessage

log = logging.getLogger(__name__)

SEND_URL = "https://unify.smsgateway.center/SMSApi/send"

_client = httpx.Client(
    timeout=httpx.Timeout(connect=5.0, read=15.0, write=10.0, pool=5.0),
    transport=httpx.HTTPTransport(retries=0),   # never auto-retry a billable POST
)


def business_key(tenant, event_type, event_id, msisdn, template_id) -> str:
    raw = f"{tenant}:{event_type}:{event_id}:{msisdn}:{template_id}"
    return hashlib.sha256(raw.encode("utf-8")).hexdigest()


@app.task(
    bind=True,
    name="messaging.send_sms",
    autoretry_for=(),          # deliberately empty: we control retries ourselves
    max_retries=3,
)
def send_sms(self: Task, *, tenant, event_type, event_id, msisdn,
             template_id, entity_id, body, senderid):

    key = business_key(tenant, event_type, event_id, msisdn, template_id)
    attempt_key = uuid.uuid4().hex

    log.info("send.start", extra={"business_key": key, "attempt_key": attempt_key,
                                  "celery_task_id": self.request.id,
                                  "delivery_count": self.request.delivery_info})

    # ---- CLAIM ----------------------------------------------------------
    with session_scope() as s:
        row = OutboundMessage(
            business_key=key, tenant_id=tenant, channel="sms",
            recipient=msisdn, template_id=template_id,
            body_hash=hashlib.sha256(body.encode()).hexdigest(),
            state="claimed", attempt_count=1, last_attempt_key=attempt_key,
        )
        s.add(row)
        try:
            s.flush()
        except Exception:            # IntegrityError on uq_outbound_business_key
            s.rollback()
            existing = s.query(OutboundMessage).filter_by(business_key=key).one()
            log.info("send.skipped_duplicate",
                     extra={"business_key": key, "existing_state": existing.state})
            raise Ignore()           # not an error, not a retry, just stop

    # ---- SEND -----------------------------------------------------------
    payload = {
        "userid": SGC_USERID,
        "password": SGC_PASSWORD,
        "sendMethod": "quick",
        "mobile": msisdn,
        "msg": body,
        "senderid": senderid,
        "msgType": "text",
        "dltEntityId": entity_id,
        "dltTemplateId": template_id,
        "duplicatecheck": "true",
        "output": "json",
    }

    try:
        # data=, never json=. This endpoint is application/x-www-form-urlencoded.
        resp = _client.post(SEND_URL, data=payload)
    except httpx.ConnectTimeout:
        # Nothing was submitted. Safe to release the claim and re-drive.
        _set_state(key, "pending")
        raise self.retry(countdown=_backoff(self.request.retries))
    except (httpx.ReadTimeout, httpx.WriteTimeout, httpx.RemoteProtocolError):
        # AMBIGUOUS. The message may already be sent and billed.
        _set_state(key, "unknown")
        log.warning("send.ambiguous", extra={"business_key": key})
        raise Ignore()               # reconciliation resolves this, not a retry

    if resp.status_code >= 500:
        _set_state(key, "unknown")
        raise Ignore()

    data = resp.json()
    if data.get("status") == "success":
        _set_state(key, "submitted", transaction_id=str(data.get("transactionId")),
                   provider_status=str(data.get("statusCode")))
        return {"transactionId": data.get("transactionId"),
                "invalidMobile": data.get("invalidMobile", "")}

    # Structured error from the platform. Classify before deciding.
    code = str(data.get("statusCode", ""))
    if code in PERMANENT_CODES:      # e.g. 188 template mismatch, invalid sender
        _set_state(key, "failed_terminal", provider_status=code)
        log.error("send.permanent_failure",
                  extra={"business_key": key, "statusCode": code,
                         "reason": data.get("reason")})
        raise Ignore()

    _set_state(key, "pending")
    raise self.retry(countdown=_backoff(self.request.retries))

Four things in there are load-bearing and worth calling out individually.

autoretry_for=() is deliberately empty. Celery’s automatic retry decorator does not know the difference between a connect timeout and a read timeout, so it will happily re-drive an ambiguous failure. Handle retries by hand on a billable path.

raise Ignore() on a duplicate is not an error. If you raise a normal exception, Celery marks the task failed, your error rate alarms fire, and someone spends an afternoon investigating a system that worked perfectly. A suppressed duplicate is a success.

ReadTimeout goes to unknown, never to retry. This is rule two from the state machine, expressed in code. It is the single most important line in the file.

data= not json=. The endpoint is form-encoded. Sending a JSON body to it produces a confusing authentication error rather than a content-type error, and engineers lose hours to it. This trap is covered in more depth in the Python integration tutorial.

The reconciliation sweep

The unknown state is only useful if something drains it. Run this on a schedule.

from datetime import datetime, timedelta, timezone

SLA = {"otp": timedelta(seconds=60),
       "transactional": timedelta(minutes=10),
       "promotional": timedelta(hours=1)}


@app.task(name="messaging.reconcile_unknown")
def reconcile_unknown():
    now = datetime.now(timezone.utc)
    with session_scope() as s:
        rows = (s.query(OutboundMessage)
                 .filter(OutboundMessage.state == "unknown")
                 .filter(OutboundMessage.submitted_at < now - timedelta(seconds=60))
                 .limit(500).all())

        for row in rows:
            dlrs = fetch_dlr(mobile_no=row.recipient,
                             fromdate=(row.claimed_at - timedelta(minutes=5)),
                             todate=now)
            match = _match_dlr(dlrs, row)
            if match:
                # It WAS submitted. Adopt the platform's record, do not re-send.
                row.transaction_id = match["msgId"]
                row.provider_status = match["status"]
                row.state = "delivered" if match["status"] == "DELIVRD" else "submitted"
                row.resolved_at = now
                continue

            window = SLA.get(row.message_class, timedelta(minutes=10))
            if now - row.claimed_at > window:
                # No trace after the SLA window. Now a re-drive is defensible.
                row.state = "pending"
                row.attempt_count += 1

fetch_dlr here wraps GET https://unify.smsgateway.center/SMSApi/reports/status with method=getDlr and a mobileNo filter, documented on the delivery report API page. Matching on recipient plus a time window plus the dltTemplateId is usually enough to identify the record with confidence. If it is not, you have an observability gap that no amount of retry tuning will close.

Node.js: BullMQ, Stalled Jobs and Redis Claims

BullMQ’s failure mode is different from Celery’s but arrives at the same place, and the stalled jobs documentation explains the mechanism precisely. A BullMQ worker holds a lock on an active job and renews it every lockDuration / 2 milliseconds. If the worker’s event loop blocks, or the process dies, or Redis fails over, the lock expires. The stalled-job checker then moves the job back to wait and another worker picks it up. maxStalledCount defaults to 1, so this happens at least once before the job is failed.

The subtle version of this bug has no crash in it at all. A synchronous CPU-bound operation on the event loop, such as rendering a large recipient file or a heavy regex, blocks lock renewal. The job is still running, but BullMQ believes it stalled and hands it to another worker. Now two workers are actively sending the same message concurrently. A guard with a TTL is the only thing that stops that.

// queue.js
import { Queue, Worker, UnrecoverableError } from 'bullmq';
import IORedis from 'ioredis';
import { createHash, randomUUID } from 'node:crypto';
import { setTimeout as sleep } from 'node:timers/promises';

const connection = new IORedis({ maxRetriesPerRequest: null });

export const smsQueue = new Queue('sms', { connection });

const SEND_URL = 'https://unify.smsgateway.center/SMSApi/send';

function businessKey({ tenant, eventType, eventId, msisdn, templateId }) {
  return createHash('sha256')
    .update(`${tenant}:${eventType}:${eventId}:${msisdn}:${templateId}`)
    .digest('hex');
}

/**
 * Atomic claim. Returns true only for the caller that wins.
 * TTL must exceed worst-case send duration including in-job backoff.
 */
async function claim(key, attemptKey, ttlSeconds = 900) {
  const res = await connection.set(`idem:${key}`, attemptKey, 'NX', 'EX', ttlSeconds);
  return res === 'OK';
}

The worker:

const worker = new Worker('sms', async (job) => {
  const key = businessKey(job.data);
  const attemptKey = randomUUID().replace(/-/g, '');

  // Claim in Redis first (fast path), then persist to Postgres (durable backstop).
  const won = await claim(key, attemptKey);
  if (!won) {
    job.log(`duplicate suppressed for business_key=${key}`);
    return { suppressed: true };          // resolve, do not throw
  }

  const durable = await db.claimDurable(key, job.data, attemptKey);
  if (!durable) {
    // Redis was flushed at some point; the database remembers. Trust the database.
    return { suppressed: true, source: 'db' };
  }

  const body = new URLSearchParams({
    userid: process.env.SGC_USERID,
    password: process.env.SGC_PASSWORD,
    sendMethod: 'quick',
    mobile: job.data.msisdn,
    msg: job.data.body,
    senderid: job.data.senderid,
    msgType: 'text',
    dltEntityId: job.data.entityId,
    dltTemplateId: job.data.templateId,
    duplicatecheck: 'true',
    output: 'json',
  });

  let res;
  try {
    res = await fetch(SEND_URL, {
      method: 'POST',
      headers: { 'content-type': 'application/x-www-form-urlencoded' },
      body,
      // fetch in Node has NO default timeout. Without this the job hangs forever.
      signal: AbortSignal.timeout(15_000),
    });
  } catch (err) {
    if (err.name === 'TimeoutError' || err.name === 'AbortError') {
      // AMBIGUOUS. Persist unknown, keep the Redis claim, do NOT throw.
      await db.setState(key, 'unknown');
      return { state: 'unknown' };
    }
    if (err.cause?.code === 'ENOTFOUND' || err.cause?.code === 'ECONNREFUSED') {
      // Nothing left the process. Release the claim so a retry can proceed.
      await connection.del(`idem:${key}`);
      await db.setState(key, 'pending');
      throw err;                      // let BullMQ retry with its backoff
    }
    await db.setState(key, 'unknown');
    return { state: 'unknown' };
  }

  if (res.status >= 500) {
    await db.setState(key, 'unknown');
    return { state: 'unknown' };
  }

  const data = await res.json();

  if (data.status === 'success') {
    await db.setState(key, 'submitted', {
      transactionId: String(data.transactionId),
      providerStatus: String(data.statusCode),
    });
    return { transactionId: data.transactionId, invalidMobile: data.invalidMobile };
  }

  if (PERMANENT_CODES.has(String(data.statusCode))) {
    await db.setState(key, 'failed_terminal', { providerStatus: String(data.statusCode) });
    // UnrecoverableError stops BullMQ retrying. Without it, a template mismatch
    // is retried three times and fails identically three times.
    throw new UnrecoverableError(`permanent ${data.statusCode}: ${data.reason}`);
  }

  await connection.del(`idem:${key}`);
  await db.setState(key, 'pending');
  throw new Error(`transient ${data.statusCode}: ${data.reason}`);
}, {
  connection,
  concurrency: 8,
  lockDuration: 60_000,        // must exceed worst-case job duration
  stalledInterval: 30_000,
  maxStalledCount: 1,
});

Three Node-specific points.

Returning rather than throwing on a suppressed duplicate. Throwing marks the job failed and, worse, triggers BullMQ’s own retry, which attempts the same suppressed send again. Resolve the job.

Releasing the claim on unambiguous failures only. On ENOTFOUND or ECONNREFUSED nothing was sent, so deleting the Redis key lets the retry proceed normally. On a TimeoutError the claim is deliberately left in place for its full TTL, because releasing it would allow a retry of a request that may already have been billed.

String(data.statusCode). The Send SMS API returns statusCode as a quoted string, while the WhatsApp API returns it as an unquoted number. Coerce at the boundary or your comparison silently fails on one of the two channels. This asymmetry and the rest of the Node failure surface are covered in the Node.js integration tutorial.

Queue-level uniqueness is not enough

BullMQ supports a custom jobId, and setting it to your business key gives you free deduplication at enqueue time:

await smsQueue.add('send', payload, {
  jobId: businessKey(payload),
  removeOnComplete: { age: 86400, count: 10000 },
  attempts: 3,
  backoff: { type: 'exponential', delay: 2000 },
});

This is worth doing, and it is not sufficient. jobId uniqueness only holds while the job exists in Redis. Once it completes and removeOnComplete evicts it, the same jobId can be added again and will run again. Queue-level uniqueness has the lifetime of the queue; business-key uniqueness has the lifetime of your data. Use both, and rely on the second.

PHP and Laravel: ShouldBeUnique Is Not Idempotency

Laravel’s ShouldBeUnique interface is frequently mistaken for an idempotency guarantee. It is not one, and the distinction matters.

ShouldBeUnique acquires a cache lock keyed on uniqueId() when the job is dispatched, and releases it when the job finishes processing. It prevents a second copy of the job from being queued while the first is pending or running. It does nothing once the first job completes, and critically, it does nothing about a job that is retried after a timeout, because that is the same job instance, holding the same lock, running its send code a second time.

<?php

namespace App\Jobs;

use Illuminate\Bus\Queueable;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Contracts\Queue\ShouldBeUnique;
use Illuminate\Foundation\Bus\Dispatchable;
use Illuminate\Queue\InteractsWithQueue;
use Illuminate\Queue\SerializesModels;
use Illuminate\Support\Facades\DB;
use GuzzleHttp\Client;
use GuzzleHttp\Exception\ConnectException;
use GuzzleHttp\Exception\RequestException;

class SendSms implements ShouldQueue, ShouldBeUnique
{
    use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;

    public int $tries = 3;
    public int $uniqueFor = 900;      // lock TTL, seconds

    public function __construct(
        public string $tenant,
        public string $eventType,
        public string $eventId,
        public string $msisdn,
        public string $templateId,
        public string $entityId,
        public string $body,
        public string $senderid,
    ) {}

    public function uniqueId(): string
    {
        return $this->businessKey();
    }

    public function businessKey(): string
    {
        return hash('sha256', implode(':', [
            $this->tenant, $this->eventType, $this->eventId,
            $this->msisdn, $this->templateId,
        ]));
    }

And the handler, where the real guard lives:

    public function handle(Client $http): void
    {
        $key = $this->businessKey();
        $attemptKey = bin2hex(random_bytes(16));

        // ---- CLAIM: durable, atomic, survives cache flushes ----
        $claimed = DB::transaction(function () use ($key, $attemptKey) {
            try {
                DB::table('outbound_message')->insert([
                    'business_key'     => $key,
                    'tenant_id'        => $this->tenant,
                    'channel'          => 'sms',
                    'recipient'        => $this->msisdn,
                    'template_id'      => $this->templateId,
                    'body_hash'        => hash('sha256', $this->body),
                    'state'            => 'claimed',
                    'attempt_count'    => 1,
                    'last_attempt_key' => $attemptKey,
                    'claimed_at'       => now(),
                    'created_at'       => now(),
                ]);
                return true;
            } catch (\Illuminate\Database\UniqueConstraintViolationException $e) {
                return false;
            }
        });

        if (! $claimed) {
            logger()->info('sms.duplicate_suppressed', ['business_key' => $key]);
            return;                    // NOT an exception. Just stop.
        }

        $form = [
            'userid'         => config('sgc.userid'),
            'password'       => config('sgc.password'),
            'sendMethod'     => 'quick',
            'mobile'         => $this->msisdn,
            'msg'            => $this->body,
            'senderid'       => $this->senderid,
            'msgType'        => 'text',
            'dltEntityId'    => $this->entityId,
            'dltTemplateId'  => $this->templateId,
            'duplicatecheck' => 'true',
            'output'         => 'json',
        ];

        try {
            $res = $http->post('https://unify.smsgateway.center/SMSApi/send', [
                'form_params'     => $form,
                'connect_timeout' => 5,
                'timeout'         => 15,
                'http_errors'     => false,   // keep JSON error bodies intact
            ]);
        } catch (ConnectException $e) {
            // Connection never established. Nothing sent. Safe to release and retry.
            $this->setState($key, 'pending');
            $this->release($this->backoffSeconds());
            return;
        } catch (RequestException $e) {
            // Request was written, response never completed. AMBIGUOUS.
            $this->setState($key, 'unknown');
            logger()->warning('sms.ambiguous', ['business_key' => $key]);
            return;                    // reconciliation resolves it
        }

        if ($res->getStatusCode() >= 500) {
            $this->setState($key, 'unknown');
            return;
        }

        $data = json_decode((string) $res->getBody(), true) ?? [];

        if (($data['status'] ?? null) === 'success') {
            $this->setState($key, 'submitted', [
                'transaction_id'  => (string) ($data['transactionId'] ?? ''),
                'provider_status' => (string) ($data['statusCode'] ?? ''),
                'submitted_at'    => now(),
            ]);
            return;
        }

        $code = (string) ($data['statusCode'] ?? '');
        if (in_array($code, self::PERMANENT_CODES, true)) {
            $this->setState($key, 'failed_terminal', ['provider_status' => $code]);
            $this->fail(new \RuntimeException("permanent {$code}: " . ($data['reason'] ?? '')));
            return;
        }

        $this->setState($key, 'pending');
        $this->release($this->backoffSeconds());
    }
}

The distinction between $this->release() and $this->fail() is worth internalising. release() puts the job back for another attempt and is correct for transient failures. fail() marks it permanently failed and stops the retry chain, which is what you want for a template mismatch or an invalid sender ID, because those will fail identically on every attempt and each attempt costs you a queue slot and a log line. Returning without either, which is what the ambiguous branch does, simply completes the job and leaves the durable record in unknown for reconciliation.

ConnectException versus RequestException in Guzzle maps almost exactly onto ConnectTimeout versus ReadTimeout in httpx. ConnectException is thrown when the connection could not be established, so nothing was transmitted. RequestException covers everything after that, and is therefore ambiguous. Do not catch them together. The broader PHP integration surface is covered in the PHP integration tutorial.

Laravel mechanism	Prevents	Does not prevent
`ShouldBeUnique`	Two dispatches while one is pending	Re-running after completion, or a retried instance re-sending
`$tries` cap	Unbounded retry loops	Duplicates within the cap
`$backoff`	Thundering herd on recovery	Anything about identity
DB unique index on `business_key`	Every duplicate submission, permanently	Operator-level retransmission

cURL and the Raw HTTP Contract

Everything above is language plumbing over one HTTP call. Here is the call, unadorned, so you can reason about exactly what crosses the wire.

curl -X POST \
  https://unify.smsgateway.center/SMSApi/send \
  -H 'apikey: YOUR_API_KEY' \
  -H 'content-type: application/x-www-form-urlencoded' \
  --connect-timeout 5 \
  --max-time 15 \
  --no-retry-all-errors \
  -d 'sendMethod=quick' \
  -d 'mobile=919999999999' \
  -d 'msg=Your order ORD-88213 has shipped.' \
  -d 'senderid=SGCTST' \
  -d 'msgType=text' \
  -d 'dltEntityId=1201159xxxxxxxxxxx' \
  -d 'dltTemplateId=1207161xxxxxxxxxxx' \
  -d 'duplicatecheck=true' \
  -d 'output=json'

Note --connect-timeout and --max-time are separate flags for the same reason httpx separates them. Note also the deliberate absence of --retry. Adding --retry 3 to a billable POST in a shell script is the shell equivalent of the urllib3 trap, and it is a common way for an operations runbook to quietly double-send during an incident.

A success looks like this:

{
  "status": "success",
  "mobile": "919999999999",
  "invalidMobile": "",
  "transactionId": "6305583318236810379",
  "statusCode": "200",
  "reason": "success"
}

There is no idempotency header on this endpoint. That is not a criticism, it is a design fact you have to build around: the deduplication responsibility sits with the client. Every code sample above exists because of that one sentence. If you want a request-level idempotency key honoured by the platform, ask support whether it is on the roadmap rather than assuming an undocumented header works.

For interactive exploration, import the official SMS Postman collection and set your credentials as collection variables. Do your duplicate testing there against a number you control before you write a line of application code. Send the same request twice, thirty seconds apart, and observe what actually arrives on the handset. That two-minute experiment will teach you more about your duplicate exposure than any amount of reading.

Batch Sends: One transactionId, Many Recipients

Batch sends are where idempotency reasoning most often breaks down, because the natural unit of the API call and the natural unit of your business logic are different sizes.

The Send SMS endpoint accepts a comma-separated recipient list in the mobile parameter. The documentation describes the endpoint as handling “single SMS or lists of comma-separated mobile numbers”, and the code samples confirm the format, showing values such as mobile=919xxxxxxxxx,919999999998.

Two properties of the response shape drive everything else:

There is one top-level transactionId. The documented success response carries a single transactionId field, not an array of per-recipient identifiers. That means you cannot map a specific recipient back to a specific identifier from the send response alone. Per-recipient correlation has to come from the delivery report API, where each report row carries mobileNo, msgId and uuId.

There is an invalidMobile field. The response includes invalidMobile, empty in the success sample. This is the partial-rejection channel: numbers the platform could not accept are enumerated there while the overall status still reads success. Any batch send handler that checks only status and ignores invalidMobile will record a batch as fully sent when part of it was rejected, and those recipients will silently never receive anything.

data = resp.json()

if data.get("status") == "success":
    invalid = [m for m in (data.get("invalidMobile") or "").split(",") if m.strip()]
    accepted = [m for m in submitted_numbers if m not in invalid]

    mark_submitted(accepted, transaction_id=str(data["transactionId"]))
    mark_terminal(invalid, reason="invalidMobile")   # do NOT retry these

    if invalid:
        log.warning("send.partial", extra={"accepted": len(accepted),
                                           "invalid": len(invalid)})

Marking invalid numbers terminal rather than retryable matters. A malformed MSISDN will be rejected identically on every attempt, so retrying it burns queue capacity to no effect. Fix it upstream with proper E.164 normalisation.

How to make batches idempotent

The pattern is to keep the business key per recipient while keeping the HTTP call per batch:

Generate one business key per recipient, exactly as for a single send.
Claim all of them in one database statement, using INSERT ... ON CONFLICT DO NOTHING RETURNING business_key. The returned set is the recipients you actually own.
Build the comma-separated mobile value from only the returned set. Recipients that were already claimed are simply absent from the request.
Send. On success, mark the returned set submitted against the shared transactionId, minus anything in invalidMobile.
On an ambiguous timeout, mark all claimed recipients in that batch unknown. You do not know which subset was processed, so you must assume all of them might have been.

INSERT INTO outbound_message (business_key, tenant_id, channel, recipient,
                              template_id, body_hash, state, claimed_at)
SELECT k.business_key, k.tenant_id, 'sms', k.recipient,
       k.template_id, k.body_hash, 'claimed', now()
FROM   unnest($1::outbound_claim[]) AS k
ON CONFLICT (business_key) DO NOTHING
RETURNING business_key, recipient;

Step five is the one people resist, because marking 5,000 records unknown feels like giving up. It is not. It is refusing to guess. The reconciliation sweep will resolve every one of them from delivery reports within minutes, and the alternative, re-sending the batch, is a guaranteed 5,000-message duplicate event if the first submission actually landed.

Keep batches small enough that an ambiguous outcome is cheap to reconcile. A batch of 500 that has to be reconciled is a minor query. A batch of 50,000 in unknown is an incident. Batch size is a reliability parameter, not just a throughput parameter.

Batch failure	What you know	Correct action
`status: success`, empty `invalidMobile`	All accepted	Mark all `submitted`
`status: success`, populated `invalidMobile`	Partial acceptance, listed numbers rejected	Mark accepted `submitted`, listed `failed_terminal`
HTTP 4xx with error body	Nothing accepted	Fix the request, then re-drive
Connect timeout	Nothing sent	Release claims, re-drive
Read timeout	Unknown for every recipient	Mark all `unknown`, reconcile
HTTP 5xx	Unknown for every recipient	Mark all `unknown`, reconcile

Cross-Channel Deduplication Across SMS and WhatsApp

Multi-channel fallback is the feature most likely to turn a well-behaved pipeline into a duplicate machine, because the failure it is designed to handle is the exact failure that leaves you uncertain.

The scenario: you send an order update over WhatsApp using POST https://unify.smsgateway.center/WAApi/send. It times out. Your fallback logic fires and sends the same content over SMS via SMSApi/send. That one succeeds. Then the original WhatsApp request, which was in fact accepted, delivers. The customer gets the same message on two channels.

The fix is structural, and it is why the schema earlier deliberately excludes channel from business_key.

The business key identifies the notification, not the delivery attempt on a particular channel. One order-shipped event for one customer produces one business key, regardless of how many channels you try. The channel column records which channel won. It is not part of the identity.

CHANNEL_ORDER = ["whatsapp", "sms"]

def notify(event, recipient):
    key = business_key(event.tenant, event.type, event.id,
                       recipient.msisdn, event.template_id)   # channel-free

    if not claim(key):
        return "already_handled"

    for channel in CHANNEL_ORDER:
        outcome = attempt(channel, key, event, recipient)

        if outcome.state == "submitted":
            set_channel(key, channel)
            return "sent_via_" + channel

        if outcome.state == "unknown":
            # The message may already be live on this channel.
            # Falling through would be a cross-channel duplicate.
            set_state(key, "unknown", channel=channel)
            return "unknown_no_fallback"

        if outcome.state == "failed_terminal" and not outcome.channel_specific:
            # e.g. the recipient number itself is invalid. No channel will help.
            return "terminal"

        # Only a clean, channel-specific, definite failure falls through.

    set_state(key, "failed_terminal")
    return "exhausted"

The critical branch is the unknown one. Most fallback implementations treat “I did not get a success response” as “it failed, try the next channel”. That is wrong, and it is wrong specifically for timeouts, which are the most common trigger for fallback in the first place. Fall back only on definite, channel-specific failure. An ambiguous outcome must stop the chain.

This is more conservative than the platform’s own default. The knowledge base entry on trigger error codes in multi-channel fallback states that “By default, any failure on a channel (except the last) can trigger the next step. You can narrow this to specific delivery error codes only.” Narrow it. Enumerate the delivery error codes that genuinely mean the channel cannot deliver, such as an unreachable handset or an absent subscriber, and exclude anything that could mean “accepted but unconfirmed”. The platform-side multi-channel fallback feature and your application-side fallback should not both be active on the same message, or you get two independent fallback engines racing.

WhatsApp has an additional trap

WhatsApp fails differently from SMS, and it fails in a way that is hostile to timeout-based reasoning. An SMS that cannot be delivered produces a delivery report with a cause code, eventually. A WhatsApp message sent to a number that is not on WhatsApp, or outside a valid session window, can be accepted by the API and then simply never arrive, with no timeout and no error at your layer.

That asymmetry has two consequences for deduplication. First, you cannot use silence as evidence of non-delivery on WhatsApp the way you partially can on SMS. Second, no WhatsApp-specific delivery report endpoint is documented on the site, so the reconciliation pattern that works for SMS does not have a confirmed equivalent for WhatsApp. Until that is confirmed, treat a WhatsApp unknown as unresolvable by automation and route it to a manual queue or let it expire, rather than building a reconciler on an assumed endpoint. The WhatsApp reports page is the place to check for current capability before you build.

The parsing asymmetry matters here too. SMSApi/send returns "statusCode": "200" as a quoted string. WAApi/send returns "statusCode": 200 as a number, alongside messageId rather than transactionId and description rather than reason. A shared response parser must normalise all three or your cross-channel state machine will mis-classify one channel silently.

def normalise(channel: str, data: dict) -> dict:
    if channel == "sms":
        return {"ok": data.get("status") == "success",
                "id": str(data.get("transactionId") or ""),
                "code": str(data.get("statusCode") or ""),
                "reason": data.get("reason") or ""}
    return {"ok": data.get("status") == "success",
            "id": str(data.get("messageId") or ""),
            "code": str(data.get("statusCode") or ""),
            "reason": data.get("description") or ""}

The full channel-selection reasoning, including why WhatsApp is a poor primary OTP channel, is covered in the SMS API versus WhatsApp Business API comparison.

Channel	Success identifier	statusCode type	Ambiguity signal	Reconcilable
SMS via `SMSApi/send`	`transactionId`	Quoted string	Read timeout, 5xx	Yes, via `SMSApi/reports/status`
WhatsApp via `WAApi/send`	`messageId`	Unquoted number	Read timeout, 5xx, plus silent non-delivery	Not confirmed publicly

OTP Is a Special Case

Every rule above holds for OTP with one inversion: for OTP, the cost of a missing message exceeds the cost of a duplicate one.

A customer who receives two identical OTP codes is mildly confused for four seconds. A customer who receives none abandons the signup. So the reconciliation SLA for OTP should be aggressive, measured in tens of seconds, and you should accept a higher duplicate rate in exchange for a lower drop rate. Encode that as policy rather than treating all traffic identically.

But OTP has its own duplicate mechanic that has nothing to do with timeouts, and it is a security problem rather than a UX one. If a user, or a script, triggers OTP generation repeatedly, each call produces a new code. Beyond the direct cost, which is real if someone is running an SMS pumping attack against your signup endpoint, you get a race where the user reads code A from their first message while your system now only accepts code C.

The OTP generate endpoint returns retryAfter alongside createTime and expiryTime, all as millisecond epochs, with statusCode 300 on success. Use it:

import time

resp = client.post("https://unify.smsgateway.center/SMSApi/otp", data={
    "userid": USERID, "password": PASSWORD,
    "sendMethod": "generate",
    "mobile": msisdn,
    "msgType": "text",
    "msg": "Your verification code is $otp$. Valid for 5 minutes.",
    "medium": "sms",
    "codeType": "num",
    "codeLength": 6,
    "codeExpiry": 300,
    "senderid": SENDERID,
    "output": "json",
})
data = resp.json()

if str(data.get("statusCode")) == "300":
    retry_after_ms = int(data["retryAfter"])
    seconds = max(0, (retry_after_ms - int(time.time() * 1000)) // 1000)
    store_resend_floor(msisdn, seconds)      # enforce server-side, not just in the UI

Note that msg carries the literal $otp$ placeholder and the platform substitutes the generated code. You never generate the code yourself on this endpoint, which also means you never have a client-side value to key on. Deduplicate on the verification session, not the code: one business key per signup attempt, not per OTP request.

Rate limiting the resend is not optional. Enforce it server-side against the MSISDN with a rolling window, because a client-side countdown is trivially bypassed and SMS pumping fraud specifically targets endpoints that only enforce it in the UI. Product context for the OTP flow is on the OTP SMS page.

Reconciling With Delivery Reports

Reconciliation is what makes the unknown state safe. Without it, unknown is just a place records go to die.

The delivery report API is a GET or POST to https://unify.smsgateway.center/SMSApi/reports/status with method=getDlr, a fromdate and todate, a pageLimit, and optional mobileNo, uuId or startCursor filters. The response contains a reports_dlrList array whose rows carry mobileNo, msgId, uuId, status, cause, submitTime, deliveryTime, dltTemplateId, cost, length, network and senderName.

Note that this is a read endpoint, so unlike the send path it is safe to configure transport-level retries against it.

def reconcile_one(row) -> str | None:
    """Return the resolved state, or None if no matching report exists yet."""
    reports = fetch_dlr(
        mobile_no=row.recipient,
        fromdate=(row.claimed_at - timedelta(minutes=2)).strftime("%Y-%m-%d %H:%M:%S"),
        todate=(row.claimed_at + timedelta(minutes=30)).strftime("%Y-%m-%d %H:%M:%S"),
        page_limit=100,
    )

    for r in reports:
        if r.get("mobileNo") != row.recipient:
            continue
        if row.template_id and r.get("dltTemplateId") != row.template_id:
            continue
        submit = parse_ts(r.get("submitTime"))
        if not (row.claimed_at - timedelta(minutes=2) <= submit
                <= row.claimed_at + timedelta(minutes=5)):
            continue

        row.transaction_id = r.get("msgId")
        row.provider_status = r.get("status")
        row.cost = Decimal(str(r.get("cost", "0")))
        return "delivered" if r.get("status") == "DELIVRD" else "submitted"

    return None

Three filters compose the match: recipient, template ID, and a submit-time window bracketing the claim. Any one alone is too loose. Recipient alone will match an unrelated message to the same customer. The time window alone will match a different customer. Together they are usually decisive, and where they are not, you have discovered that your traffic pattern sends multiple identical messages to the same person in the same minute, which is a finding worth acting on independently.

The cost field is what makes this a billing control and not just a state cleanup. Summing cost across reconciled rows and comparing against your own submitted count gives you a running duplicate-spend figure. The cost model behind that number is worked through in the bulk SMS pricing breakdown.

Webhooks are faster but do not replace polling

Registering a webhook via POST https://unify.smsgateway.center/SMSApi/webhook/create gets reports pushed to you in near real time, which shrinks the unknown window from minutes to seconds. Details are on the create webhook page and the real-time DLR webhooks article.

Two cautions. First, your webhook receiver must itself be idempotent, because a webhook that is not acknowledged fast enough will be redelivered, and a receiver that blindly appends state on every call will corrupt its own record. Persist raw, deduplicate on the report identifier, then process asynchronously. Second, a webhook you never receive is indistinguishable from a message that was never submitted, so the polling reconciler stays as the backstop. Push for latency, poll for correctness.

The meaning of each DLR status value is catalogued in the DLR status meanings reference, and the general model of delivery reporting is covered in understanding delivery reports.

Duplicates You Cannot Prevent

Honesty about the boundary of your control is part of building a credible system.

Below your application, below the platform, the operator’s SMSC performs its own retransmission. If a handset does not acknowledge receipt over the air, because it was in a lift, or switching cells, or had a full inbox, the SMSC may retransmit. The subscriber sees the message twice. Your logs show one submission, one delivery report, one charge. Nothing in your stack is wrong.

Vendor documentation across the industry describes this behaviour: where a device cannot signal that it received an SMS, the operator treats it as undelivered and retries delivery until it receives a successful acknowledgement. That mechanism is invisible and unreachable from any API.

There is a second category, also outside your control: aggregator-level retries on a route between the platform and the operator. Multi-hop delivery paths can retransmit at a hop you have no visibility into.

Practical consequences:

Do not chase a zero-duplicate SLA. It is unachievable, and pursuing it leads to over-conservative logic that drops real messages.
Instrument what you can attribute. Distinguish “duplicate submissions from our systems”, which should be zero and is your responsibility, from “duplicate handset receipts”, which will not be zero and is not.
Design message content to survive duplication. A message that reads naturally when received twice costs you nothing. “Your OTP is 449182” is fine twice. “You have 1 new message” followed by an identical copy is confusing but harmless. Anything phrased as an incremental update, such as “your balance has been reduced by 500”, is actively dangerous when duplicated, because the reader cannot tell whether it happened once or twice. State absolutes, not deltas.

That last point is a content design rule that engineers rarely think of as their problem, and it is the cheapest duplicate mitigation available. Write templates that are idempotent to read.

Exactly-Once Is a Myth, Effectively-Once Is Achievable

It is worth being precise about the theory, because the vocabulary shapes the architecture.

Exactly-once delivery is impossible in a distributed system. This is not an engineering limitation to be solved with a better library, it is a consequence of the two generals problem: two parties communicating over an unreliable channel cannot reach certain agreement, no matter how many messages they exchange. The sender can never know whether the last acknowledgement was lost, and neither can the receiver. Any protocol that appears to achieve exactly-once delivery has moved the uncertainty somewhere else, not eliminated it. The argument is laid out at length in You Cannot Have Exactly-Once Delivery, which remains the clearest treatment of it.

What is achievable is exactly-once processing, sometimes called effectively-once, and it is built from two ordinary parts:

At-least-once delivery. The transport retries until acknowledged. This produces duplicates and that is fine and expected.
An idempotent receiver. The processing side recognises repeats by identity and executes the side effect only once.

Everything in this article is an instance of that pattern. Celery acks_late and BullMQ stalled-job recovery supply part one. The business key and its unique constraint supply part two. Neither alone is correct. Together they are.

The reason this framing matters practically is that it tells you where to spend effort. Teams often try to make part one perfect, tuning timeouts and retry counts to make duplicates rare. That work has diminishing returns and never reaches zero. The same effort spent on part two reaches correctness, because a system with an idempotent receiver is indifferent to how many duplicates part one generates. Make the transport aggressive and the receiver strict, not the other way around.

The one caveat specific to messaging: your side effect is a physical SMS arriving on a handset, which you cannot roll back. In a database system, an idempotent receiver can detect a duplicate and undo. Here, once submitted, it is submitted, billed, and en route. That is why the guard must be before the call, not after it. In most idempotency literature the check happens on receipt of a request; here it must happen before emission of one.

Guarantee	Achievable	What it costs	Right for messaging
At-most-once	Yes	Silent message loss	No. A dropped OTP is worse than a duplicate
At-least-once	Yes	Duplicates	Yes, as the transport layer
Exactly-once delivery	No	Not available at any price	Not a design option
Effectively-once processing	Yes	A unique index and discipline	Yes. This is the target

Ten Steps to an Idempotent Send Path

Use this as the implementation order. Each step is independently shippable and each one reduces duplicate exposure on its own.

Step 1. Define the business key. Write down, in one line, what makes a message unique in your domain. Compose the key from a tenant identifier, an event type, an event identifier, the recipient MSISDN and the template identifier. Verify by hand that processing the same event twice produces byte-identical keys.

Step 2. Add the table and the unique constraint. Create outbound_message with UNIQUE (business_key). The constraint is the guarantee. Everything else in the table is observability.

Step 3. Move the claim before the HTTP call. Find every place your code calls the send endpoint and insert the claim immediately above it, inside the same function. If the claim and the send can be separated by a queue boundary, a network hop, or a conditional, they will eventually diverge.

Step 4. Turn off transport-level retries on the send path. Set retries=0 in httpx, max_retries=0 on a requests adapter, remove --retry from shell scripts, and disable route-level retries in any service mesh in front of the call. Leave them enabled on the delivery report endpoint.

Step 5. Split your timeout handling. Catch connect failures separately from read failures. Connect failures go to pending. Read failures, write failures, connection resets and 5xx responses go to unknown. Never let an ambiguous outcome reach the retry branch.

Step 6. Classify platform errors into permanent and transient. Pull the authoritative list from POST https://unify.smsgateway.center/SMSApi/info/responsecodes, documented on the API response error code list page. Permanent codes such as a template mismatch must terminate, not retry.

Step 7. Build the reconciliation sweep. A scheduled job that reads unknown records older than a per-message-class SLA, queries the delivery report API filtered by recipient and time window, and resolves each record to submitted, delivered, or back to pending only after the SLA window has elapsed with no trace.

Step 8. Make the batch path per-recipient. Claim recipients individually with ON CONFLICT DO NOTHING RETURNING, build the comma-separated mobile value from the returned set only, and parse invalidMobile on every success response.

Step 9. Make fallback conservative. Remove channel from the business key. Fall back only on definite channel-specific failure. An ambiguous outcome stops the chain rather than advancing it.

Step 10. Instrument and alarm. Emit a counter for suppressed duplicates, a gauge for the depth of the unknown backlog, and a counter for reconciliation outcomes split by resolution type. A rising suppressed-duplicate count means an upstream system started replaying. A growing unknown backlog means reconciliation is falling behind and you are one incident away from a mass re-send decision made under pressure.

Decision Matrix: Which Guard Belongs Where

Situation	Business key	Redis claim	DB unique index	Queue job ID	Rate limit	Reconciler
Single transactional SMS, low volume	Required	Optional	Required	Nice to have	No	Required
High-volume transactional, many workers	Required	Required	Required	Required	No	Required
OTP generate	Per session	Required	Required	Required	Required	Required, short SLA
Promotional campaign, one shot	Required	Optional	Required	Required	No	Required, long SLA
Batch of 500 recipients	Per recipient	Optional	Required	Per batch	No	Required
Cross-channel fallback	Channel-free	Required	Required	Required	No	Required per channel
Scheduled or recurring sends	Per occurrence	Optional	Required	Required	No	Required
Inbound webhook triggering a send	From upstream event ID	Required	Required	Required	Per source	Required
User-initiated resend button	Per session	Required	Required	Required	Required	Required
WhatsApp session message	Channel-free	Required	Required	Required	No	Manual, no confirmed DLR endpoint

The pattern across the rows: the database unique index is required everywhere, the reconciler is required everywhere, and everything else varies by volume and by whether a human can trigger the send.

Pre-Production Checklist

Twenty-two items. Every one of them corresponds to a duplicate incident someone has already had.

Business key is deterministic and contains no timestamp, UUID or random component.
Business key includes the template identifier, so different notifications about the same entity do not collide.
Business key excludes the channel, so fallback cannot double-notify.
A unique constraint exists on business_key in the database, not only in application code.
The claim happens before the HTTP call, in the same function, with no intervening await on an unrelated resource.
A unique-constraint violation is handled as a normal control-flow outcome, not an exception that fails the job.
Transport-level retries are disabled on the send path.
Transport-level retries are enabled on the delivery report path.
Connect timeouts and read timeouts are caught in separate branches.
Read timeouts, write timeouts, connection resets and 5xx responses all route to unknown.
No code path moves a record from unknown directly back into the send queue without reconciliation.
Queue visibility timeout or lock duration exceeds worst-case job runtime including in-job backoff.
Redis claim TTL exceeds worst-case end-to-end send duration.
A durable claim exists behind the Redis claim, so a cache flush cannot open the gate.
invalidMobile is parsed on every success response and those recipients are marked terminal.
Batch ambiguity marks every recipient in the batch unknown, not a guessed subset.
Permanent error codes are enumerated from the platform’s own error code list and terminate rather than retry.
Fallback advances only on definite, channel-specific failure.
Response parsing normalises transactionId versus messageId and quoted versus unquoted statusCode across channels.
Server-side rate limiting exists on any endpoint a user can trigger repeatedly, enforced against the MSISDN.
The reconciliation sweep runs on a schedule, is monitored, and has an alarm on backlog depth.
Message templates are worded as absolute statements rather than incremental deltas, so a duplicate handset receipt is harmless.

Ten Mistakes That Cause Duplicate Sends

1. Generating the idempotency key inside the retry loop. A fresh UUID per attempt is the same as no key. The key must be derived from the event, outside the loop.

2. Using the platform’s transactionId as the idempotency key. You only receive it on success, and the failure you are guarding against is the one where you receive nothing.

3. Assuming duplicatecheck suppresses repeated requests. It removes duplicate mobile numbers within a single request. It is documented that way and nothing broader is documented.

4. Catching all timeouts in one branch. except Timeout in Python and a single catch in JavaScript both collapse the safe case and the ambiguous case into one, and the handler inevitably retries both.

5. Leaving urllib3.Retry with allowed_methods including POST. Your code sees one call, the platform sees three, and nothing in your logs reveals it.

6. Enabling acks_late without an idempotency guard. This converts a worker crash from a lost message into a guaranteed duplicate. It is still the right setting; it just needs the guard.

7. Treating ShouldBeUnique or a BullMQ jobId as the whole answer. Both have the lifetime of the queue entry. Business identity outlives the queue.

8. Re-sending an entire batch after a timeout. You do not know which subset landed. Mark them all unknown and reconcile.

9. Falling back to the next channel on a timeout. Timeouts are the most common fallback trigger and the least valid one, because the first channel may have succeeded.

10. Writing state after the send instead of before it. If the process dies between the send and the write, the message is out and your records say it never happened. The next run sends it again. Claim first, always.

FAQs

What is message idempotency?

Message idempotency means that submitting the same logical message any number of times produces exactly one delivered message and one charge. It is achieved by giving each logical message a deterministic identity, claiming that identity in a store with a uniqueness guarantee before any API call is made, and refusing to send when the claim fails.

Does the SMS API have an idempotency key header?

No idempotency key header is documented for the Send SMS endpoint. Deduplication is the client’s responsibility, which is why the patterns in this article place the guard in your own database or cache rather than relying on a platform-side mechanism.

What does the duplicatecheck parameter actually do?

The documentation describes it as “Enable to remove duplicate mobile numbers. Default is true.” It removes repeated mobile numbers from the recipient list within a single request. It is not documented as suppressing repeated requests over time, so do not model it as a retry guard.

Is there a time window in which duplicatecheck suppresses repeats?

No such window is documented on the site. Treat the parameter as within-request recipient deduplication with no documented cross-request memory. If your design needs a time-windowed suppression behaviour, confirm the exact semantics with support before depending on it, because undocumented behaviour is not a contract.

If my request times out, was the SMS sent?

You cannot tell from the timeout. A connect timeout means nothing was submitted. A read timeout means the request was fully transmitted and may have been accepted, billed and delivered. Treat read timeouts as unknown and resolve them by querying the delivery report API, not by re-sending.

Am I charged for a message that timed out?

If the platform accepted and submitted it, yes. SMSGatewayCenter states that “Credits are non-refundable once SMS is successfully submitted to the operator” and that the per-SMS rate is deducted from your wallet while sending. Billing is tied to submission, not to your receipt of a response.

What should the idempotency key be made of?

A tenant identifier, an event type, an event identifier, the recipient MSISDN and the template identifier, concatenated and hashed. No timestamps, no UUIDs, no attempt counters. If the same business event produces two different keys, you have no guard.

Can I use the transactionId as the idempotency key?

No. It is issued by the platform in the success response, so it is unavailable in exactly the failure scenario you need protection for. Use it for delivery report correlation and billing reconciliation instead.

Does a batch send return one transactionId or one per recipient?

The documented success response carries a single top-level transactionId field. To correlate a specific recipient with a specific outcome, use the delivery report API, where each row carries mobileNo, msgId and uuId.

How do I detect that part of a batch was rejected?

Parse the invalidMobile field in the response. Numbers the platform could not accept appear there while the overall status may still read success. Code that checks only status will record rejected recipients as sent.

Does Celery’s acks_late cause duplicate messages?

By design, yes. It provides at-least-once delivery, which means a task interrupted mid-execution is redelivered and the send code runs again. That is the correct setting, because the alternative silently drops messages, but it must be paired with an idempotency guard or it becomes a duplicate generator.

Is Laravel’s ShouldBeUnique enough to prevent duplicate SMS?

No. It prevents a second copy of the job from being queued while the first is pending or running. It does not prevent a retried job instance from executing its send code twice, and it stops protecting once the job completes. Add a unique index on the business key in the database.

Should I fall back from WhatsApp to SMS when the WhatsApp request times out?

No. A timeout does not mean the WhatsApp message failed, and falling back on an ambiguous outcome is how cross-channel duplicates happen. Fall back only on definite, channel-specific failures identified by delivery error code.

Can I ever achieve exactly-once delivery?

Not as a delivery guarantee. Exactly-once delivery is impossible in a distributed system because of the two generals problem. What you can build is effectively-once processing: at-least-once transport plus an idempotent send path. That is what every production messaging system actually implements.

Why do customers sometimes receive a message twice even though my logs show one submission?

The operator’s SMSC can retransmit a message when the handset does not acknowledge receipt over the air. That happens below the API and cannot be prevented by any parameter. Track duplicate submissions from your own systems separately from duplicate handset receipts, and write templates that read sensibly if received twice.

Build it!

Build it once, correctly. Every pattern in this guide runs against the same live endpoints: SMSApi/send for SMS, WAApi/send for WhatsApp, SMSApi/reports/status for reconciliation. Create a free SMSGatewayCenter account to get your API key, import the SMS Postman collection, and test your duplicate handling against a real gateway before it matters. If you are wiring up an India-facing pipeline, our team can review your DLT template mapping and route configuration alongside it. Talk to support.