Claim Check Pattern

Claim Check Pattern — katta message payloadini queue yoki event orqali to‘liq uzatish o‘rniga, uni alohida storage’ga joylashtirib, message ichida faqat reference yoki “claim check” yuboradigan integration pattern.

Bu pattern message broker limitlari, network xarajati va duplicate payload nusxalarini kamaytiradi.

Muammo

Image processing taski 200 MB video faylni qayta ishlashi kerak.

Faylni queue message ichiga joylashtirish:

• broker limitidan oshishi;
• memory sarfi;
• sekin replication;
• retryda qayta uzatish;
• queue storage o‘sishi

muammolarini yaratadi.

Fayl object storage’ga yozilib, message’da faqat manzil beriladi.

Claim check

Claim check payloadni topish uchun reference.

U quyidagilarni saqlashi mumkin:

• object ID;
• storage bucket;
• version;
• checksum;
• size;
• content type;
• encryption metadata;
• expiration;
• tenant.

Reference taxmin qilinmaydigan va authorization bilan himoyalangan bo‘lishi kerak.

Oqim

Asosiy jarayon:

1. producer katta payloadni storage’ga yozadi;
2. storage muvaffaqiyatli ekanini tekshiradi;
3. reference bilan message yaratadi;
4. consumer message’ni oladi;
5. reference orqali payloadni yuklaydi;
6. processing bajaradi;
7. retention qoidasi bo‘yicha payload tozalanadi.

Storage

Payload quyidagilarda saqlanishi mumkin:

• object storage;
• file storage;
• blob database;
• distributed filesystem;
• temporary secure store.

Storage queue’dan mustaqil scale qiladi.

Katta binary uchun object storage ko‘pincha qulay.

Atomicity

Payload storage’ga yozildi, ammo message publish xato bo‘lsa orphan object qoladi.

Message publish bo‘ldi, ammo object to‘liq yozilmagan bo‘lsa consumer topolmaydi.

Jarayon state va cleanup bilan boshqariladi.

Avval temporary object, keyin finalize va publish ishlatilishi mumkin.

Checksum

Consumer payloadni olgach checksumni tekshiradi.

Bu:

• incomplete upload;
• corruption;
• noto‘g‘ri object;
• transfer xatosi

ni aniqlaydi.

Checksum message metadata’sida yoki trusted storage metadata’sida saqlanadi.

Security

Claim checkning o‘zi maxfiy access credential bo‘lishi mumkin.

Yondashuvlar:

• opaque object ID va service identity;
• qisqa muddatli signed URL;
• brokerdan kelgan tenant metadata;
• storage ACL;
• encryption.

Permanent public URL ishlatilmaydi.

Authorization

Consumer reference mavjudligini bilishi payloadni o‘qishga avtomatik ruxsat bermaydi.

Storage service consumer identity va tenantga qarab accessni tekshiradi.

Cross-tenant object ID bilan data leak bo‘lmasligi kerak.

Signed URL

Storage’ga vaqtinchalik kirish uchun signed URL berilishi mumkin.

U:

• expiration;
• operation;
• object;
• signature

bilan cheklangan.

URL log, error yoki analytics’da oshkor qilinmasligi kerak.

Encryption

Payload transportda va storage’da shifrlanadi.

Juda maxfiy data uchun application-level encryption ishlatilishi mumkin.

Key metadata claim checkda saqlansa u ham himoyalanadi.

Retention

Payload message’dan uzoqroq yoki qisqaroq saqlanishi mumkin.

Consumer retry qilishi uchun object yetarli vaqt mavjud bo‘lishi kerak.

Juda erta delete retryni buzadi.

Juda uzun retention storage va privacy xarajatini oshiradi.

Cleanup

Orphan va processed objectlarni tozalash jarayoni kerak.

Usullar:

• expiration lifecycle;
• status table;
• consumer completion event;
• periodic reconciliation;
• reference count.

Cleanup message ack bilan bir transactionda bo‘lmasligi mumkin.

Retry

Consumer payloadni yuklab olgach yoki processing o‘rtasida xato berishi mumkin.

Message qayta kelganda object hali mavjud bo‘lishi kerak.

Processing idempotent bo‘ladi.

Katta faylni har retryda boshidan yuklash qimmat bo‘lsa local cache yoki resumable read ishlatilishi mumkin.

Version

Object mutable bo‘lsa reference qaysi versiyani bildirishi kerak.

Aks holda consumer retryda boshqa payload olishi mumkin.

Immutable object ID yoki versioned key reproducibility beradi.

Message hajmi

Claim check message’i kichik bo‘lsa ham metadata juda ko‘p bo‘lib ketmasligi kerak.

Broker uchun zarur fieldlar:

• reference;
• operation;
• correlation;
• minimal validation metadata.

Katta business documentning o‘zi yana messagega qo‘shilmaydi.

Payload locality

Consumer va storage boshqa regionda bo‘lsa egress va latency oshadi.

Object replica yoki regional placement ishlatilishi mumkin.

Data residency talabi payload qayerda saqlanishini cheklaydi.

Streaming

Consumer butun faylni memory’ga yuklamasdan stream qilib qayta ishlashi mumkin.

Bu katta payload uchun muhim.

Checksum incremental hisoblanadi.

Temporary disk va maximum size limit bilan himoyalanadi.

Observability

Trace message IDni object ID va processing job bilan bog‘laydi.

Kuzatiladi:

• upload time;
• queue wait;
• download time;
• byte;
• checksum xatosi;
• object not found;
• cleanup;
• orphan count.

Reference logda ko‘rsatilsa access token redaction qilinadi.

Claim Check va Queue-Based Load Leveling

Queue-Based Load Leveling ish tezligini silliqlaydi.

Claim Check esa queue’dagi payload hajmini kamaytiradi.

Katta media processing pipeline’da ikkalasi birga ishlatiladi.

Upload session

Katta payload multipart yoki resumable upload bilan yozilishi mumkin.

Producer barcha qismlar va checksum muvaffaqiyatli bo‘lgach objectni finalized deb belgilaydi.

Consumer faqat finalized object reference’ini qabul qiladi.

Malware tekshiruvi

Foydalanuvchi yuklagan fayl consumerga yetkazilishidan oldin antivirus yoki content validationdan o‘tishi mumkin.

Status:

• uploaded;
• scanning;
• clean;
• rejected

ko‘rinishida saqlanadi.

Clean bo‘lmagan object processing queue’siga yuborilmaydi.

Reference rotation

Signed URL expiration tugasa retry qiluvchi consumer yangi URL olishi mumkin.

Permanent message ichida qisqa muddatli URL o‘rniga stable object ID saqlash ko‘proq moslashuvchan.

Bog‘liq tushunchalar

Message queue, Object storage, Blob, Reference, Signed URL, Checksum, Payload, Queue-Based Load Leveling, Data retention, Idempotency