Semisupervised — oz miqdordagi labeled data va katta miqdordagi unlabeled data’ni birgalikda ishlatadigan machine-learning yondashuvi. To‘liq nomi Semi-Supervised Learning. U label yaratish qimmat, ammo raw data ko‘p bo‘lgan vision, speech, text, tibbiyot va sanoat vazifalarida qo‘llanadi.
Maqsad unlabeled samplelardan data distributioni haqida signal olib, faqat kichik labeled dataset bilan o‘qitilgan modeldan yaxshiroq natija olishdir.
Labeled va unlabeled to‘plam
Dataset ikki qism:
L = {(x, y)}
U = {x}
Labeled qismda target mavjud.
Unlabeled qismda faqat input.
Ikki to‘plam bir xil yoki yaqin production distributiondan kelishi muhim.
Asosiy taxmin
Semisupervised usullar ko‘pincha quyidagi taxminlarga tayanadi:
- yaqin samplelar bir xil labelga ega;
- data low-density boundary bilan ajraladi;
- data past o‘lchamli manifold atrofida joylashgan.
Agar bu taxminlar noto‘g‘ri bo‘lsa unlabeled data sifatni yomonlashtirishi mumkin.
Pseudo-labeling
Avval labeled data’da model o‘qitiladi.
U unlabeled samplelarga prediction beradi.
Yuqori confidence predictionlar vaqtinchalik label sifatida trainingga qo‘shiladi.
Jarayon bir yoki bir necha marta takrorlanadi.
Confidence threshold
Faqat confidence thresholddan yuqori pseudo-labellar olinadi.
Threshold katta bo‘lsa sample kam, lekin tozaroq.
Past bo‘lsa ko‘p, ammo noisy label ko‘payadi.
Har class uchun alohida threshold imbalance’ni boshqarishi mumkin.
Confirmation bias
Model boshida xato prediction beradi.
Pseudo-label sifatida shu xatoda qayta train qilinsa model uni yanada kuchaytiradi.
Bu confirmation bias.
Teacher model, augmentation, balancing va human review ta’sirni kamaytiradi.
Consistency regularization
Bir samplega kichik transform qo‘llanganda model predictioni o‘zgarmasligi kerak.
Masalan, rasmning yengil crop yoki rang o‘zgarishi.
Model original va augmented view outputlarini yaqinlashtiradi.
Transform label semantikasini saqlashi shart.
Weak va strong augmentation
Weak augmentation pseudo-label yaratadi.
Strong augmentation shu label bilan train qilinadi.
Visionda crop, color va geometric transform ishlatilishi mumkin.
Textda ma’noni saqlovchi strong augmentation yaratish qiyinroq.
Teacher-student
Teacher unlabeled data uchun target beradi.
Student shu target va labeled data’da o‘rganadi.
Teacher:
- fixed pretrained model;
- studentning oldingi checkpointi;
- exponential moving average
bo‘lishi mumkin.
EMA teacher studentdan barqarorroq prediction berishi ehtimoli bor.
Mean Teacher
Teacher weightlari student weightlarining exponential moving average’si.
Teacher gradient olmaydi.
Student turli perturbationlarda teacher predictioniga moslashadi.
Bu temporal ensemblega yaqin barqaror signal beradi.
Self-training
Model o‘z predictionlaridan yangi training data yaratadi.
Pseudo-labeling self-trainingning keng tarqalgan ko‘rinishi.
Bir necha iterationda model unlabeled poolning ko‘proq qismini qamrab oladi.
Quality nazoratisiz xato yig‘iladi.
Graph-based usul
Samplelar similarity graphda node sifatida ifodalanadi.
Label known nodelardan qo‘shnilarga tarqaladi.
Bu label propagation.
Graph qurilishi va similarity metric natijani belgilaydi.
Katta datasetda graph hisoblash qimmat.
Entropy minimization
Model unlabeled samplelarda aniqroq, past entropy prediction chiqarishga undaladi.
Bu decision boundaryni data zich bo‘lmagan hududga surishi mumkin.
Noto‘g‘ri confidence model xatolarini kuchaytirishi mumkin.
Class imbalance
Unlabeled poolda class distribution noma’lum.
Model majority classga yuqori confidence berib, pseudo-label datasetni yanada noteng qiladi.
Distribution alignment, class-aware threshold va balanced sampling ishlatiladi.
Out-of-distribution data
Unlabeled dataset ichida training classlariga tegishli bo‘lmagan samplelar bo‘lishi mumkin.
Model ularni majburan ma’lum classga pseudo-label qiladi.
OOD detection yoki source filtering zarur.
Validation
Validation va test setlar haqiqiy, yuqori sifatli labelga ega bo‘lishi kerak.
Unlabeled data natijani baholash uchun yetarli emas.
Semisupervised model faqat supervised baseline bilan solishtiriladi.
Labeled sample soni bir xil saqlanadi.
Active learning bilan birga
Model noaniq yoki muhim samplelarni inson annotationiga yuboradi.
Qolgan yuqori confidence samplelar pseudo-label oladi.
Bu human labeling budgetidan samarali foydalanadi.
Active va semisupervised loop bir-birini to‘ldiradi.
Transfer learning bilan birga
Pretrained model avval umumiy representation beradi.
Kichik labeled dataset bilan fine-tuning va unlabeled data bilan consistency training qilinadi.
Bu ayniqsa domain-specific kichik datasetda foydali.
Evaluation metric
Task metricdan tashqari:
- pseudo-label accuracy;
- coverage;
- confidence calibration;
- per-class count;
- OOD rate;
- labeled efficiency
kuzatiladi.
Faqat unlabeled lossning pasayishi sifatni ko‘rsatmaydi.
Unlabeled ratio
Labeled va unlabeled loss bir xil weightda bo‘lishi shart emas. Unlabeled data juda ko‘p bo‘lsa uning noisy signali supervised signalni bosib ketishi mumkin. Ramp-up schedule training boshida supervised modelni barqarorlashtirib, keyin consistency weightini oshiradi.
Distribution mismatch
Unlabeled pool boshqa region, device yoki davrdan kelishi mumkin. Model yangi distributionni foydali o‘rganishi yoki known class boundary’sini buzishi ehtimoli bor. Source metadata va domain shift alohida baholanadi.
FixMatchga oid model
Keng yondashuvlardan biri weak augmentationdan yuqori confidence pseudo-label olib, strong augmentationga target sifatida qo‘llaydi. Threshold va augmentation taskga mos bo‘ladi. Bu umumiy prinsip, implementationlar farq qiladi.
Label budget
Semisupervised natija bir xil annotation budgetidagi supervised baseline bilan solishtiriladi. Maqsad faqat accuracy emas, har bir labeled sampledan ko‘proq foyda olishdir.
Bog‘liq tushunchalar
Semi-Supervised Learning, Pseudo-labeling, Consistency regularization, Teacher-student model, Mean Teacher, Label propagation, Active learning, Unlabeled data, Supervised learning, Self-training