CS7643l Quizl 4l (Latestl 2025/l 2026l Update)l Questionsl &l Answers|l Gradel A|l 100%l Correctl (Verifiedl Solutions)
Q:l Languagel Modell Goal
Answer:
-l estimatel thel probabilityl ofl sequencesl ofl words -l p(s)l =l p(w1,l w2,l ...,l wn)
Q:l Maskedl Languagel Modeling
Answer:
-l pre-trainingl taskl -l anl auxiliaryl taskl differentl froml thel finall taskl we'rel reallyl interestedl in,l butl whichl canl helpl usl achievel betterl performancel findingl goodl initiall parametersl forl thel model
-l Byl pre-trainingl onl maskedl languagel modelingl beforel trainingl onl ourl finall task,l itl isl usuallyl possiblel tol obtainl higherl performancel thanl byl simplyl trainingl onl thel finall task
Q:l Knowledgel Distillationl tol Reducel Modell Sizes
Answer:
-l Havel fullyl parameterizedl teacherl model -l Havel al muchl smallerl studentl model -l Studentl modell attemptsl tol minimizel predictionl errorl andl distancel tol teacherl modell simultaneously
L(dist)l =l CEl b/wl studentl andl teacherl predictions 1 / 2
L(student)l =l CEl b/wl predictedl outputl andl actual Ll =l alphal *l L(dist)l +l betal *l L(student)
Advantages:
-l mayl workl welll b/cl ofl softl predictionsl ofl teacherl model -l ifl wel don'tl havel enoughl labeledl textl wel canl stilll trainl studentl modell tol alignl predictions
Q:l Collobertl andl Westonl Vectorl Idea
Answer:
al wordl andl itsl contextl isl al positivel trainingl sample;l al randoml wordl inl thatl samplel contextl givesl al negativel trainingl sample
Q:l Word2vecl Overview
Answer:
Word2vecl -l al frameworkl forl learningl wordl vector
Idea:
-l wel havel al largel corpusl ofl text -l everyl wordl in.l fixedl vocabularyl representedl byl al vector -l Gol throughl eachl positionl tl inl thel text,l whichl hasl al centerl wordl cl andl contextl wordsl o -l Usel thel similarityl ofl thel wordl vectorsl forl cl andl ol tol calculatel thel probabilityl ofl ol givenl cl (orl vicel versa) -l Keepl adjustingl thel wordl vectorsl tol maximizel thisl probability
Q:l Word2vecl Variants
- / 2