Posts tagged with

ํŠธ๋žœ์Šคํฌ๋จธ ๋ชจ๋ธ ๊ตฌ์กฐ ๋ถ„์„

ํŠธ๋žœ์Šคํฌ๋จธ(Transformer) ๋ชจ๋ธ์€ 2017๋…„ 'Attention is All You Need'๋ผ๋Š” ๊ตฌ๊ธ€ ์—ฐ๊ตฌํŒ€์˜ ๋…ผ๋ฌธ์—์„œ ์†Œ๊ฐœ๋œ ์ธ์ฝ”๋”(Encoder)-๋””์ฝ”๋”(Decoder)๊ตฌ์กฐ๋ฅผ ๊ฐ€์ง„ ๋ชจ๋ธ์ž…๋‹ˆ๋‹ค. ํŠธ๋žœ์Šคํฌ๋จธ๋Š” ํ˜„๋Œ€ ๋งŽ์€ LLM(Large-Language-Model)์˜ ์–ด๋จธ๋‹ˆ๊ฒฉ ๋ชจ๋ธ๋กœ, ๊ทธ ๊ตฌ์กฐ๋ฅผ ์ดํ•ดํ•˜๋Š” ๊ฒƒ์€ ์ค‘์š”ํ•ฉ๋‹ˆ๋‹ค. ๋”ฐ๋ผ์„œ ์ด๋ฒˆ ๊ธ€์—์„œ๋Š” ํŠธ๋žœ์Šคํฌ๋จธ ๋ชจ๋ธ์˜ ๊ตฌ์กฐ์— ๋Œ€ํ•ด์„œ ๋ถ„์„ํ•˜์˜€์Šต๋‹ˆ๋‹ค.

Attention ๊ทธ๋ฆฌ๊ณ  Self-Attention. ์™œ ์ด๊ฒƒ๋“ค์ด ์ค‘์š”ํ•œ๊ฐ€?

Got me looking for attention