1 Transformer 各子模块作用

自然语言处理基础V4.0

• [ ] 第一章自然语言处理入门

第一章自然语言处理入门

◦ [1自然语言处理入门](<https://iamwsll.notion.site/1-35b1f7d9511f8101ba20ee9a31ffddfa>)

• [ ] 第二章文本预处理

第二章文本预处理

◦ [1 认识文本预处理](<https://iamwsll.notion.site/1-35b1f7d9511f819a8f45f46dbc7ef035>)
◦ [2 文本处理的基本方法](<https://iamwsll.notion.site/2-35b1f7d9511f814c9d5afe0648956318>)
◦ [3 文本张量表示方法](<https://iamwsll.notion.site/3-35b1f7d9511f8108a9cadd30b68e2407>)
◦ [4 文本数据分析](<https://iamwsll.notion.site/4-35b1f7d9511f8105817ef232ca5a48ef>)
◦ [5 文本特征处理](<https://iamwsll.notion.site/5-35b1f7d9511f817aa262d0e35a493b74>)
◦ [6 文本数据增强](<https://iamwsll.notion.site/6-35b1f7d9511f814a8695f0dd5e2da08d>)
◦ [7 jieba词性对照表](<https://iamwsll.notion.site/7-jieba-35b1f7d9511f8188bc81cf401c940ee4>)

• [ ] 第三章 RNN及其变体

第三章 RNN及其变体

◦ [1 认识RNN模型](<https://iamwsll.notion.site/1-RNN-35b1f7d9511f8176afadd381817849bb>)
◦ [2 传统RNN模型](<https://iamwsll.notion.site/2-RNN-35b1f7d9511f818ebc99d020cf177f44>)
◦ [3 LSTM模型](<https://iamwsll.notion.site/3-LSTM-35b1f7d9511f8156ad12fb1f2151e0b4>)
◦ [4 GRU模型](<https://iamwsll.notion.site/4-GRU-35b1f7d9511f817eaa32cf958d9b91e5>)
◦ [5 RNN案例-人名分类器](<https://iamwsll.notion.site/5-RNN-35b1f7d9511f8132bb25d6997ff9b669>)
◦ [6 注意力机制介绍1](<https://iamwsll.notion.site/6-1-35b1f7d9511f81d1b816f3fab4b356b7>)
◦ [7 注意力机制介绍2](<https://iamwsll.notion.site/7-2-35b1f7d9511f811ca94ae90e4e563f7a>)
◦ [8 RNN案例-seq2seq英译法](<https://iamwsll.notion.site/8-RNN-seq2seq-35b1f7d9511f81888a5cdeb70bc99f25>)

• [ ] 第四章 Transformer

第四章 Transformer

◦ [1 Transformer背景介绍](<https://iamwsll.notion.site/1-Transformer-35b1f7d9511f81ce8c31e2e40ad6cca4>)
◦ [2 认识Transformer架构](<https://iamwsll.notion.site/2-Transformer-35b1f7d9511f81c5bdbbefb453ea43a5>)
◦ [3 输入部分实现](<https://iamwsll.notion.site/3-35b1f7d9511f813ab6cee8626e1250d3>)
◦ [4 编码器部分实现](<https://iamwsll.notion.site/4-35b1f7d9511f815a8520f743f6264d07>)
◦ [5 解码器部分实现](<https://iamwsll.notion.site/5-35b1f7d9511f8152a648fa2677961845>)
◦ [6 输出部分实现](<https://iamwsll.notion.site/6-35b1f7d9511f81119ed2eaa8945760cc>)
◦ [7 模型构建](<https://iamwsll.notion.site/7-35b1f7d9511f8141b086eb702da088d3>)

• [ ] 第五章迁移学习

第五章迁移学习

◦ [1 fasttext工具介绍](<https://iamwsll.notion.site/1-fasttext-35b1f7d9511f81409bafd2beea057d2f>)
◦ [2 fasttext模型架构](<https://iamwsll.notion.site/2-fasttext-35b1f7d9511f81ad8df1ffd79b88c6b1>)
◦ [3 fasttext文本分类](<https://iamwsll.notion.site/3-fasttext-35b1f7d9511f81bebb09f97f4f7ee51e>)
◦ [4 训练词向量](<https://iamwsll.notion.site/4-35b1f7d9511f81709fecc9e8b947ea0e>)
◦ [5 词向量迁移](<https://iamwsll.notion.site/5-35b1f7d9511f813f8612eacdc205f4a9>)
◦ [6 迁移学习概念](<https://iamwsll.notion.site/6-35b1f7d9511f81fa852adad7469a2750>)
◦ [7 NLP中的常用预训练模型](<https://iamwsll.notion.site/7-NLP-35b1f7d9511f81058ad2e6c39eee088c>)
◦ [8 Transformers库使用](<https://iamwsll.notion.site/8-Transformers-35b1f7d9511f81c98baae890754fa25d>)
◦ [9 迁移学习实践](<https://iamwsll.notion.site/9-35b1f7d9511f8189b363dd7f5c7fc2a2>)
◦ [10 NLP中的标准数据集(拓展资料)](<https://iamwsll.notion.site/10-NLP-35b1f7d9511f8118905bc0e6750fa3a1>)

• [ ] 第六章 Bert系列模型

第六章 Bert系列模型

◦ [1 BERT模型介绍](<https://iamwsll.notion.site/1-BERT-35b1f7d9511f8101b99afb71943b1d09>)
◦ [2 BERT模型特点](<https://iamwsll.notion.site/2-BERT-35b1f7d9511f811391bbd006fe8c43a7>)
◦ [3 BERT系列模型介绍](<https://iamwsll.notion.site/3-BERT-35b1f7d9511f8113905debed8e320bd8>)
◦ [4 ELMo模型介绍](<https://iamwsll.notion.site/4-ELMo-35b1f7d9511f81cebb17e92f90ed53b0>)
◦ [5 GPT模型介绍](<https://iamwsll.notion.site/5-GPT-35b1f7d9511f8143b893f0f19aa19aab>)
◦ [6 BERT GPT ELMo模型的对比](<https://iamwsll.notion.site/6-BERT-GPT-ELMo-35b1f7d9511f810abd45f786920b6d22>)

• [x] 第七章 Transformer精选问答(拓展资料)

第七章 Transformer精选问答(拓展资料)

◦ [ ]

1 Transformer 各子模块作用

1 Transformer 各子模块作用目录

    ▪ [学习目标](about:blank#_1)
    ▪ [1 Encoder模块](about:blank#1-encoder)
        • [1.1 Encoder模块的结构和作用:](about:blank#11-encoder)
        • [1.2 关于Encoder Block](about:blank#12-encoder-block)
        • [1.3 多头自注意力层(self-attention)](about:blank#13-self-attention)
        • [1.4 Decoder模块](about:blank#14-decoder)
        • [1.5 Add & Norm模块](about:blank#15-add-norm)
        • [1.6 位置编码器Positional Encoding](about:blank#16-positional-encoding)
    ▪ [2 小结](about:blank#2)
◦ [2 Transformer Decoder模块](<https://iamwsll.notion.site/2-Transformer-Decoder-35b1f7d9511f81a78d5dca85421204c5>)
◦ [3 Self-attention机制详解](<https://iamwsll.notion.site/3-Self-attention-35b1f7d9511f813eb145c9d34e5e7747>)
◦ [4 Multi-head Attention详解](<https://iamwsll.notion.site/4-Multi-head-Attention-35b1f7d9511f81659e6bcb7a07a0b70d>)
◦ [5 Transformer优势](<https://iamwsll.notion.site/5-Transformer-35b1f7d9511f810f9fa7dbf613ce0466>)

logo

•

第一章自然语言处理入门

  ◦ 
  ◦ [1自然语言处理入门](<https://iamwsll.notion.site/1-35b1f7d9511f8101ba20ee9a31ffddfa>)

◦ ◦ 1自然语言处理入门

•

第二章文本预处理

  ◦ 
  ◦ [1 认识文本预处理](<https://iamwsll.notion.site/1-35b1f7d9511f819a8f45f46dbc7ef035>)
  ◦ 
  ◦ [2 文本处理的基本方法](<https://iamwsll.notion.site/2-35b1f7d9511f814c9d5afe0648956318>)
  ◦ 
  ◦ [3 文本张量表示方法](<https://iamwsll.notion.site/3-35b1f7d9511f8108a9cadd30b68e2407>)
  ◦ 
  ◦ [4 文本数据分析](<https://iamwsll.notion.site/4-35b1f7d9511f8105817ef232ca5a48ef>)
  ◦ 
  ◦ [5 文本特征处理](<https://iamwsll.notion.site/5-35b1f7d9511f817aa262d0e35a493b74>)
  ◦ 
  ◦ [6 文本数据增强](<https://iamwsll.notion.site/6-35b1f7d9511f814a8695f0dd5e2da08d>)
  ◦ 
  ◦ [7 jieba词性对照表](<https://iamwsll.notion.site/7-jieba-35b1f7d9511f8188bc81cf401c940ee4>)

•

第三章 RNN及其变体

  ◦ 
  ◦ [1 认识RNN模型](<https://iamwsll.notion.site/1-RNN-35b1f7d9511f8176afadd381817849bb>)
  ◦ 
  ◦ [2 传统RNN模型](<https://iamwsll.notion.site/2-RNN-35b1f7d9511f818ebc99d020cf177f44>)
  ◦ 
  ◦ [3 LSTM模型](<https://iamwsll.notion.site/3-LSTM-35b1f7d9511f8156ad12fb1f2151e0b4>)
  ◦ 
  ◦ [4 GRU模型](<https://iamwsll.notion.site/4-GRU-35b1f7d9511f817eaa32cf958d9b91e5>)
  ◦ 
  ◦ [5 RNN案例-人名分类器](<https://iamwsll.notion.site/5-RNN-35b1f7d9511f8132bb25d6997ff9b669>)
  ◦ 
  ◦ [6 注意力机制介绍1](<https://iamwsll.notion.site/6-1-35b1f7d9511f81d1b816f3fab4b356b7>)
  ◦ 
  ◦ [7 注意力机制介绍2](<https://iamwsll.notion.site/7-2-35b1f7d9511f811ca94ae90e4e563f7a>)
  ◦ 
  ◦ [8 RNN案例-seq2seq英译法](<https://iamwsll.notion.site/8-RNN-seq2seq-35b1f7d9511f81888a5cdeb70bc99f25>)