<aside> 💡

相關頁面: AI相關工具非常非常的多，RAG開發工具會整理在RAG開發工具，LLM以及相關工具會整理在Large Language Models，Graph RAG開發工具會整理在Graph RAG，其他工具整理在Generative AI開發工具中。目前還有一些資料在Generative AI及RAG中，將逐步整理這些內容。

</aside>

介紹

<aside> 🚧

未來會將LLM相關連結移到此頁

</aside>

• LLM十大觀念
  • 【LLM 10大觀念-1】Scaling Law
    • Scaling Law 定義：我們可以用模型大小、Dataset大小、總計算量，來預測模型最終能力。
  • 【LLM 10大觀念-2】Good Data is All You Need
    • Data是幾乎所有公司在開發自己的LLM時都會遇到的問題。Data需要考量的面相又多又複雜，我自己最重視的就是兩點：Data Mixture跟Data Quality。
      • Data Mixture思考的是：我們要在Training data中加入哪些種類的Data？帶給LLM哪些能力？
      • Data Quality則要從微觀跟巨觀考量，圍觀考量每一個Data point是否可學、好學？巨觀則考量整的Dataset是否有某些糟糕的性質沒有被處理好？
  • 【LLM 10大觀念-3】快速建造自己個instruction tuning dataset
    • 如何finetune出一個公司內部解決特定任務的LLM。
• Where are facts stored in Large Language Models? (Friend Link)
• Understanding LLMs from scratch using middle school math
  • What makes large language models work so well?
    • Embeddings
    • Subword Tokenizers
    • Self Attention
    • Softmax
    • Residual connections
    • Layer Normalization
    • Dropout
    • Multi-head Attention
    • Positional encoding and embedding
  • The GPT architecture
  • The transformer architecture
• Decoding an LLM’s Thoughts: Logit Lens in Just 25 Lines of Code
• Coding the Brain of an LLM to See How it Thinks (Friend Link)
• LServe: system for faster long-sequence LLM serving via hybrid sparse attention
• The Language Model Landscape — Version 7
  • Zone 1 — Language Models Disruption
  • Zone 2 — General Use-Cases
  • Zone 3 — Specific Implementations
  • Zone 4 — Commercial Model Providers
  • Zone 5 — Model Diversification
  • Zone 6 — Foundation Tooling
  • Zone 7— End User UIs
• Language Model Categorisation
  • Understanding Model Size From Small to Extra-Large Models
    • Large Language Models
    • Medium to Small Language Models
    • Expanding Beyond Language with Multi-Modal Models
    • Action Models (also known as Large Action Models)
• LLM Architectures Explained
  • NLP Fundamentals (Part 1) (Friend Link)
  • Word Embeddings (Part 2) (Friend Link)
  • RNNs, LSTMs & GRUs (Part 3) (Friend Link)
  • LLM Architectures Explained: Encoder-Decoder Architecture (Part 4) (Friend Link)
  • LLM Architectures Explained: Attention Mechanism (Part 5) (Friend Link)
  • LLM Architectures Explained: Transformers (Part 6) (Friend Link)
  • LLM Architectures Explained: Coding a Transformer (Part 7) (Friend Link)
  • LLM Architectures Explained: BERT (Part 8) (Friend Link)
• Transformer
  • 何謂 Transformer 模型？
  • Transformer：自然語言處理的里程碑
  • Transformer Explainer: A visualization Tool To understand the Magic Behind Modern LLMs (Friend Link)
    • 可以透過Transformer Explainer來「看到」大型語言模型如何「算」出下一個字
    • Temperature
      • if you set the temperature to 0.1, the LLM will always generate the most likely next word which can be super useful for stuff like answering questions. However, if you set the temperature to 1, the LLM will be more likely to generate less likely next words, which could result in more creative text.
      • 注意，每個LLM的temperature不太一樣，有的中間值是1，有的是0.5，但是，基本上都是越大越有創意。
    • Embedding
    • Multi-head Self Attention
  • Transformers Demystified: The Brain Behind Modern AI
    • From Transformers to Today: Evolution of AI Models
      • Encoder-Only Models: BERT (Bidirectional Encoder Representations from Transformers)
      • Decoder-Only Models: GPT (Generative Pre-trained Transformer), Gemini and Claude are decoder-only models, widely known for their capabilities in generating relevant text.
      • Encoder-Decoder Models: T5 (Text-to-Text Transfer Transformer) and BART
  • Mechanistic Interpretability in Action: Understanding Induction Heads and QK Circuits in Transformers
  • Understanding FlashAttention-3: One of the Most Important Algortihms to Make Transformers Fast
  • Vision Transformers Completely Redefine How AI Perceives The Real World (Friend Link)
  • Explaining Transformers as Simple as Possible through a Small Language Model (Friend Link)
  • Explainable AI: Visualizing Attention in Transformers
    • BertViz
  • Demystifying the Transformer Model
  • From Costly Attention to FlashAttention: A Deep Dive into Transformer Efficiency (Friend Link)
• Andrej Karpathy’s Deep Dive into LLMs like ChatGPT YouTube video summarized
• Unmasking the Surprising Diversity of AI Hallucinations
• The Dream Machine: Decoding Why LLMs Hallucinate Reality (Friend Link)
  • HALoGEN: Benchmarking and Verifying Hallucinations in Generative Language Models
• Your LLM knows when it’s lying to you. (Friend Link)
  • the Statement Accuracy Prediction, based on Language Model Activations, is a method that uses the activations to predict the accuracy of a statement outputted by an LLM.
• Step-by-Step Exploration of Transformer Attention Mechanisms (Friend Link)
• Which AI Model is the Best in 2024?
  • ChatPlayground is a comprehensive AI platform that offers 16 powerful AI tools in one subscription. It provides industry-leading AI models, a prompt library for various use cases, real-time web search capabilities, image generation, history recall, multilingual support, and more. It is designed for developers, data scientists, students, researchers, content creators, writers, and AI enthusiasts.
• Advanced LLM Chain Orchestration: Building Complex AI Systems That Scale (Friend Link)