Videos de llm training
Videos etiquetados con "llm training"
llm training 2 videos
GRPO Fine-Tuning with Practical | DeepSeekMath, PPO vs GRPO, Hugging Face & Unsloth
Learn GRPO (Group Relative Policy Optimization) from scratch and fine-tune an LLM using Hugging Face TRL and Unsloth. In this video, we understand how GRPO works, why it was used in DeepSeekMath, how it differs from PPO, and how language models learn from reward signals. We will also implement GRPO fine-tuning practically using Hugging Face and Unsloth. Topics covered in this video: ✅ What is GRPO? ✅ GRPO full form and intuition ✅ Quick background of DeepSeekMath ✅ DeepSeekMath training pipeline ✅ PPO vs GRPO ✅ Problems with the PPO approach ✅ Why GRPO does not require a critic/value model ✅ Group-based reward comparison in GRPO ✅ GRPO step-by-step with a simple example ✅ Reward signal vs reward model ✅ Rule-based and verifiable rewards ✅ Correctness, helpfulness and clarity reward functions ✅ LoRA-based GRPO fine-tuning ✅ GRPO practical using Hugging Face TRL ✅ GRPO practical using Unsloth ✅ Loading and testing the fine-tuned model GRPO stands for Group Relative Policy Optimization. Instead of using a separate value or critic model, GRPO generates multiple answers for the same prompt, compares their rewards within the group and improves the policy using relative performance. This video is useful for machine learning engineers, Generative AI developers, data scientists and anyone learning LLM fine-tuning, reinforcement learning, RLHF, PPO, GRPO, DeepSeek and post-training techniques. Subscribe for more videos on Generative AI, LLM fine-tuning, RAG, Agentic AI, RLHF, PPO and GRPO. Code: https://github.com/sunnysavita10/Complete-LLM-Finetuning/tree/main/LLM%20Fine-Tuning-27-GRPO #GRPO #LLMFineTuning #DeepSeek #HuggingFace #Unsloth 📌 Subscribe for more videos on: LLM Fine-Tuning, RLHF, Quantization, Hugging Face, LangChain, Agentic AI, RAG, AI Systems, and Production-Grade AI Projects. #RLHF #PreferenceAlignment #LLM #PPO #ReinforcementLearning #DPO #ORPO #Qlearning #DQN #LLMFineTuning #GenerativeAI #MachineLearning #SunnySavita #AgenticAI #LangChain #ArtificialIntelligence 📌 Keywords Covered: #MultimodalLLM #VisionLanguageModel #MultimodalFineTuning #LLMFineTuning #Unsloth #LLaVA #QwenVL #Pixtral #LlamaVision #LoRA #QLoRA #VisionEncoder #ProjectionLayer #HuggingFace #Transformers #GenerativeAI #AIForDevelopers #CustomDataset #ImageToText #AITraining #SunnySavita #SemanticSearch #RAG Multimodel RAG Playlist: https://www.youtube.com/watch?v=7CXJWnHI05w&list=PLQxDHpeGU14D6dm0rmAXhdLeLYlX2zk7p&pp=gAQBiAQB RAG detailed playlist: https://www.youtube.com/watch?v=wTVTkOb3SZc&list=PLQxDHpeGU14Blorx3Ps1eZJ4XvKET1_vx&pp=gAQBiAQB GenAI Foundation Playlist: https://www.youtube.com/watch?v=ajWheP8ZD70&list=PLQxDHpeGU14D7NiPgqxC9qhKkx4jMQcDk&pp=gAQBiAQB Connect with me on social media LinkedIn: https://www.linkedin.com/in/sunny-savita/ One-to-One Call: https://topmate.io/sunny_savita10 GitHub: https://github.com/sunnysavita10
Muon Explained: Adam's First Real Challenger
AdamW has been the default optimizer for training large neural networks for nearly a decade. But a new optimizer called Muon may be its first serious challenger. In this video, we visually explain how Muon optimizer works, why it is different from Adam and AdamW, and why researchers are paying attention to it for large-scale LLM training. Instead of treating every weight independently, Muon looks at weight matrices geometrically. It orthogonalizes the momentum update, reshapes the singular value spectrum, and pushes training updates across more useful directions. We’ll break down the core idea behind momentum orthogonalization, Newton-Schulz iteration, polar factors, and why Muon can be more compute-efficient than AdamW. We’ll also explain the catch: why vanilla Muon can destabilize attention layers at frontier scale, and how QK-Clip turns it into MuonClip, making it more stable for large language model training. Topics covered: Why AdamW became the default optimizer Adam’s blind spot with matrix weights Singular values, SVD, and matrix geometry How Muon orthogonalizes momentum updates Newton-Schulz iteration explained visually Why Muon can reduce training compute Why attention logits can explode with Muon QK-Clip and MuonClip explained Why Muon matters for future LLM training If you’re interested in LLM training, optimizers, transformer architecture, AdamW, Muon, Newton-Schulz, scaling laws, and frontier AI training, this video gives you a visual explanation of one of the most interesting optimizer ideas in modern deep learning. Muon optimizer Muon explained Muon optimizer explained AdamW vs Muon Adam vs Muon Adam optimizer AdamW optimizer LLM optimizer LLM training optimizer deep learning optimizer neural network optimizer AI optimizer transformer optimizer training large language models LLM training large language model training how LLMs are trained optimizer explained AdamW explained MuonClip MuonClip explained QK-Clip QK Clip explained attention logits attention instability frontier LLM training Kimi K2 Muon Moonlight Muon Moonshot AI Muon Newton Schulz iteration Newton-Schulz iteration momentum orthogonalization orthogonalized momentum polar factor singular value decomposition SVD explained singular values matrix geometry matrix weights fast LLM training compute efficient training training compute scaling laws LLM scaling laws Pareto frontier AI Megatron Core Muon NVIDIA Megatron Core transformer training deep learning explained machine learning explained AI research explained large language models explained modern AI training future of LLMs Adam’s blind spot Muon vs AdamW optimizer for transformers training at scale LLM training stability loss spikes query key clipping QK norm multi head latent attention MLA attention frontier AI models AI training explained