Machine Learning

[video] 🧊 Los servidores de IA consumen cantidades brutales de agua para no sobrecalentarse. Una sola instalación como la del video usa 30 millones de litros de agua al mes solo para enfriar los servidores. Mientras tanto, en muchas partes del mundo falta agua potable… ¿Vale la pena el

[image] day 4/60 of summer break - almost completed the attention chapter - wrote a casual attention mask - learnt about multi head attention - extended single head attention to multi head attention once am done with this, i'll spend some more time into revising all this

[image] day 9/60 of summer break - wanted to learn about RAG so started reading this paper - read about categories of rag - pausing the book for now, done with what i needed from the book - will start working on embeddings and cosine similarity after i complete the first 3 sections

[image] day 3/60 of summer break - started chapter of the book - started reading about attention mechanisms - wrote a small attention mechanism without trainable weights and then another with trainable weights i have completed half of the chapter and the other half i'll complete today

[image] > revising all the basic LLM concepts by rebuilding nanogpt today will be a fun ride

[image] day 5/60 of summer break - pulled an allnighter - completely understood masked attention, casual attention and multi head attention - started with implementing a gpt model - will try to cover chapter 4 today rerevising was really worth it, watched rasbt's video and iterated a

[image] day 1/60 of summer break - completed chapter 1 of building a large language model from scratch - started chapter 2, completed 25% of it - morning workout + 45 min cardio the book has a lot of depth and i want to cover it whole. i dont wanna rush it so i'll try to give it 5-6

[image] day 11/60 of summer break > continued learning about building rag pipelines > read about document loaders > read about types of chunking > why semantic and late chunking are used if you wanna read more about late chunking use this :