Gradient Descent gives us the opportunity to build any intelligent system where the output can be optimized by calculating the loss with respect to the system parameters and updating the parameters in the direction that minimized the loss. Chain Rule on the other hand, another mechanism that helps …

When we talked about Gradient Descent in part 3, we found it out to be an extremely meaningful optimization technique. Though gradient descent and its variations like mini batch and stochastic gradient descent act as dependable building blocks of AI optimization, they do have some limitations. One …

In part 5 we explored optimization strategies that ensure a solution to local optima problems and can navigate through flat saddle points. From the foundational approach of Gradient Descent we have moved to its refined version of Momentum based methods and techniques that involve Adaptive Learning …