When we talked about Gradient Descent in part 3, we found it out to be an extremely meaningful optimization technique. Though gradient descent and its variations like mini batch and stochastic gradient descent act as dependable building blocks of AI optimization, they do have some limitations. One …