OpenAI, Week 3-4 // Implementing ResNet in PyTorch
Last week, I studied CNNs (mostly through Andrej Karpathy's CS231n lectures and notes). This week, I implemented ResNet end-to-end in PyTorch and trained it on the CIFAR-10 dataset. I was initially a bit confused, because the ResNet paper describes two slightly different sets of architectures: the main one optimized for ImageNet, and a narrower one optimized for CIFAR-10. I ended up writing my code such that it's flexible enough to implement both versions, depending on how it's called. While I found many helpful guides and discussions on ResNet online, none quite laid out the details of these architectures in a "cheat sheet" way. In case it's useful to anyone else, I'm posting my handwritten notes. Below is a summary of the architecture of ResNet34. I use the notation where $K$ is the number of filters, $F$ is the size of the filter, $S$ is the stride, and $P$ is the padding. And here is an explicit work-through of the dimensionality of this problem:...