This page is where I share useful resources in software engineering, machine learning and so on.

Pytorch Implementation

Annotated Deep Learning Paper Implementations: 59 Implementations/tutorials of deep learning papers with side-by-side notes; including transformers (original, xl, switch, feedback, vit, …), optimizers (adam, adabelief, …), gans(cyclegan, stylegan2, …), reinforcement learning (ppo, dqn), capsnet, distillation, etc.

Pytorch Attention Implementations: Pytorch implementation of various Attention Mechanisms, MLP, Re-parameter, Convolution, which is helpful to further understand papers.

End-to-End Object Detection with Transformers: The official implementation of DETR (DEtection TRansformer) of Facebook Research. The paper is published in Computer Vision–ECCV 2020: 16th European Conference.

Pytorch Vision Transformer of Phil Wang: Implementation of Vision Transformer, a simple way to achieve SOTA in vision classification with only a single transformer encoder, in Pytorch. Many versions of the ViTs are implemented in this repo.

Convolutional Xformers for Vision: Implementation of Convolutional Xformers for Vision, a more efficient way to implement attention for Vision Transformer.

Pytorch Image Models of Huggingface: PyTorch image models, scripts, pretrained weights – ResNet, ResNeXT, EfficientNet, EfficientNetV2, NFNet, Vision Transformer, MixNet, MobileNet-V3/V2, RegNet, DPN, CSPNet, and more. This is the repository of timm.

Neural Networks for NLP Code Examples: This is a repository of code examples for the 2017 edition of CMU CS 11-747 Neural Networks for NLP. This is a course of Carnegie Mellon University.

Treasure of Transformers: Awesome Treasure of Transformers Models for Natural Language processing contains papers, videos, blogs, official repo along with colab Notebooks.

Transformer Tutorials: Pytorch Transformer implementations in Colab Notebooks.

The Annotated Transformer: The Transformer has been on a lot of people’s minds over the last year five years. This post presents an annotated version of the paper in the form of a line-by-line implementation. Code is available here.

Git/Github

A Visual Git Reference: This visualization of git is really easy to understand.

Git Branching -Rebasing: Working with branch in Git is a bit confusing. Here is an illustration of Git Rebase for those who forget how this command works (it’s me).

Markdown Cheatsheet: A useful revision of Markdown syntax. I use it to create pages in this website.

Books and courses

Minimum Viable Study Plan for Machine Learning Interviews: A good resource for those planning to take a machine learning interview.

Designing Machine Learning Systems (Chip Huyen 2022): Summaries and resources for Designing Machine Learning Systems book (Chip Huyen, O’Reilly 2022). Chip is a writer, a lecturer at Stanford University, and a computer scientist.

Academic word list: Academic Word List Coxhead (2000) would be usefule if you are writing papers, documents, etc.

Bytepawn: Tech blog of Marton Trencseni.