Things I’ve built outside of my day job. For a complete list, please browse my GitHub.

GPT-2 from Scratch in PyTorch

A from-scratch implementation of GPT-2 in PyTorch Lightning, pretrained on FineWeb-Edu. Implements both the full 124M-parameter model architecture and a smaller 30M-parameter model. Includes evaluation scripts on HellaSwag.

  • LLMs
  • Pretraining
[Code]

Analyzing Localization Abilities of Vision Transformers

Extends Grad-CAM beyond CNNs to vision transformers and to the object detection domain, the first work to do so to our knowledge. Compares where ResNet, DeiT, YOLOv3, and DETR attend when classifying and detecting objects.

  • Vision Transformers
  • Interpretability
[Paper]

Unhinged: Settle It

An iOS app (currently on the App Store) for friend groups who settle arguments with data: create a prediction about anything and let everyone weigh in with their pick.

  • iOS
  • React Native
  • Supabase
[App Store]

Low-Rank Adaptation (LoRA)

A from-scratch implementation of LoRA fine-tuning, with notebooks walking through the math and an IMDB classification example.

  • LLM Fine-Tuning
  • Transformers
[Code]

PyTorch Experimental Framework

Reusable PyTorch templates for autoencoders, VAEs, GANs, transfer learning, and custom CNNs. Covers various optimizers, learning-rate schedulers, and hyperparameter sweeps tracked in TensorBoard.

  • PyTorch
  • TensorBoard
  • Hyperparameter Tuning
[Code]