I was working on a personal project last month, trying to identify different types of fruits from images I’d taken in my garden. The challenge wasn’t just building a model—it was creating an entire system that could go from raw images to a working application. This experience made me realize how many developers struggle with the complete pipeline, not just the modeling part. Today, I’ll walk you through building a custom image classification system using PyTorch, covering every step from data preparation to deployment.

Creating a custom dataset is where most projects begin. Have you ever faced the frustration of organizing messy image data? I certainly have. PyTorch’s Dataset class makes this manageable. Here’s a simple way to structure your data loading:

class FruitDataset(Dataset):
    def __init__(self, root_dir, transform=None):
        self.root_dir = Path(root_dir)
        self.transform = transform
        self.classes = sorted([d.name for d in self.root_dir.iterdir() if d.is_dir()])
        self.samples = self._load_samples()
    
    def _load_samples(self):
        samples = []
        for class_name in self.classes:
            class_dir = self.root_dir / class_name
            for img_path in class_dir.glob('*'):
                if img_path.suffix.lower() in ['.jpg', '.jpeg', '.png']:
                    samples.append((img_path, self.classes.index(class_name)))
        return samples
    
    def __getitem__(self, idx):
        img_path, label = self.samples[idx]
        image = Image.open(img_path).convert('RGB')
        if self.transform:
            image = self.transform(image)
        return image, label

Data augmentation transforms your training process. Why does rotating an image or changing its brightness help? It teaches the model to recognize objects from various angles and lighting conditions. I use these transforms in my projects:

train_transforms = transforms.Compose([
    transforms.RandomHorizontalFlip(p=0.5),
    transforms.RandomRotation(degrees=15),
    transforms.ColorJitter(brightness=0.2, contrast=0.2),
    transforms.ToTensor(),
    transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])
])

Building the model architecture comes next. Did you know that starting with a pre-trained model can save weeks of training time? I often use ResNet as a base and fine-tune it for specific tasks. Here’s how to modify the final layer:

model = models.resnet50(pretrained=True)
num_features = model.fc.in_features
model.fc = nn.Linear(num_features, num_classes)  # num_classes for your dataset
model = model.to(device)

Training the model involves more than just running epochs. How do you know if your learning rate is too high or too low? I implement learning rate scheduling to adjust it dynamically:

optimizer = optim.Adam(model.parameters(), lr=0.001)
scheduler = optim.lr_scheduler.StepLR(optimizer, step_size=7, gamma=0.1)

Mixed precision training can speed up the process on compatible hardware. Have you tried reducing memory usage without sacrificing accuracy? This code snippet helps:

from torch.cuda.amp import autocast, GradScaler
scaler = GradScaler()
with autocast():
    outputs = model(inputs)
    loss = criterion(outputs, labels)
scaler.scale(loss).backward()
scaler.step(optimizer)
scaler.update()

Evaluating your model goes beyond accuracy. What parts of the image is your model focusing on? I use Grad-CAM to visualize attention maps:

from pytorch_grad_cam import GradCAM
cam = GradCAM(model=model, target_layers=[model.layer4[-1]])
grayscale_cam = cam(input_tensor=input_tensor)

Deploying the model is the final step. Wouldn’t it be great to have your model serving predictions via an API? TorchServe makes this straightforward. First, package your model:

torch-model-archiver --model-name fruit_classifier --version 1.0 --serialized-file model.pth --handler image_classifier

Then, start the server and your model is ready for production inference. This entire process transforms your code into a practical application that others can use.

Building this pipeline taught me the importance of each component working together. From handling data quirks to optimizing inference speed, every detail matters. I encourage you to try implementing these steps in your own projects. If you found this guide helpful, please share it with others who might benefit. I’d love to hear about your experiences—feel free to leave a comment below with your questions or success stories!