DermAI — Technical documentation

Origin and motivation

DermAI started as a capstone project for the Systems Engineering degree at ORT Uruguay. The goal was to apply computer vision to a real health problem: a pre-diagnosis dermatology assistant that lets anyone photograph a skin lesion with a smartphone and receive automated analysis as a triage tool before seeing a specialist.

The problem is concrete: dermatologist access is not always immediate, and many people cannot tell when a lesion needs urgent attention. DermAI acts as an informed first filter, with melanoma detection prioritized as the highest clinical-risk case.

---

Dataset and problem challenges

The model was trained on HAM10000 (Human Against Machine with 10000 training images): 10,015 dermatoscopic images across 7 lesion classes:

Two technical challenges shaped most design decisions:

1. Severe class imbalance — nv is ~67% of the dataset. A naive model can predict the majority class every time and reach ~67% global accuracy without learning melanoma.
2. Duplicate leakage — HAM10000 includes the same lesion photographed multiple times under different image_ids. Splitting by image can put the same lesion in train and test, inflating reported accuracy.

---

Model architecture

ResNet-50 with transfer learning from ImageNet weights. Low-level texture and edge features transfer well to dermatoscopy, reducing data needed to converge.

The final classification head was replaced with:

model.fc = nn.Sequential(
    nn.Dropout(p=0.5),
    nn.Linear(2048, 7)
)

Dropout(0.5) is the main regularizer against overfitting, identified early in development.

---

Training pipeline

Data split

80 / 10 / 10 (train / val / test) by lesion_id, not image_id, so no lesion appears in more than one split. This fixed leakage from the initial image-level split.

Class imbalance

Two complementary mechanisms were tried:

• CrossEntropyLoss with class_weight inversely proportional to class frequency.
• WeightedRandomSampler to balance classes per epoch.

After experiments, the sampler caused instability with a frozen backbone; class_weight in the loss was kept as the primary mechanism.

Progressive training (three rounds)

Regularization and control

• ReduceLROnPlateau — factor 0.5, patience 3.
• Early stopping — patience 5 on validation loss.
• Checkpoint — best model by val_loss.

Train-time augmentation

Horizontal flip, vertical flip, rotation ±20°, brightness / contrast / saturation jitter.

---

Results

The main metric is per-class accuracy, not global accuracy. On this dataset, global accuracy is misleading — always predicting nv yields ~67% without learning.

Approximate results from the best checkpoint:

~53% melanoma accuracy on held-out dermatoscopic images, without real-world clinical images or specialist validation, is a solid result for an academic prototype focused on triage.

---

System architecture

Client–server flow in three layers:

Mobile app (React Native + Expo)
    ↓  POST /analyze — image + JWT
Backend (FastAPI · Railway)
    ↓  preprocessing + inference
Model (ResNet-50 · ONNX Runtime)
    ↓  class probabilities
Backend persists result
    ↓  structured JSON
Mobile app shows outcome to the user

Infrastructure decisions

• PyTorch → ONNX Runtime for production: full PyTorch install ~800MB vs ONNX Runtime ~50MB — decisive for free-tier hosting. The model loads once at server startup and stays in memory.
• Supabase for PostgreSQL, image storage (S3-compatible), and auth including Google OAuth, reducing the number of external services to operate.

---

Full technology stack

---

Regulatory note

Publishing on the App Store and Play Store without medical-device certification is supported by positioning the product as screening, not diagnosis — probabilities plus clear disclaimers throughout the UX.