Lab Specification — Module FT03: Base Model Selection

Course: Course 3 — LLM Fine-Tuning Masterclass Module: FT03 — Base Model Selection Duration: 45–60 minutes (judgment and architecture — no GPU required) Environment: None. This is a paper-and-reasoning lab. A browser to read model cards is helpful but not required.

Learning objectives

By the end of this lab you will have:

Applied the five-dimension rubric (task, hardware, license, openness, ecosystem) to three realistic use cases with differing weight profiles.
Produced a defensible base selection for each use case — and, just as importantly, a documented rejection of the bases you did not pick.
Chosen the correct starting checkpoint (base / instruct / chat) for each use case and justified it.
Recognized the "wrong base" anti-patterns in the constraints (provenance gaps, capability ceilings, tokenizer mismatches) and named them.

This lab is deliberately GPU-free. The point is that base selection is a judgment problem upstream of any training run — and the most expensive mistakes (license failure, provenance loss, capability ceiling) are made here, before a single token is trained.

The selection template

For each use case, produce one selection card with this structure. Copy it three times.

### Selection Card — [Use Case Name]

**1. Dimension weights** (rate each 0.0–1.0; they must sum to a rationale, not necessarily to 1.0)
- Task:      ___  — reasoning: ___
- Hardware:  ___  — reasoning: ___
- License:   ___  — reasoning: ___
- Openness:  ___  — reasoning: ___
- Ecosystem: ___  — reasoning: ___

**2. Constraints inherited from the scenario** (list the hard gates)

**3. Shortlist** (2–3 candidate bases, with the family noted)

**4. Selected base**
- Family / checkpoint: ___
- Starting checkpoint (base / instruct / chat): ___
- Parameter count: ___

**5. Defense** — one paragraph per dimension, explaining why this base passes the gate.

**6. Rejections** — for each shortlisted base you did NOT pick, one sentence on why.

**7. Anti-pattern check** — name any "wrong base" anti-pattern the scenario tempts, and how you avoided it.

The three use cases

Use Case A — Medical QA bot (HIPAA-regulated hospital)

A hospital network wants an assistant that answers clinician questions about drug interactions, dosing, and contraindications. It must cite sources, flag uncertainty, and structure answers as a differential when appropriate. The model is served inside the hospital's existing governed environment; outputs are reviewed by a clinician before any clinical action.

Constraints:

Regulated under HIPAA. The hospital's compliance team requires the ability to prove what the training corpus contained — they will not accept "trust the publisher" for a model that touches patient-adjacent decisions.
Commercial deployment. The license must permit commercial use without negotiation.
The base must already know medicine well — fine-tuning is for the behavior (cite, flag uncertainty, structure as differential), not for teaching medicine. (FT00 thesis.)
Served on-prem on hospital GPUs (assume 2× A100 80GB available for training; 1× A100 for serving).
Latency target: clinician-facing, so under ~3 seconds/token at serving is fine; this is not a phone constraint.

Questions to answer on the card:

Which dimension dominates? (Hint: openness, given the audit requirement.)
Is open-weights-only acceptable here? Defend your answer.
Which checkpoint (base / instruct / chat) do you start from, and why?

Use Case B — Security exploit-code generator (authorized pentests)

A red-team consultancy wants a model that generates exploit code, shellcode, and attack chains for authorized penetration tests. The model must not refuse legitimate offensive-security requests, but it will be deployed inside a harness that bounds what it may do (target scoping, audit logging, allow-listed engagement IDs). The consultancy has no regulatory audit obligation for the model's training data.

Constraints:

The model must already know offensive security — exploit syntax, common CVE patterns, shellcode conventions. Fine-tuning steers the willingness and format, not the knowledge.
The refusal-removal pipeline (abliteration, FT17) will be applied. You want a base with a well-trodden ablation path and community precedent.
Commercial use; license must permit derivative uncensored work.
Training hardware: 4× RTX 4090 (24GB each). Serving: a single 4090 workstation on-prem.
No openness/audit requirement — the consultancy accepts open-weights-only.
Ecosystem: the abliteration tooling (Axolotl, Unsloth, community abliteration scripts) must support the base cleanly.

Questions to answer on the card:

Which family is the dominant ablation target, and what is the openness trade-off you are accepting?
Which checkpoint do you start from — and why is "chat" likely correct here (preserve alignment quality, remove only the refusal)?
Name the FT00 three-outcome test outcome this corresponds to (outcome 2: refuses but capability is there).

Use Case C — Phone-based on-device assistant

A consumer-electronics company wants an on-device assistant that runs entirely on a mid-range smartphone (no cloud calls). It handles calendar, messaging draft replies, summaries, and basic Q&A. Privacy is a marketing claim — all inference is local. The assistant ships pre-installed on millions of devices.

Constraints:

The model must fit in phone RAM alongside the OS — a hard ceiling of ~2–3GB of model weights at the chosen quantization (assume Q4 GGUF).
Latency: must feel responsive on a phone SoC — target >15 tokens/sec at Q4.
License: the model ships on consumer hardware at scale. Check the Llama community license's 700-million-monthly-active-user clause — this is exactly the scenario it was written for.
The base must be capable enough for the task (drafting, summarizing, simple Q&A) — but this is not hard reasoning. A 1B-class dense model is the realistic ceiling.
Ecosystem: must convert cleanly to GGUF/MLX for on-device serving; the company will not ship trust_remote_code.

Questions to answer on the card:

Which dimension dominates, and by how much? (Hint: hardware is near-absolute.)
Why is the Llama license a problem here specifically (and not in the other two use cases)?
Which family is purpose-built for this regime? Name the specific checkpoint.

Deliverables

Submit ft03-lab-report.md containing three completed selection cards (one per use case), each filling in the template above. Specifically:

Card A — Medical QA bot: dimension weights, shortlist, selected base + checkpoint, per-dimension defense, rejections, anti-pattern check.
Card B — Exploit-code generator: same structure.
Card C — Phone assistant: same structure.
A final 3–5 sentence reflection: which dimension was the decisive gate in each case, and what would have changed if that gate were relaxed?

No code. No training runs. This is the architecture decision that happens before you open a notebook.

Solution key

These are defensible choices, not single right answers. A card that selects a different base but defends it correctly against the rubric passes. A card that selects the "right" base with no defense fails.

Card A — Medical QA bot

Dominant dimension: Openness (the audit requirement is a hard gate) and License (commercial). Task sets the floor (must already know medicine).
Defensible selection: A Qwen2.5/Qwen3 instruct checkpoint in the 7–14B range (fits 2× A100 80GB for training, 1× A100 for serving), OR — if the audit requirement is interpreted strictly — an OLMo/Tülu or MiniCPM open-data base at the largest size that fits the hardware. The defensible move is to name the tension: Qwen wins on capability and ecosystem but is open-weights-only; if compliance requires open-data, you fall back to the open-data family and accept the capability/iteration trade-off.
Checkpoint: Instruct. You want reliable instruction-following to steer on top of; you are not building medicine from scratch (the base already knows it), and you are not ablating.
Rejections: Llama 3.x — open-weights-only and the license clause; not the audit story compliance wants. DeepSeek-R1-distill — reasoning-focused, overkill and the wrong profile for a citation-and-differential task.
Anti-pattern check: The scenario tempts "fine-tuning to teach medicine" — resisted by recognizing the FT00 thesis (behavior, not knowledge; RAG for the knowledge). Also tempts ignoring openness as a tie-breaker — it is a gate here.

Card B — Exploit-code generator

Dominant dimension: Task (must know offensive security) and Ecosystem (abliteration tooling). License must permit derivative uncensored work. Openness is not required.
Defensible selection: A Llama 3.x family checkpoint (the dominant ablation target) in the 8–14B range that fits 4× 4090 for QLoRA training. The openness trade-off (open-weights-only, not open-data) is explicitly accepted because there is no regulatory audit obligation. A Mistral/Mixtral or Dolphin-lineage base is a defensible alternative.
Checkpoint: Chat. You want to preserve the existing chat quality and steer narrowly — remove only the refusal (FT17 abliteration). Starting from base would throw away the alignment quality you want to keep.
Rejections: MiniCPM5-1B — wrong size class and not the ablation ecosystem. Qwen3 — defensible but less community ablation precedent than Llama for this specific pipeline (a card that picks Qwen3-chat and defends the ecosystem point still passes).
Anti-pattern check: FT00 outcome 2 (refuses, capability there). The scenario tempts "merged community model with no provenance" (SuperMega-Uncensored merges) — resisted by choosing the canonical upstream checkpoint and applying the abliteration yourself, preserving provenance (ties to FT02).

Card C — Phone assistant

Dominant dimension: Hardware (near-absolute — phone RAM and SoC throughput). License is the secondary gate (the 700M MAU clause).
Defensible selection: MiniCPM5-1B (or MiniCPM5 at the 1B class). Purpose-built for on-device, Apache-2.0 (no MAU clause), dense, converts cleanly to GGUF, runs at Q4 in ~2GB. SmolLM3 is a defensible open-data alternative at a similar size.
Checkpoint: Instruct. You want reliable instruction-following for the assistant behaviors; you are not ablating.
Rejections: Llama 3.2 (the small Llama) — the community license's 700M MAU clause is triggered by shipping on millions of devices; this is exactly the scenario it covers. Qwen at any size above ~1.5B — will not fit the phone RAM budget at the target quantization and latency. Any MoE — expert-loading latency is wrong for a phone SoC.
Anti-pattern check: The scenario tempts "ignoring the VRAM filter" (picking a 7B because it's "better") — the FT01 math vetoes it. Also tempts the Llama license trap — the MAU clause is a real gate at this deployment scale, not a tie-breaker.

Reflection (model answer)

In A the decisive gate was openness (audit); relaxing it would have made Qwen the clear winner on capability and ecosystem. In B the decisive gate was ecosystem (abliteration precedent); relaxing the task's offensive-security knowledge requirement would have opened Qwen3 as a peer choice. In C the decisive gate was hardware (phone physics); relaxing the on-device constraint to "cloud-assisted" would have made a 7B Qwen or Llama viable and changed the license picture entirely. The lesson: the same rubric produces radically different selections when the weights shift — which is why writing the weights down first, before browsing the leaderboard, is the discipline this module teaches.

Stretch goals

Add a fourth use case of your own — an air-gapped government deployment (FT21/FT22 preview). Which dimension dominates, and why does openness plus license-plus-air-gap interact?
Stress-test your Card A selection — if compliance insists on open-data and the open-data base you chose scores below the hospital's reasoning floor on the medical eval, what do you do? (Hint: this is the FT00 outcome-3 trap — the answer may be a different intervention than fine-tuning.)
Tokenizer audit — pick your Card C base and your Card A base. Estimate tokens-per-character for a representative domain sample (a dosing table; a calendar event string). Does tokenizer fit favor or disfavor your selection?

# Lab Specification — Module FT03: Base Model Selection **Course**: Course 3 — LLM Fine-Tuning Masterclass **Module**: FT03 — Base Model Selection **Duration**: 45–60 minutes (judgment and architecture — no GPU required) **Environment**: None. This is a paper-and-reasoning lab. A browser to read model cards is helpful but not required. --- ## Learning objectives By the end of this lab you will have: 1. **Applied the five-dimension rubric** (task, hardware, license, openness, ecosystem) to three realistic use cases with differing weight profiles. 2. **Produced a defensible base selection** for each use case — and, just as importantly, a documented *rejection* of the bases you did not pick. 3. **Chosen the correct starting checkpoint** (base / instruct / chat) for each use case and justified it. 4. **Recognized the "wrong base" anti-patterns** in the constraints (provenance gaps, capability ceilings, tokenizer mismatches) and named them. This lab is deliberately GPU-free. The point is that base selection is a *judgment* problem upstream of any training run — and the most expensive mistakes (license failure, provenance loss, capability ceiling) are made here, before a single token is trained. --- ## The selection template For each use case, produce one **selection card** with this structure. Copy it three times. ```markdown ### Selection Card — [Use Case Name] **1. Dimension weights** (rate each 0.0–1.0; they must sum to a rationale, not necessarily to 1.0) - Task: ___ — reasoning: ___ - Hardware: ___ — reasoning: ___ - License: ___ — reasoning: ___ - Openness: ___ — reasoning: ___ - Ecosystem: ___ — reasoning: ___ **2. Constraints inherited from the scenario** (list the hard gates) **3. Shortlist** (2–3 candidate bases, with the family noted) **4. Selected base** - Family / checkpoint: ___ - Starting checkpoint (base / instruct / chat): ___ - Parameter count: ___ **5. Defense** — one paragraph per dimension, explaining why this base passes the gate. **6. Rejections** — for each shortlisted base you did NOT pick, one sentence on why. **7. Anti-pattern check** — name any "wrong base" anti-pattern the scenario tempts, and how you avoided it. ``` --- ## The three use cases ### Use Case A — Medical QA bot (HIPAA-regulated hospital) A hospital network wants an assistant that answers clinician questions about drug interactions, dosing, and contraindications. It must cite sources, flag uncertainty, and structure answers as a differential when appropriate. The model is served inside the hospital's existing governed environment; outputs are reviewed by a clinician before any clinical action. **Constraints:** - Regulated under HIPAA. The hospital's compliance team requires the ability to *prove* what the training corpus contained — they will not accept "trust the publisher" for a model that touches patient-adjacent decisions. - Commercial deployment. The license must permit commercial use without negotiation. - The base must already know medicine well — fine-tuning is for the *behavior* (cite, flag uncertainty, structure as differential), not for teaching medicine. (FT00 thesis.) - Served on-prem on hospital GPUs (assume 2× A100 80GB available for training; 1× A100 for serving). - Latency target: clinician-facing, so under ~3 seconds/token at serving is fine; this is not a phone constraint. **Questions to answer on the card:** - Which dimension dominates? (Hint: openness, given the audit requirement.) - Is open-weights-only acceptable here? Defend your answer. - Which checkpoint (base / instruct / chat) do you start from, and why? ### Use Case B — Security exploit-code generator (authorized pentests) A red-team consultancy wants a model that generates exploit code, shellcode, and attack chains for *authorized* penetration tests. The model must not refuse legitimate offensive-security requests, but it will be deployed inside a harness that bounds what it may do (target scoping, audit logging, allow-listed engagement IDs). The consultancy has no regulatory audit obligation for the model's training data. **Constraints:** - The model must already know offensive security — exploit syntax, common CVE patterns, shellcode conventions. Fine-tuning steers the *willingness* and *format*, not the knowledge. - The refusal-removal pipeline (abliteration, FT17) will be applied. You want a base with a well-trodden ablation path and community precedent. - Commercial use; license must permit derivative uncensored work. - Training hardware: 4× RTX 4090 (24GB each). Serving: a single 4090 workstation on-prem. - No openness/audit requirement — the consultancy accepts open-weights-only. - Ecosystem: the abliteration tooling (Axolotl, Unsloth, community abliteration scripts) must support the base cleanly. **Questions to answer on the card:** - Which family is the dominant ablation target, and what is the openness trade-off you are accepting? - Which checkpoint do you start from — and why is "chat" likely correct here (preserve alignment quality, remove only the refusal)? - Name the FT00 three-outcome test outcome this corresponds to (outcome 2: refuses but capability is there). ### Use Case C — Phone-based on-device assistant A consumer-electronics company wants an on-device assistant that runs entirely on a mid-range smartphone (no cloud calls). It handles calendar, messaging draft replies, summaries, and basic Q&A. Privacy is a marketing claim — all inference is local. The assistant ships pre-installed on millions of devices. **Constraints:** - The model must fit in phone RAM alongside the OS — a hard ceiling of ~2–3GB of model weights at the chosen quantization (assume Q4 GGUF). - Latency: must feel responsive on a phone SoC — target >15 tokens/sec at Q4. - License: the model ships on consumer hardware at scale. Check the Llama community license's 700-million-monthly-active-user clause — this is exactly the scenario it was written for. - The base must be capable enough for the *task* (drafting, summarizing, simple Q&A) — but this is not hard reasoning. A 1B-class dense model is the realistic ceiling. - Ecosystem: must convert cleanly to GGUF/MLX for on-device serving; the company will not ship `trust_remote_code`. **Questions to answer on the card:** - Which dimension dominates, and by how much? (Hint: hardware is near-absolute.) - Why is the Llama license a problem *here specifically* (and not in the other two use cases)? - Which family is purpose-built for this regime? Name the specific checkpoint. --- ## Deliverables Submit `ft03-lab-report.md` containing three completed selection cards (one per use case), each filling in the template above. Specifically: - [ ] **Card A — Medical QA bot**: dimension weights, shortlist, selected base + checkpoint, per-dimension defense, rejections, anti-pattern check. - [ ] **Card B — Exploit-code generator**: same structure. - [ ] **Card C — Phone assistant**: same structure. - [ ] A final 3–5 sentence reflection: which dimension was the decisive gate in each case, and what would have changed if that gate were relaxed? No code. No training runs. This is the architecture decision that happens *before* you open a notebook. --- ## Solution key These are *defensible* choices, not single right answers. A card that selects a different base but defends it correctly against the rubric passes. A card that selects the "right" base with no defense fails. ### Card A — Medical QA bot - **Dominant dimension:** Openness (the audit requirement is a hard gate) and License (commercial). Task sets the floor (must already know medicine). - **Defensible selection:** A Qwen2.5/Qwen3 instruct checkpoint in the 7–14B range (fits 2× A100 80GB for training, 1× A100 for serving), **OR** — if the audit requirement is interpreted strictly — an OLMo/Tülu or MiniCPM open-data base at the largest size that fits the hardware. The defensible move is to name the tension: Qwen wins on capability and ecosystem but is open-weights-only; if compliance requires open-data, you fall back to the open-data family and accept the capability/iteration trade-off. - **Checkpoint:** Instruct. You want reliable instruction-following to steer *on top of*; you are not building medicine from scratch (the base already knows it), and you are not ablating. - **Rejections:** Llama 3.x — open-weights-only and the license clause; not the audit story compliance wants. DeepSeek-R1-distill — reasoning-focused, overkill and the wrong profile for a citation-and-differential task. - **Anti-pattern check:** The scenario tempts "fine-tuning to teach medicine" — resisted by recognizing the FT00 thesis (behavior, not knowledge; RAG for the knowledge). Also tempts ignoring openness as a tie-breaker — it is a gate here. ### Card B — Exploit-code generator - **Dominant dimension:** Task (must know offensive security) and Ecosystem (abliteration tooling). License must permit derivative uncensored work. Openness is *not* required. - **Defensible selection:** A Llama 3.x family checkpoint (the dominant ablation target) in the 8–14B range that fits 4× 4090 for QLoRA training. The openness trade-off (open-weights-only, not open-data) is explicitly accepted because there is no regulatory audit obligation. A Mistral/Mixtral or Dolphin-lineage base is a defensible alternative. - **Checkpoint:** Chat. You want to preserve the existing chat quality and steer *narrowly* — remove only the refusal (FT17 abliteration). Starting from base would throw away the alignment quality you want to keep. - **Rejections:** MiniCPM5-1B — wrong size class and not the ablation ecosystem. Qwen3 — defensible but less community ablation precedent than Llama for this specific pipeline (a card that picks Qwen3-chat and defends the ecosystem point still passes). - **Anti-pattern check:** FT00 outcome 2 (refuses, capability there). The scenario tempts "merged community model with no provenance" (SuperMega-Uncensored merges) — resisted by choosing the canonical upstream checkpoint and applying the abliteration yourself, preserving provenance (ties to FT02). ### Card C — Phone assistant - **Dominant dimension:** Hardware (near-absolute — phone RAM and SoC throughput). License is the secondary gate (the 700M MAU clause). - **Defensible selection:** MiniCPM5-1B (or MiniCPM5 at the 1B class). Purpose-built for on-device, Apache-2.0 (no MAU clause), dense, converts cleanly to GGUF, runs at Q4 in ~2GB. SmolLM3 is a defensible open-data alternative at a similar size. - **Checkpoint:** Instruct. You want reliable instruction-following for the assistant behaviors; you are not ablating. - **Rejections:** Llama 3.2 (the small Llama) — the community license's 700M MAU clause is triggered by shipping on millions of devices; this is exactly the scenario it covers. Qwen at any size above ~1.5B — will not fit the phone RAM budget at the target quantization and latency. Any MoE — expert-loading latency is wrong for a phone SoC. - **Anti-pattern check:** The scenario tempts "ignoring the VRAM filter" (picking a 7B because it's "better") — the FT01 math vetoes it. Also tempts the Llama license trap — the MAU clause is a real gate at this deployment scale, not a tie-breaker. ### Reflection (model answer) In A the decisive gate was openness (audit); relaxing it would have made Qwen the clear winner on capability and ecosystem. In B the decisive gate was ecosystem (abliteration precedent); relaxing the task's offensive-security knowledge requirement would have opened Qwen3 as a peer choice. In C the decisive gate was hardware (phone physics); relaxing the on-device constraint to "cloud-assisted" would have made a 7B Qwen or Llama viable and changed the license picture entirely. The lesson: the *same* rubric produces radically different selections when the weights shift — which is why writing the weights down first, before browsing the leaderboard, is the discipline this module teaches. --- ## Stretch goals 1. **Add a fourth use case of your own** — an air-gapped government deployment (FT21/FT22 preview). Which dimension dominates, and why does openness plus license-plus-air-gap interact? 2. **Stress-test your Card A selection** — if compliance insists on open-data *and* the open-data base you chose scores below the hospital's reasoning floor on the medical eval, what do you do? (Hint: this is the FT00 outcome-3 trap — the answer may be a different intervention than fine-tuning.) 3. **Tokenizer audit** — pick your Card C base and your Card A base. Estimate tokens-per-character for a representative domain sample (a dosing table; a calendar event string). Does tokenizer fit favor or disfavor your selection?