Localization training methods: forget–utility Pareto

8B, streaming (1-QA-at-a-time), head frozen. Numbered dot = method. Blue=full-FT, orange=LoRA; green ring = meets goal (fkm≤0.17 & official MU≥0.60; base MU=0.628). B=base (no unlearning), T=target. x=Forget-KnowMem (left=more forgotten), y=official TOFU model_utility. Click a legend entry to expand its layer×module training grid (green=trained).

Legend — click to see which modules each method trains

1. full all-params — fkm 0.117, MMLU 0.452 (224 modules)

	q	k	v	o	gate	up	down
0
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31

2. LoRA all-modules ✅ — fkm 0.167, MMLU 0.600 (224 modules)

	q	k	v	o	gate	up	down
0
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31

3. full ΔW>0.01 (drop k_proj) — fkm 0.116, MMLU 0.464 (184 modules)

	q	k	v	o	gate	up	down
0
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31

4. LoRA ΔW>0.01 (drop k_proj) ✅ — fkm 0.168, MMLU 0.619 (184 modules)

	q	k	v	o	gate	up	down
0
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31

5. full first3+ovdu — fkm 0.125, MMLU 0.480 (137 modules)

	q	k	v	o	gate	up	down
0
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31

6. LoRA first3+ovdu — fkm 0.224, MMLU 0.619 (137 modules)

	q	k	v	o	gate	up	down
0
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31

7. full v+down all-layers — fkm 0.131, MMLU 0.550 (64 modules)

	q	k	v	o	gate	up	down
0
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31

8. full v+down @LR5e-6 ✅ — fkm 0.130, MMLU 0.655 (64 modules)

	q	k	v	o	gate	up	down
0
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31

9. LoRA v+down all-layers — fkm 0.327, MMLU 0.598 (64 modules)

	q	k	v	o	gate	up	down
0
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31

10. LoRA v+down @LR5e-6 — fkm 0.312, MMLU 0.580 (64 modules)

	q	k	v	o	gate	up	down
0
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31

11. full v+up all-layers ✅ — fkm 0.120, MMLU 0.616 (64 modules)

	q	k	v	o	gate	up	down
0
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31

12. LoRA v+up all-layers — fkm 0.168, MMLU 0.585 (64 modules)

	q	k	v	o	gate	up	down
0
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31

13. full up+down all-layers — fkm 0.113, MMLU 0.548 (64 modules)

	q	k	v	o	gate	up	down
0
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31

14. LoRA up+down all-layers — fkm 0.208, MMLU 0.584 (64 modules)

	q	k	v	o	gate	up	down
0
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31

15. full v+up+down all-layers — fkm 0.120, MMLU 0.523 (96 modules)

	q	k	v	o	gate	up	down
0
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31

16. LoRA v+up+down all-layers — fkm 0.214, MMLU 0.592 (96 modules)

	q	k	v	o	gate	up	down
0
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31

17. full down_proj-only — fkm 0.121, MMLU 0.591 (32 modules)

	q	k	v	o	gate	up	down
0
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31

18. full v_proj-only — fkm 0.309, MMLU 0.615 (32 modules)

	q	k	v	o	gate	up	down
0
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31

19. LoRA localized (L0-1+vproj) — fkm 0.268, MMLU 0.538 (44 modules)

	q	k	v	o	gate	up	down
0
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31

20. full hotspot (vL0-2+dpL1) — fkm 0.492, MMLU 0.621 (4 modules)

	q	k	v	o	gate	up	down
0
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31

21. LoRA hotspot (vL0-2+dpL1) — fkm 0.397, MMLU 0.659 (4 modules)

	q	k	v	o	gate	up	down
0
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31

B. Base model (no unlearning) — fkm 0.979, MMLU 0.628 (0 trained)

Starting point: memorizes authors 0-4 (high fkm), full utility.

T. Target (ideal, approx) — fkm ~0.10, MMLU 0.628

Forgotten + base utility. No 8B retrained oracle exists for authors 0-4; shown as the ideal top-left corner.