A guided reading of one paper
Focal Loss for Dense Object Detection.
Why would a model need to spend less effort on examples it already gets right? Work through the idea, try it yourself, and check each step against the paper.
Browse the complete paper 10 pages · original PDF
AI questions are for readers 18 or older.
If you use English voice, this browser downloads about 65 MB of voice files on first use and transcribes on your device. Audio is discarded; only your submitted question reaches AI.
By continuing, you agree to the provider terms and AtlasCloud terms. Reading stays available.
Paper history
Your questions, explanations and drawings. Replay uses the saved reply. This history does not sync with Google.
The device file contains only available copies for this paper and current reader. It may omit server or unreadable records and is not a full server archive.
We’ll follow one idea: changing how much each training example contributes to the loss. Start with the problem, explore the equation, then see what the authors tested. The full paper is here whenever you need it.
Your notes stay on this device. Privacy details
Press V to ask. Circle public content if useful.
English voice runs on this device. First use downloads about 65 MB of voice files, including a model from Hugging Face, and caches model files. Audio stays in memory and is discarded. Only the question you submit goes to AI.
Intro · start with the problem
Why easy examples can get in the way.
Imagine teaching a model to find objects in a photo. It checks many locations and asks: “Is there an object here?” Most locations are background.
Many background locations An object
Margin illustration · not an image or measurement from the paperTraining uses a loss to measure how far each prediction is from the known answer. Once the model gets a background example right with high confidence, that example adds only a small loss. A huge number of small losses can still add up.
Pause here. If most of the total loss comes from examples the model already gets right, what might happen to the examples it still struggles with?
The authors argue that many easy background examples can overwhelm the useful training signal. Their proposed fix is focal loss: reduce the contribution of examples the model already gets right with high confidence. The crowded grid illustrates the problem; the evidence comes from their RetinaNet experiments, which we’ll examine later.
Save your prediction to unlock the next step.
Source: the authors' problem statement in the abstract and introduction, pp. 1–2.
What makes an example easy?
Think about one location in that photo. We know whether it contains an object. The model makes a prediction. To judge that prediction, we need three ideas.
- The label
- The answer we know from the training data: object or background. It’s what we compare the model’s prediction with.
- Confidence in the correct label
- How much probability the model gives that known answer. If it confidently predicts background where there is an object, it is confidently wrong.
- The loss
- A penalty for the prediction. The usual loss, called cross entropy, is small when the model gives the correct answer high probability. It is large when the model gives that answer very little probability.
Keep the model’s prediction fixed: it strongly predicts background. Change the actual label below. Is the same prediction still good?
The model is right and confident. This is an easy example, so cross entropy gives it a small loss.
So “easy” means confident in the correct answer. Focal loss uses this distinction to shrink easy examples’ contributions further. That gives difficult examples a larger share of the remaining loss.
Margin's teaching examples explain §3.1–3.2 and Eqs. 1–4, p. 3.
Turn down the easy examples.
Focal loss starts with cross entropy and multiplies it by a weight. When focusing is on, easy examples get smaller weights than difficult ones. A setting called gamma controls how strongly this happens. Let’s change gamma while keeping the predictions fixed.
One example is easy; the other is difficult. Before moving the slider, which one’s weight do you expect to fall more as gamma increases?
Show the numbers behind these weights
Fixed true-class probabilities: easy = 0.9, difficult = 0.1. Weight = (1 − pₜ)γ. These bars show the multiplier, not total loss.
Calculated from Eq. 4. Predictions stay fixed; no detector is trained and no accuracy or AP result is produced.
See how the equation makes this happen
The weight has a simple recipe: take the probability of the correct label, subtract it from one, then raise the result to gamma. The paper calls that probability pₜ.
In , the loss is FL(pₜ) = −(1 − pₜ)γ log(pₜ) before the optional class weight. The −log(pₜ) part is cross entropy. The (1 − pₜ)γ part is the weight you just explored.
Loading this part of the paper…
Choose the probability of the correct label: p for an object, 1 − p for background. The paper labels these +1 and −1; our examples use 1 and 0 for the same two cases.
Open original page ↗Loading this part of the paper…
Find the cross-entropy term, −log(pₜ), then the weight multiplying it. For an easy example, pₜ is near 1, so 1 − pₜ is small.
Open original page ↗Now compare that with Figure 1. Its curves show the loss after applying the weight. Our bars show just the weight.
Loading this part of the paper…
Look near the right edge, where the model is confident in the correct label. How does increasing gamma change the loss there?
Open original page ↗Which probability goes into the equation?
Let’s put numbers to the label example. Here p is the model’s probability of an object. We need pₜ, its probability of the correct label. For an object, use p. For background, use 1 − p.
The target is background (y = 0), and the model gives the positive/object class probability p = 0.2. Which pₜ enters the focal factor?
For a background target, the true-class probability is 1 − p. The focal factor is calculated after you find pₜ.
For y = 0, pₜ = 1 − p = 0.8. At γ = 2, the focal multiplier is (1 − 0.8)² = 0.04. When γ = 0, the multiplier is 1 and the loss becomes ordinary cross entropy; larger γ suppresses already-confident examples more. This is an equation-level calculation, not a RetinaNet run or a reproduced COCO result.
Did it help the detector?
The slider shows what the equation does to two fixed predictions. To find out whether the idea helps during training, we need an experiment. Let’s look at the authors’ comparison.
Open and compare α-balanced cross entropy with focal loss. The result column, AP, means average precision: the detector’s score on the stated COCO evaluation. It is not the percentage of examples classified correctly.
Loading this part of the paper…
Read across gamma, alpha and AP. The authors choose alpha separately for each gamma. The caption fixes the setup: ResNet-50-FPN, 600-pixel images and COCO minival.
Open original page ↗Pick two rows and note their AP values. Which settings change? Explain what you can learn from the comparison and what you still cannot tell.
Find a baseline row and a focal-loss row. Check both gamma and alpha before interpreting the difference in AP.
One useful pair is γ = 0, α = 0.75 at 31.1 AP, and γ = 2, α = 0.25 at 34.0 AP. The focal-loss configuration scores higher in this experiment. But gamma and alpha both change, so this pair cannot tell us the effect of changing gamma alone at fixed alpha. It also does not establish a gain for every detector.
Figure 4 lets us inspect where the loss falls within each class. Each curve is normalized separately, so it cannot tell us how much total loss comes from background versus objects.
Table 2’s 39.1 AP test-dev result is for a full RetinaNet configuration. It does not isolate the loss by itself. These are the authors’ reported results; Margin has not rerun the experiments.
Loading this part of the paper…
The negative examples run from lower to higher loss. The curve adds up their loss, scaled to total 1 within this class. Look at how much is concentrated near the difficult end; this does not compare total negative and positive loss.
Open original page ↗Loading this part of the paper…
The positive examples are sorted the same way, but their loss is scaled to total 1 separately. Both plots use predictions from one model trained with gamma = 2.
Open original page ↗Use the idea yourself.
Try a fresh example to see if you can use the two steps yourself. We’ll close the paper and earlier notes while you try. You can reopen help at any time; we’ll record that you used it.
A positive/object target (y = 1) has positive-class probability p = 0.6, with γ = 2. Enter pₜ and the focal multiplier (1 − pₜ)γ, then explain your steps.
Show the original equation as help
Loading this part of the paper…
Use the true label to choose the correct branch. Opening this counts as help for the check.
Open original page ↗The true label is positive, so pₜ = p = 0.6. The multiplier is (1 − 0.6)² = 0.16. This weights the example relative to cross entropy; it does not predict detector AP. Compare your explanation with these two steps.
What would you tell someone else?
Imagine a friend asks what focal loss changes and why it matters. Leave yourself four short notes you could use to explain it. Check the source links where you need them. Your notes stay in this browser.
Want to implement this? The calculation is a start. You’ll also need stable loss computation, initialization and normalization by positive anchors. The reading list below points to those details.
Continue through the full paper
You’ve worked through the loss idea. Building the detector takes a few more pieces. Here’s where to find them:
- : combine sigmoid and loss computation for numerical stability.
- : initialization, class imbalance, and what two-stage sampling changes.
- : classifier initialization and loss normalized by positive/assigned anchors, rather than all anchors.
- : architecture choices and complete-detector comparisons.
- : alternative loss form and derivatives.
As you continue, ask what the loss comparison isolates, what else changes in RetinaNet, and what you would need to reproduce the reported result. The checks here cover the loss calculation; they do not establish that you can implement or reproduce the full detector.
Can you still use it tomorrow?
Try the new case in Apply first. Another case will be ready 24 hours later, so you can see what you remember.
A background target (y = 0) has positive-class probability p = 0.25. This time γ = 1. Enter pₜ and the focal multiplier, and explain why you used that label branch.
Show the original equation as help
Loading this part of the paper…
Use the true label to choose the correct branch. Opening this counts as help for the check.
Open original page ↗For background, pₜ = 1 − 0.25 = 0.75. With γ = 1, the multiplier is 1 − 0.75 = 0.25. Compare the label choice and exponent with your explanation. This is one small check of recall, not a measure of mastery.
Did this help you read?
Could you explain why the weight changes, or spot a limit in the experiment? Tell us whether the guide helped you do that.
What could be clearer? Optional