PingalaPingalaOpen app

Two observers independently rate the same 30 pressure-injury photographs using a four-category scale. Their ratings agree on 18 photographs. Which conclusion about inter-rater reliability is most reasonable?

Free Applied Statistics in Healthcare practice question on Reliability and validity for BSN and Post-RN nursing students. Choose your answer, then open the answer and the explanation.

  1. AAgreement should be ignored, because observers rated the same photographs rather than patients
  2. BAgreement is high enough to accept the scale for routine use without further analysis
  3. CAgreement shows the scale is valid, since consistent ratings reflect a true measure
  4. DAgreement is modest and should be checked against chance agreement before accepting the scale
Show answer and explanation

✓ Correct answer D. Agreement is modest and should be checked against chance agreement before accepting the scale

Explanation

Inter-rater reliability is the degree to which two or more independent raters give the same score to the same thing. Raw agreement alone can look good just by chance, especially when one category is used very often, so kappa or another chance-corrected statistic is used to check how much agreement exceeds chance. Agreement between raters tells us the measurement is consistent, not that it measures the right thing.

Why each option is right or wrong

  • ✗A. Agreement should be ignored, because observers rated the same photographs rather than patients — Rating the same photographs is exactly the correct design for inter-rater reliability, since both raters must judge identical material.
  • ✗B. Agreement is high enough to accept the scale for routine use without further analysis — Raw agreement of 60% is not strong evidence, and consistency between raters still needs chance correction before routine use is justified.
  • ✗C. Agreement shows the scale is valid, since consistent ratings reflect a true measure — Consistency between raters concerns reliability, while validity is whether the scale truly captures the intended construct.
  • ✓D. Agreement is modest and should be checked against chance agreement before accepting the scale — Sixty percent raw agreement is modest, and because chance alone can produce some agreement, a chance-corrected index such as kappa is needed before accepting the scale.
ConceptWhat it measuresTypical statistic
Inter-rater reliabilityAgreement between two or more ratersCohen's kappa, ICC
Test-retest reliabilityStability of scores over timePearson r, ICC
Internal consistencyConsistency among items of a scaleCronbach's alpha
ValidityWhether the tool measures the intended constructContent, criterion, construct evidence
EXAM HINT Reliability = repeatable; validity = correct. Kappa corrects raw agreement for chance.
اردو وضاحت
Inter-rater reliability کیا ہے؟
Inter-rater reliability کا مطلب یہ ہے کہ دو یا زیادہ raters ایک ہی چیز کو دیکھ کر کتنے ملتے جلتے score دیتے ہیں۔ اگر دونوں کا score قریب قریب ہو تو کہا جاتا ہے کہ measurement consistent ہے۔ لیکن صرف خام agreement دیکھنا کافی نہیں، کیونکہ کبھی کبھی اتفاق محض اتفاق سے بھی ہو جاتا ہے۔
اس صورت میں کیا نتیجہ نکالنا چاہیے؟
جب 30 میں سے صرف 18 پر اتفاق ہو تو agreement درمیانی ہے، اور اسے chance agreement کے خلاف جانچنا چاہیے، جیسے kappa نکال کر۔ صرف خام اتفاق دیکھ کر scale کو قبول کر لینا یا اسے validity سمجھ لینا غلط ہے، کیونکہ reliability اور validity دو الگ چیزیں ہیں۔

Key terms

  • chance agreement — Agreement that happens just by luck rather than because the raters truly agree. (ایسا اتفاق جو محض اتفاق سے ہو، اصل رضا مندی سے نہیں۔)

✓ Checked by Pingala Judge · Updated 17 Nov 2026

Practise 10 more Applied Statistics in Healthcare questions free →

Adaptive MCQs with explanations in English and Urdu, for BSN and Post-RN nursing students.

More Applied Statistics in Healthcare MCQs

All Applied Statistics in Healthcare MCQs · All subjects