Biostatistics Seminar: Jarcy Zee, PhD

Abstract: Novel kidney biopsy descriptors must be evaluated for reproducibility across pathologists prior to validation with clinical outcomes. Existing agreement statistics correct for chance agreement by adjusting observed agreement by the probability of chance agreement based on different assumptions. We propose a novel agreement statistic that accounts for the empirical probability of chance agreement, estimated by collecting additional data on rater uncertainty for each rating. A standard error estimator for the proposed statistic is derived. Simulation studies show our proposed statistic has the least bias in estimating the probability of agreement after removing chance agreement. Our proposed statistic and existing statistics are compared in a real data example from the Nephrotic Syndrome Study Network (NEPTUNE).