Nov 02, 2019
GENERAL FUNCTION The primary responsibility of the Senior Psychometrician is to serve as the technical lead for the EMA Character Skills Snapshot program, conduct all aspects of the work necessary for score reporting, including scoring, item analysis, equating, quality assurance, trend analysis, and research to shape and fulfill the requirements for the Character Skills Snapshot program. PRIMARY RESPONSIBILITIES Plan and coordinate test development, operational and psychometric work of the CSS program, including item pretesting plan, item analysis, test analysis, scoring, equating, and other aspects of operational work to ensure the quality and the smooth execution of the program. Lead the CSS research work. Work with test developers, technology architects and Director of the CSS program developing CSS assessment blueprint; examine scoring method; explore non-cognitive equating design and equating methods; lead operational validity research, manage data collection process, conduct analysis and generate reports. Provide psychometric guidance and support for the SSAT program when needed. Conduct equating and other aspects of operational work required for score reporting and data interpretation, and to ensure the high quality and the success of the SSAT programs. Serve as a consultant on technical, psychometric and test design issues and statistical processes to EMA senior managers and clients; respond to requests for data and data interpretation from officers, clients, and users; support the creation of new examination programs as requested by the executive team. Identify research and measurement-related problems, opportunities and their psychometric implications; assume responsibility for planning, coordinating, and conducting research projects, including initiating proposals, designing the study, conducting analyses, preparing reports and giving presentations to stakeholders. Make professional contributions through articles, speeches, seminars, and participation in professional organizations; contribute to research, development, and evaluation of new testing modes, item types, psychometric methods, and technology. EDUCATION & EXPERIENCE Ph.D. or Ed.D. in educational measurement, educational psychology, developmental psychology, social psychology, industrial/organizational psychology, statistics or a closely related field or equivalent. Five-ten years of relevant experience is required. Demonstrated experience with IRT models. Experience developing, refining, and validating non-cognitive assessments. Ideally, the experience should include factor analysis of multi-dimensional items, item analysis for situational judgment tests, development of structured interviews or video interviews, or item analysis of personality tests. Preferred background includes an understanding of methods for identification and control of faking behavior during assessments. Preferred background also includes familiarity with evaluating fairness and bias in assessments including DIF and differential prediction. Demonstrated fundamental psychometric concepts and techniques such as score equating, scaling, concordance, validity, reliability, factor analysis, item analysis, and so on. Experience in a testing organization is preferred: responsible for statistical analyses including item analyses, equating and score reporting for testing programs. Demonstrated knowledge of research methodologies such as the design of experiments, sampling theory, and field data collection techniques. Demonstrated knowledge of statistical analysis software such as R, MPlus, SPSS, SAS. Knowledge of commercial IRT software such as IRTPRO, Bilog, Winsteps, and Parscale Excellent writing skills preferably demonstrated in either published papers or technical manuals (samples may be requested).
The Enrollment Management Association
Skillman, Montgomery, NJ, USA