Minimum Qualifications Master’s degree in Educational Psychology, Educational Measurement, Industrial/Organizational Psychology, Psychometrics, or a closely related field AND at least 5 years of professional or managerial experience in planning, developing, evaluating, and validating high-stakes tests for large-scale testing, including at least 2 years in a leadership role, such as project lead, supervisor, manager, etc., OR
PhD in Educational Psychology, Educational Measurement, Industrial/Organizational Psychology, Psychometrics, or closely related field AND at least 4 years of professional or managerial experience in planning, developing, evaluating, and validating high-stakes tests for large-scale testing, including at least 2 years in a leadership role, such as project lead, supervisor, manager, etc.
Duties Description Under the general direction of the Director of Testing Services, the incumbent will be responsible for providing guidance and expertise in the design, analysis, and validation of tools and tests created and/or used in personnel assessment and selection by the New York State Department of Civil Service.
The incumbent will be responsible for providing guidance and assistance to the Director of Testing Services in the creation and staffing of the Division’s new Test Validation and Evaluation Unit. The incumbent will then direct and oversee the activities of the Test Validation and Evaluation Unit in carrying out its duties. The Manager Testing Services (Psychometrician) will supervise a staff of subordinate specialists including a Human Resources Specialist 4 and support staff. Incumbents operate under policy direction and broad guidelines within assigned area of responsibility with periodic review of completed work for conformance with established goals and priorities.
The incumbent will advise the Director of Testing Services and the Department executive team on high-profile issues, such as the validity, reliability, and defensibility of assessment and selection strategies; research methods and techniques; test development and validation techniques; statistical and other analytic tools and processes; professional and legal standards; and other assessment-related topics.
Incumbent responsibilities may include but are not limited to:
• Lead personnel assessment and accountability-related psychometric research for a large-scale workforce selection and development program
• Design, plan, and conduct a variety of test validation studies and research projects
• Coordinate competency modeling and traditional job analysis studies as a basis for personnel assessment and employee evaluation programs
• Coordinate development of valid and legally defensible personnel assessment and evaluation strategies for selection, retention, employee development, and performance management
• Guide development and administration of job-related tests in a variety of types and formats, including written, multiple-choice, situational judgment, job knowledge, job simulation, constructed response, evaluation of training and experience, oral, as well as structured interview and performance evaluation systems and similar tools
• Conduct ongoing statistical analysis and validation of existing tests and assessment programs
• Provide analysis of special populations and assist in ensuring that test items are free from statistical, racial, gender, cultural, or other bias
• Ensure tests and assessments comply with legal and professional standards (e.g., Uniform Guidelines on Employee Selection, Standards for Educational & Psychological Testing, Principles for the Validation and Use of Personnel Selection Procedures, etc.) and industry best practices
• Prepare technical documentation on test validity sufficient for legal inquires and in accordance with the Uniform Guidelines on Employee Selection Procedures
• Conduct test equating analyses and demonstrate equivalency in test forms
• Develop and train staff in test and measurement concepts, statistical analysis, and test validation techniques
• Align tests and assessments with organizational goals and performance expectations
• Advise executive management on psychometric and technical assessment and selection issues, programs, and trends
• Monitor section’s expenditures and prepare budget requests for Group
• Represent the Division and the Department in legal and Commission proceedings, and in public forums
• Develop and maintain effective working relationships with agencies, staff from within the Department, and various entities such as the Personnel Council
• Draft proposed legislation and recommend rule changes to improve the testing program
Analyze and prepare comments on legislation impacting the Department’s testing program
Additional Comments NOTE ON TELECOMMUTING: DCS Employees are required to apply and obtain approval through management to telecommute according to DCS Telecommuting Pilot Program Guidelines
NOTE ON COVID-19 VACCINATION/WEEKLY TESTING: All State employees in Executive controlled agencies who are not subject to a mandatory vaccination requirement shall be tested for COVID-19 weekly unless they are fully vaccinated.
Some positions may require additional credentials or a background check to verify your identity.
Office of Human Resources and Administrative Planning