The mission of the Office of Assessment and Evaluation (OAE) is to enhance the educational programs in the Johns Hopkins University School of Medicine (JHUSOM) through evidence-based best practices in the assessment of learners and the evaluation of programs. OAE partners with JHUSOM faculty and staff members, administrators, and learners to design, launch, and maintain assessment and evaluation initiatives in graduate biomedical education, postdoctoral education, undergraduate medical education, graduate medical education, and continuing medical education. OAE also supports educational activities initiated by the Simulation Center, the Institute for Excellence in Education, the Office of Student Affairs, the Office of Faculty Development, and the Professional Development and Career Office.
As part of its core mission, OAE:
Supports the assessment of learners and the evaluation of programs through the selection, design, vetting, and implementation of assessment and evaluation tools (e.g., rubric design, multiple-choice question exam development, survey design, interview guide development)
Facilitates understanding of the assessment of learners and the evaluation of programs through the analysis, reporting, and use of related data (e.g., statistical analysis of quantitative data, thematic content analysis of qualitative data)
Conducts faculty development activities in areas related to learner assessment and program evaluation (e.g., validity, reliability, exam blueprints, standard setting, test and item analysis)
As a secondary aim, OAE also provides consultation in research design, statistical analysis and interpretation, and manuscript preparation intended for scholarly reports in medical and biomedical education, as projects relate to assessment and evaluation initiatives at JHUSOM.
With the director and the OAE team, the Program Evaluation Specialist will provide a vision for, and implementation of, program evaluation for JHUSOM medical and biomedical education, and the initiatives that support educational excellence, such as the Institute for Excellence in Education, the Office of Student Diversity, and the Office of Professional and Career Development.
The position reports to the Director of the Office of Assessment and Evaluation, and works closely with the OAE team and other stakeholders, including faculty and staff.
Specific Duties & Responsibilities
Specific responsibilities of the Program Evaluation Specialist include, but are not limited to, the following:
Assist the OAE Director with strategic planning initiatives related to program evaluation
Serve as lead OAE project manager on development of evaluation plans and design of comprehensive evaluation systems
Serve as lead OAE writer on evaluation plans for grant proposals
Serve as lead OAE project manager on development, implementation, and use of program evaluation instruments/data collection
Use program evaluation models including Logic Models to build strategic program evaluation plans
Manage process to determine data necessary for program evaluation; collect and/or oversee collection of such data for all SOM educational programs
Support course revisions and curriculum planning from evidence-based program evaluation results
Provide support to faculty for education research involving program evaluation
Serve as an active member of various committees as needed
Supervise staff and/or serve as team leader for program evaluation initiatives
Advise faculty members, staff members, and learners in best practices related to program evaluation
Assist the Director of OAE with systemic program evaluation process improvement strategies to advance day-to-day operations
Advise faculty members, staff members, and learners in the interpretation of program evaluation reports to facilitate data-driven decision making and action planning
Assist the Director of OAE with the production of recurring OAE newsletters
Maintain professional and effective working relationships with JHUSOM educational program stakeholders and with partnering units, including the JHUSOM Office of Information Technology, Office of Medical Student Affairs, Office of Medical Student Admissions, Office of Graduate Biomedical Education, Office of Graduate Medical Education, Office of Postdoctoral Affairs, and Office of the Registrar
Coordinate with the Precision Education Data Governance Specialist on the collection, management, and access of program evaluation data stored in a data lake environment
Assist the OAE Director with annual program evaluation of the office
Consultation and Service
Provide support to faculty members, staff members, and learners for medical and biomedical education research or scholarly activities, especially as they relate to program evaluation, quantitative and qualitative data analysis
Assist OAE Director with the planning and conduct of professional development activities related to program evaluation
Other Program Evaluation Activities
Assist the Director of OAE and other OAE team members, as appropriate, in facilitating program evaluation-related activities with faculty members, staff members, and learners (e.g., evaluation planning, survey design, focus group moderation, qualitative data analysis)
Master’s degree in Education, Evaluation, Education Psychology, Social Sciences or related field and six years related work experience, or a PhD and Fours year of related work experience.
Experience should include a combination of the following;
Direct experience with data systems and reporting software
Experience in program evaluation activities in an educational setting
Experience in the use of institutional data systems and reporting software
Experience writing scholarly reports
Experience using data to drive decision making and action planning with stakeholders
Special Knowledge, Skills, and Abilities
Demonstrated ability to work independently and as a member of a team
Demonstrated ability to work with a diverse group of stakeholders. Working with learners, faculty, and staff in a health sciences environment preferred.
Demonstrated ability to prioritize tasks, manage time, and meet deadlines effectively
Knowledge of medical and biomedical education programs, and familiarity with the academic medicine environment, preferred
Knowledge of learning management systems, including those that support life-long learning, preferred
Strong organizational skills and keen attention to detail
Strong written and verbal communication skills
Effective use of diplomacy, tact, and judgment
Proficiency with the use of Qualtrics, Microsoft Word, Excel, and PowerPoint
Intermediate to advanced skills in the use of statistical analysis software (e.g., SAS, SPSS)
Knowledge of inferential statistical analyses, including independent and dependent samples t-test, chi-squared test, ANOVA, ANCOVA, Factor Analysis, simple and multiple linear regression, and simple and multiple logistic regression
Strong data visualization skills
Knowledge of Logic Models to build program evaluation systems strongly preferred
Knowledge of standards and policies related to the MSCHE, LCME, ACGME, NIH, or other accrediting bodies and funding agencies, preferred
Supervisory experience preferred
Knowledge of systems related to program evaluation; data management, analysis, and reporting; and survey design and dissemination (e.g., Qualtrics, B-Line, Blackboard, New Innovations)
Classified Title: Monitoring & Evaluation Officer Working Title: Program Evaluation Specialist Role/Level/Range: ACRP/04/MF Starting Salary Range: $70,805 - $97,435 /Commensurate with Experience Employee group: Full Time Schedule: Monday-Friday; 8:30a.m-5:00p.m / 37.5 Hrs/Wk Exempt Status: Exempt Location: 04-MD:School of Medicine Campus Department name: 10003320-SOM Admin Registrar General Admin Personnel area: School of Medicine
The successful candidate(s) for this position will be subject to a pre-employment background check.
If you are interested in applying for employment with The Johns Hopkins University and require special assistance or accommodation during any part of the pre-employment process, please contact the HR Business Services Office at email@example.com. For TTY users, call via Maryland Relay or dial 711.
The following additional provisions may apply depending on which campus you will work. Your recruiter will advise accordingly.
During the Influenza ("the flu") season, as a condition of employment, The Johns Hopkins Institutions require all employees who provide ongoing services to patients or work in patient care or clinical care areas to have an annual influenza vaccination or possess an approved medical or religious exception. Failure to meet this requirement may result in termination of employment.
The pre-employment physical for positions in clinical areas, laboratories, working with research subjects, or involving community contact requires documentation of immune status against Rubella (German measles), Rubeola (Measles), Mumps, Varicella (chickenpox), Hepatitis B and documentation of having received the Tdap (Tetanus, diphtheria, pertussis) vaccination. This may include documentation of having two (2) MMR vaccines; two (2) Varicella vaccines; or antibody status to these diseases from laboratory testing. Blood tests for immunities to these diseases are ordinarily included in the pre-employment physical exam except for those employees who provide results of blood tests or immunization documentation from their own health care providers. Any vaccinations required for these diseases will be given at no cost in our Occupational Health office.
Equal Opportunity Employer Note: Job Postings are updated daily and remain online until filled.
Johns Hopkins University remains committed to its founding principle, that education for all students should be grounded in exploration and discovery. Hopkins students are challenged not just to learn but also to advance learning itself. Critical thinking, problem solving, creativity, and entrepreneurship are all encouraged and nourished in this unique educational environment. After more than 130 years, Johns Hopkins remains a world leader in both teaching and research. Faculty members and their research colleagues at the university's Applied Physics Laboratory have each year since 1979 won Johns Hopkins more federal research and development funding than any other university. The university has nine academic divisions and campuses throughout the Baltimore-Washington area. The Krieger School of Arts and Sciences, the Whiting School of Engineering, the School of Education and the Carey Business School are based at the Homewood campus in northern Baltimore. The schools of Medicine, Public Health, and Nursing share a campus in east Baltimore with The Johns Hopkins Hospital. The Peabody Institute, a leading professional school of music, is located on Mount Vernon Place in downtown Bal...timore. The Paul H. Nitze School of Advanced International Studies is located in Washington's Dupont Circle area.