More Than Data: What I Learned About Being an Evaluator

-


    Throughout this course, I have significantly expanded my understanding of evaluation, both in technical skills and in my perspective on the evaluator's role. In my self-assessment, I currently rate myself a 4 out of 6 on the evaluator competency scale. I no longer consider myself a beginner; however, I recognize that I am still gaining depth, efficiency, and confidence that come with more real-world experience. I have a solid foundation in evaluation design, data interpretation, and reporting, but I am still learning to apply these skills smoothly across different contexts and stakeholder needs.

    One important conclusion from my self-assessment is that evaluation involves much more than just analyzing data. At the beginning of the course, I thought of evaluation as mainly a technical process, involving data collection, analysis, and results presentation. Now, I see that evaluation is closely linked to decision-making, communication, and stakeholder context. Understanding findings, anticipating misinterpretation, and adapting communication for different audiences are just as crucial as the analysis itself. This shift in perspective has been one of the most valuable lessons from the course.

    I noticed significant growth in two competencies: selecting evaluation approaches and theories appropriately and clarifying diverse perspectives, stakeholder interests, and cultural assumptions. For selecting evaluation approaches and theories, I initially rated myself a 2, indicating a limited grasp of how to choose among various evaluation designs. I was not aware of when a one-shot design, mixed-methods approach, or other frameworks might be suitable. Through this course, especially while developing our final evaluation report, I started to understand that evaluation design depends on context, constraints, and purpose. For instance, realizing why a one-shot, mixed-methods design was appropriate for the UN CC: Learn course helped me view evaluation as a strategic process instead of a fixed formula. This growth is why I now rate myself a 4 in this area. I am not yet an expert, but I feel confident making informed decisions about evaluation design and backing them up.

    My second competency, clarifying diverse perspectives, stakeholder interests, and cultural assumptions, showed even more growth, moving from a 1 to a 4. At the start of the course, I did not fully understand how complex stakeholder dynamics can be or how much they affect evaluation outcomes. I tended to see evaluation results as objective and universal. However, through our work on reporting, communication strategies, and interpretation, I realized that different stakeholders can interpret findings very differently. For example, a program designer might focus on improving course structure, while a facilitator might emphasize supporting learners in real-time. Cultural and professional contexts can also shape how learners experience a course, which became clear in our analysis of the UN CC: Learn program. My growth in this area came from both the coursework and the repeated emphasis on customizing communication, avoiding misinterpretation, and understanding audience needs. Now, I feel much more prepared to consider multiple perspectives and communicate findings in a way that is meaningful and actionable for different stakeholders.

    One competency that surprised me as necessary is the ability to anticipate and prevent misinterpretation of data. Before this course, I believed that presenting accurate data would suffice. I learned that stakeholders can easily misinterpret findings, especially when it comes to averages, distributions, or correlations. For example, a mean score might suggest a neutral response, but without examining the distribution, it can obscure a highly polarized set of responses. This realization changed how I think about reporting. It is not enough to present data; evaluators must actively guide interpretation to ensure that findings are correctly understood and properly used. This competency stood out to me because it links technical analysis with communication and decision-making, reinforcing that evaluation is both a science and a practice.

    In terms of strengths, I have developed a strong ability to analyze data and connect it to meaningful claims. I feel confident working with descriptive statistics, spotting patterns in frequency distributions, and using evidence to support conclusions. Additionally, I have improved in writing clear, structured reports that connect findings with evaluation questions and recommendations. I can also think critically about limitations, such as self-report bias and sample size, and incorporate those limitations into my interpretation. These skills enhance the credibility of my evaluation work.

    However, I still see clear areas for growth. One area involves gaining more experience with advanced evaluation methods and tools, including more complex statistical analysis and software beyond basic descriptive statistics. While I am comfortable using JASP for frequency distributions and measures of central tendency, I want to become more confident with advanced analyses. Another area for growth is developing stronger skills in facilitating stakeholder discussions. Although I understand the importance of stakeholder engagement, I have limited experience leading conversations where findings are debated and decisions are made. This is an area where real-world experience will be especially important.

    Based on my self-assessment, there are several actions I can take after this course to continue growing as an evaluator. First, I plan to look for opportunities to apply evaluation skills in real-world settings, whether through my current role in education or in future positions in learning design or program evaluation. Practical experience will help me move from understanding concepts to applying them effectively. Second, I want to keep building my technical skills by exploring additional data analysis tools and methods, which will boost my ability to handle more complex evaluations. Third, I plan to continue engaging with evaluation frameworks and literature, like those discussed in this course, to deepen my understanding of best practices. Finally, I want to further develop my communication skills by practicing how to present findings in various formats, such as presentations, reports, and discussions, to better support stakeholder decision-making.

    Overall, this course has significantly changed how I view evaluation and my role in it. I started with limited knowledge and a narrow perspective, and I am leaving with a much stronger foundation and a clearer understanding of the complexities involved. While I still have areas to grow, I feel confident in my ability to contribute meaningfully to evaluation work and continue developing my skills moving forward.

Competencies Rated as a Relative Strength 

Competency

Start of Course

End of Course

Notes

I am able to reflect on evaluation formally or informally to improve practice.


5

6

Before the course: I can do this thanks to being an educator. On a scale of 1-6, 6 being an expert, I would say I am a 5.


I am able to design credible and feasible evaluations that address identified purposes and questions.

5

6

Before the course: Since I am an educator, I can look to see what students need to be tested on and what students need targeted instruction for. On a scale of 1-6, 6 being an expert, I would say I am a 5.




Competencies Rated as a Growth Area 

Competency

Start of Course

End of Course

Notes

I am able to select evaluation approaches and theories appropriately.

2

4

Before the course: I am not fluent in this skill yet as I am not familiar with the approaches of evaluation. On a scale of 1-6, 6 being an expert, I would say I am a 2.


After the course: Can now justify appropriate evaluation designs (e.g., mixed-methods, one-shot) based on purpose, context, and constraints.

I can clarify diverse perspectives, stakeholder interests, and cultural assumptions.

1

4

Before the course: I am unable to vouch for such broad and diverse data. The data I can attest for is what standards students are proficient in, need help with, or fall far behind in. On a scale of 1-6, 6 being an expert, I would say I am a 1.


After the course: More confident identifying stakeholder needs and adjusting interpretation/communication to different audiences and contexts.

Comments

Post a Comment

Popular posts from this blog

Reflections from the Late Majority

-
When I reflect on my relationship with new technologies, I see that Everett Rogers' Diffusion of Innovations theory accurately describes me as a learning design and technologies practitioner. This model groups adopters into five categories: innovators, early adopters, early majority, late majority, and laggards. Each group is defined by how quickly they accept new ideas and tools. As I assess my own habits, experiences, and patterns with technology, I realize I relate most to the Late Majority. Where I Fall on the Curve The Late Majority consists of individuals who are careful and deliberate when adopting new innovations. They usually look for validation from peers, training opportunities, and solid proof that a tool or method works before committing their time and effort. This resonates with me, especially in my current job where my workload is heavy. I often teach six or seven different subjects simultaneously, which doesn’t leave much time for exploring new technologies or teach...
Read more

Looking Beyond Training: How Needs Assessment Changed My Perspective as a Learning Designer

-
     When I first started this course, I thought of needs assessment mostly as the first step before developing training. I believed that if an organization asked for training, my job as a learning designer was to find the best way to create and deliver it. Looking back, I see that I was approaching problems with a solution-first mindset. I assumed that because someone requested training, it had to be the answer. However, throughout this course, I learned that effective learning designers do much more than create instruction; they examine performance issues before suggesting any solutions. My thinking has shifted from asking, "What training should I build?" to asking, "What problem am I really trying to solve?" This change in perspective has become one of the most valuable lessons I will take into my future career.      One major turning point for me was the EduCore Dynamics case at the start of the course. Leadership thought poor customer service stemmed f...
Read more

Reflection on Evaluative Comptencies

-
As an educator, I have developed strong foundational evaluation skills through my daily teaching. I would give myself a 5 out of 6 for both reflecting on evaluation to improve practice and designing credible, practical evaluations. These skills have grown through continuous work with student data, making instructional decisions, and adjusting learning experiences for a variety of learners. In my classroom, evaluation is not a separate task; it is an ongoing process integrated into daily instruction, lesson planning, and reflection. While I am confident in these areas, my evaluation experience comes mostly from classroom practice rather than from formal evaluation frameworks or theory. One of my key strengths is my ability to reflect on evaluations, both formal and informal, to improve my teaching. During a project-based writing unit, I noticed common patterns in student work that showed gaps in creating strong claims and using textual evidence effectively. Instead of focusing on each s...
Read more