Congratulations on Getting Your 21st CCLC Proposals Submitted!

ActKnowledge provided free consulting and technical assistance on the evaluation component to over a dozen organizations.

We hope you all get funded!

Youth and Education

This is a topic inwhich ActKnowledge has developed particular expertise.  Because of the number of youth and education projects we have, ActKnowledge staff have become active members of networks of practitioners and researchers, have published and presented at numerous conferences in the fields of after-school, community schools, and youth development.  

Some of our work is with small youth organizations, to assist building their capacity for planning and self-evaluation.  In other cases, we are evaluators over several years for large organizations, such as The Children's Aid Society and the YMCA.

Sample Projects:

Girl Scouts of the USA, New York City

ActKnowledge worked with the Girl Scouts of the USA (GSUSA) to facilitate the development of a clear and usable Theory of Change that would represent the Girl Scouts program model and connect this roadmap for progress with the rest of GSUSA and the work of local councils.  A key benefit of this project for the Girl Scouts was specifying the core beliefs and goals that apply nationally, while providing a flexible framework for the local councils to use and develop their own approaches without sacrificing consistency with the program model.  ActKnowledge developed a basic framework and then a complete Theory of Change for the Girl Scouts USA program model.  The GSUSA National Meeting held discussion groups on the Theory of change in order to link it to the work of local Councils.   The final Theory included concrete strategies for GSUSA to use in program and council planning, evaluation, dissemination, and decision-making.  ActKnowledge linked the program model Theory of Change to the rest of GSUSA by identifying overlaps between this roadmap and other areas of GSUSA, specifically the work of the Volunteer Gap team, and planning the process for developing those linkages.

Anchor, New York City

Anchor is a residential boarding program for high school students.  ActKnowledge began this evaluation project by developing a Theory of Change with Anchor to serve as an evaluation framework for its boarding program at two Brooklyn high schools.  ActKnowledge developed a longitudinal, multi-phase evaluation design to be implemented over two years because the Anchor program is setting out to achieve long-term outcomes with the instructional enrichment and structured supportive environments of the boarding schools.  Designing the evaluation in this manner allowed Anchor to examine not only those long-term outcomes, but also link them with more intermediate and short-term outcomes, which helped to keep the program on track.  The first year of the evaluation process included development of an initial evaluation framework informed by Anchor stakeholders and staff, high school staff, and a focus group with youth; development of instruments and data collection mechanisms and planning; piloting of instruments and testing their reliability and validity, and orienting schools to the instruments and data collection framework; and data analysis and presentation of the first year report to Anchor’s stakeholders.  The second year of the evaluation involved refining the evaluation framework, instruments, and procedures, determining additional outcomes to include, and a similar data collection and analysis process resulting in a final report at the end of the second year. 

Partnership for Hawaii’s Keiki.  Good Beginnings Alliance, Hawaii

The Partnership for Hawaii’s Keiki was a statewide initiative to improve the health, safety, and school-readiness of young children.   The initiative combined policy change at the state level with local change at the community level.  Based on the assumption that community building is critical to improving outcomes for children, Good Beginnings selected six communities across the state to develop and implement local plans to improve outcomes for children and families. ActKnowledge led the community capacity building effort during the planning phase and conducted evaluation of the initiative’s early progress.  Local facilitators, technical assistance mentors, Good Beginnings staff, and other consultants worked together in a participatory manner on this evaluation in order to build local and organizational capacity.  ActKnowledge engaged with the Good Beginnings Alliance and local community partners in developing Theories of Change relevant to the community partners’ efforts to bring about change in their communities, in the counties, within the Good Beginnings Alliance, and at the state level.  Our evaluation was designed with the goal of supporting the use of these Theories of Change by community partners.  Because a major focus of the Partnership for Hawaii’s Keiki is capacity-building, ActKnowledge’s evaluation focused on how and what types of capacity were developed and strengthened and the needs to sustain it; the relationship between increased capacity and improved outcomes for children and their families, and the kinds of capacity necessary to accomplish different goals of the Partnership for Hawaii’s Keiki.

YMCA Asset Labs, New York City

ActKnowledge is currently conducting the outcome and implementation evaluations for the New York City Asset Lab initiative spearheaded by the YMCA of Greater New York.  The Asset Lab initiative uses a youth development framework to provide technical assistance to urban community-based organizations and schools serving a diverse array of NYC youth.  We used the developmental assets profile (DAP) to conduct pre- and post-tests at each of the eighteen sites to evaluate outcomes.  Youth self-report measures were used in conjunction with data related to program participation, standardized tests, and teacher/staff checklists.  Additionally, ActKnowledge conducted focus groups with youth and staff, and interviews with key individuals, as part of an implementation evaluation at ten of the sites. Data from the implementation evaluation were used to create individual site reports for each Asset Lab.  The evaluation from this three-year pilot phase will serve to create improved models of support for asset labs and secure further funding to support positive youth development in New York City.

Citizens for NYC, New York

ActKnowledge used a qualitative, case study approach to document 21st Century Community Learning Center programs at high schools that partnered with the Young Citizens Center during 2004-2005 school year and the 2005-2006 school year. The programs employ a service learning model that emphasizes youth empowerment.  ActKnowledge’s approach to this evaluation aimed to capture in-depth program processes and experiences for everyone involved, employing primarily interviews and focus groups with teachers, community-based organizations, youth participants, and parents who were involved with the program. The evaluation of the 2004-2005 year was conducted in summer of 2005 and included phone interviews with teachers and community-based organizations, and a focus group with youth participants. In 2005-2006, we conducted focus groups with staff, teachers, and parents, plus three interviews per site with a group of youth participants.  In keeping with the program model of youth empowerment through service learning, ActKnowledge incorporated students’ work into the case studies in and asked them to reflect on their growth during the program year.

The Writers Room, Montclair, NJ

ActKnowledge is conducting a year-long study of The Writers’ Room™ Program in the Montclair public schools.  This study uses a mixed-method and theory-based approach, which ActKnowledge has used in a number of its education evaluation research projects. This kind of approach is particularly important in a case like this one, where a traditional experimental design (i.e., using a control group) is not possible because the program has been in place for many years and is fully integrated throughout schools in the district.  This research project includes interviews with teachers, students, and program staff; surveys with middle school and high school students; surveys with teachers at all grade levels that work with The Writers’ Room™ Program; comparisons of language arts test scores by level of involvement with the program; and content analyses of writing samples before and after feedback from Writers’ Room™ coaches. These will help us to answer questions about the effects of the program on student writing, test scores, teacher practices, and teacher and student perceptions of writing and how it develops.

National Foundation for Teaching Entrepreneurship, New York City

NFTE is a foundation which provides entrepreneurship education to low-income young people to enhance their economic productivity.  ActKnowledge worked with the Network for Teaching Entrepreneurship (NFTE to enhance their economic productivity, to clarify their program theory and articulate a Theory of Change regarding how change is brought about through curricula, training, and other factors.  This project also brought to the forefront contextual differences and differing viewpoints that exist at the multiple NFTE program sites.  Our project with NFTE, which has a long-term evaluation underway looking at educational and psycho-social outcomes for youth, focused on crafting a Theory of Change that would facilitate decisions about interventions, strategies, and activities.  NFTE used their TOC to assist in carrying out a focused evaluation that is more likely to capture achievement through establishing a common ground on NFTE’s beliefs and assumptions.  The Theory of Change process helped link NFTE’s five-year evaluation, strategic planning process, and future directions into a large, well-articulated whole, providing them with a framework for decision-making and ongoing learning.

Early to Learn, Philadelphia, PA

ActKnowledge worked with the core team of program planners and evaluators of Early to Learn to develop a Theory of Change and design an evaluation for this United Way of Southeast Pennsylvania initiative.  Early to Learn promotes school readiness for children under the age of five by providing services that focus on care, education, and healthy development for children and support for their families, so that children start school with the skills they need to successfully complete their education.  On this project ActKnowledge focused on building capacity for Early to Learn’s core team in Theory of Change, so that they would have the ability to revise, update, and use their theory throughout the Early to Learn initiative.  To that end, ActKnowledge staff trained Early to Learn’s program and evaluation staff on key Theory of Change concepts and facilitated the development of their own theory.  ActKnowledge also assisted in the use of the theory crafted by Early to Learn core staff to design an evaluation of outcomes of the initiative.  The emphasis on capacity-building in this project was critical to making our work with Early to Learnuseful, as well as to the development of their Theory of Change.

Fordham Regional Educational Technology Center, New York City

ActKnowledge developed and implemented this participatory evaluation of the R2DII professional development initiative of Fordham University’s Regional Education Technology Center (RETC) and Region 2 of the New York City public school system.  The initiative was designed to improve student learning and achievement by changing teacher practice for technology-integrated schools and classrooms.  The R2DII evaluation was designed to demonstrate the extent to which the project achieved its goal to improve student achievement in English language arts and mathematics, and to determine whether the project met its goals to increase students’ technology literacy and promote technology integration in classrooms.  The evaluation included a matched-comparison group design to examine both student achievement and the link between implementation and student outcomes.  It also incorporated qualitative and quantitative methodologies in order to provide rich, descriptive information that explain and illustrate findings.  The project provided short-term feedback on the progress made throughout the course of the project; a description of how the project was implemented and its impact on participating teachers and their students; informed future professional development efforts in instructional technology for Region 2 and the New York City School System; and provide lessons and examples for dissemination beyond New York City.  To measure progress against the project’s goals, this two-year, multi-phase evaluation utilized data sources such as site visits and observations in schools and classrooms, professional development observations, teacher surveys, student performance data, and review of key documents such as RETC curriculum and training materials and teachers’ lesson plans and assignments. 

Highbridge Community Life Center, Bronx, NY

Get and Give, a program of Highbridge Community Life Center serving youth ages 5 through 18 in the Highbridge section of the Bronx, is supported by 21st Century Community Learning Centers funding.  Prior to implementing evaluation measures, ActKnowledge worked with program staff develop a Theory of Change for the program and to select measures appropriate to outcomes identified in the theory.  ActKnowledge also helped refine a youth survey that program staff had already begun to develop.  Program staff administered the youth survey in the fall and spring of the program years 2004-2005 and 2005-2006, and ActKnowledge is analyzing survey results to look for improvements in youth outcomes targeted by the 21st Century Community Learning Centers.  In addition, we conducted focus groups and interviews in the winter and spring of 2006 with staff, parents, and program participants.  Finally, Get and Give administered the Program Quality Self-Assessment Tool, a measure of success in program implementation areas as well as a tool to plan for continuous improvement, to staff and parents, and ActKnowledge is analyzing responses. 

Young Citizen’s Centers, Citizens for NYC- New York, NY

ActKnowledge is using a qualitative, case study approach to document 21st Century Community Learning Center programs at high schools that partnered with the Young Citizens Center during 2004-2005 school year and the 2005-2006 school year. The programs employ a service learning model, which emphasizes youth empowerment.  ActKnowledge’s approach to this evaluation aims to capture in-depth program processes and experiences for everyone involved, employing primarily interviews and focus groups with teachers, community-based organizations, youth participants, and parents who are involved with the program. The evaluation of the 2004-2005 year was conducted in summer of 2005 and included phone interviews with teachers and community-based organizations, and a focus group with youth participants. In 2005-2006, we are conducting focus groups with staff, teachers, and parents, in addition to three interviews at each site with a group of youth participants.  In keeping with the program model of youth empowerment through service learning, ActKnowledge is also incorporating students’ work into the case studies in and asking them to reflect on their growth during the program year (i.e., some students who we have interviewed during the year will be asked to put more time into helping us to interpret their interviews from during the year and use their journals and portfolios to construct case studies of themselves).