Data Use

What

Examining multiple types of data for multiple purposes helps high schools and their college partners develop and maintain an informed, effective support system. A data-based approach to education includes

  • assessing student needs in order to select supports;
  • monitoring student progress in order to augment or adjust supports;
  • understanding which students utilize specific supports and how effective these supports are for each student; and
  • evaluating student, teacher, and parent feedback on specific supports.

How

  • Culture

  • Measure

  • Collect

  • Evaulate

  • 4 Domains

  • Tools

  • Reflect

Creating a culture of data use

Schools in the Woodrow Wilson Early College Network are designed to use data to inform the selection and/or implementation of student support strategies, instructional practices, support systems, and curriculum designs. Early college schools also use multiple measures to assess student progress towards college readiness (The Woodrow Wilson Early College Network, 2008).

As a first step toward building a schoolwide culture of data use, the Woodrow Wilson Early College Network recommends that schools and their postsecondary partners create a data plan. This enables faculty and support staff to create standards and guidelines to follow; identify individual and data-team roles and responsibilities; focus on training and professional development; determine timelines and calendars; and evaluate technology (hardware, software, network, tech support). 

With an agreed upon data plan, a school and postsecondary team can begin the data decision-making cycle: (1) identify desired data, (2) find and collect data, (3) analyze and summarize data, (4) create action plan and implement interventions/changes, (5) monitor and evaluate progress, and, finally, (6) start the cycle over again.

Measuring effectiveness

The National Center for Restructuring Education, Schools and Teaching (NCREST) outlines a set of measures of early college success. Though they are tailored to early college schools, these measures can help any school learn to use data to understand how school practices influence students’ college readiness (Barnett et al., 2011).


Measures

Examples

Student Demographics
  • Number of underrepresented students enrolled

Enrollment & Graduation
  • Graduation rates of early college school students compared to students at local high schools
  • Continued enrollment in college immediately after graduating from high school
  • Degree completion of former early college school students 

Coursework
  • Amount of remedial coursework, if any, early college school students take once fully enrolled in college

Student Outcomes
  • Grade-to-grade progression
  • High school and college credit accumulation

Student Behaviors
  • Attendance
  • Referrals for poor behavior

School Design
  • Alignment of school with core principles

Student Attitudes

  • Number of college applications
  • Students’ view of the school’s influence on their academic performance

Teacher and Staff Attitudes
  • Teachers’ views of school climate

NCREST also identified conditions in a school that facilitate data use (Spence, 2006). These include, among others, teachers who are knowledgeable about data management and analysis, principals who lead and support the use of data, and reliance on multiple indicators and types of data to make decisions and monitor outcomes.

To help schools collect and use data effectively, NCREST and Jobs for the Future developed an online Data Use Toolkit. The toolkit has a particular focus on data related to secondary-postsecondary alignment in high schools where college course-taking is a key design feature.

Collecting student feedback

CAL Prep constantly assesses its goals and audience in order to create and maintain effective supports. As the principal states, “Know who you are working for and what you are working towards.” To this end, the principal and staff randomly sample students to interview — both at CAL Prep and in the community — to find out which supports are most important and which are not being provided.

The principal suggests evaluating feedback from students by asking the following questions:

  • Which of these problems ring true for my school?  Which themes continue to arise? 
  • Which of these supports am I already providing?  Which ones just need honing?
  • Which of these supports am I missing?
  • How do these students re-define our problem statement?”

Evaluate effectiveness of supports

University High School of Science and Engineering (UHS) in Hartford, Connecticut has used a range of data to design and evaluate their system of student supports. For example, the school found that nearly 50 percent of their incoming 9th graders were failing two or more classes in their first semester. In response, the school developed a semester-long 9th Grade Academy, which has evolved into a series of interventions targeted to ninth graders. A transition to high school curriculum is delivered to all 9th graders as part of the Advisory curriculum; Advisory provides instruction in study skills, organization, time management, and careers and futures planning. A math lab course doubles students’ time engaged in math learning. And, additional support is provided to students through Saturday Academy, extended day tutoring and National Honor Society peer tutors. The school’s approach cut down the failure rate by 50% in the first year and the two most recent 9th grades only had approximately 5% of the students who were failing after the 9th grade interventions (Vogt, 2008).

UHS also conducted an evaluation study to understand which kinds of students were more likely to do well in concurrent enrollment courses (courses for high school and college credit, taught by high school instructors at the school) and how those courses influenced students’ readiness for college. The evaluation found that students who took a concurrent enrollment course believed they were better able to handle the increased challenge and workload that a college course demands, better able to manage their time, and better able to study.

In addition, the school found that concurrent enrollment instructors were highly supportive. These instructors had high expectations for all students, were willing to give extra help, noticed when students were struggling, and cared about all students in their class. School staff used the evaluation results to determine how they could embed increased supports inside UHS classrooms (Vogt & Cook, 2008).

Four key domains of postsecondary preparation

University High School of Science and Engineering and Science, Technology and Research (STAR) Early College School have made use of data focused specifically on college readiness skills and strategies. These data emerged from the CollegeCareerReady School Diagnostic of the Educational Policy Improvement Center (EPIC). EPIC believes successful college readiness must include deep knowledge and skills in four domains: key cognitive strategies (problem formulation, research, interpretation, communication, and precision and accuracy); key content knowledge (structure, challenge level, value, attribution, and effort); key learning skills and techniques (ownership of learning and learning techniques); and key transition knowledge and skills (postsecondary awareness, postsecondary costs, matriculation information, career awareness, role and identity, and self-advocacy). These four domains can be used as a frame to design an effective system of postsecondary preparation in which each student support links directly to a domain.

EPIC has developed a suite of tools (i.e., ThinkReady, CampusReady, and CollegeCareerReady School Diagnostic) to help students and their teachers, counselors, and parents understand (1) how well the students are doing in relation to the four domains, and (2) whether the schools are offering effective opportunities to learn and acquire relevant knowledge and skills. After collecting data from students, teachers, administrators, and counselors, University High School of Science and Engineering and STAR examined their reports for strengths and weaknesses and through professional development workshops, established action plans for improving the school’s approach to college readiness.

Tools and resources

The Data Use Toolkit from Jobs For the Future and the National Center for Restructuring Education, Schools, and Teaching provides schools with information and tools focused on using student performance data effectively, using data focused on the alignment between secondary and postsecondary teaching and learning, and facilitator tools and resources for providing professional development.

Self-reflection questions

  • To what extent is data use part of your school’s culture? Does your school focus more on data collection or on data use? How can effective data use be maintained/increased?
  • Does your school have targeted goals related to student success? Does your school have key outcomes used to measure progress/success related to these goals?
  • How could your school make use of data from sources other than state assessments (e.g., interviews, focus groups, surveys, benchmark assessments, and non-academic indicators of college readiness)?
  • To what extent do you have access to postsecondary data for graduates from your school? How might you make use of linking secondary and postsecondary data?
  • To what extent do you collaborate with a postsecondary partner on data analysis?

Why

Schools need to regularly monitor students’ college preparation progress (Weinstein, 2011). To maintain well-developed support systems, schools should also constantly evaluate their supports through multiple sources of data — including feedback from students, parents, and teachers as well as assessment data, grades, attendance, state report cards, and federal Adequate Yearly Progress. Some early college schools collect regular student and teacher feedback on how specific supports are working and what other supports are needed (Jaeger & Venezia, 2011). At the student level, the data range from benchmark and interim assessment results, to scores on summative performance series assessments, teacher progress reports, and course grades and GPAs. This assortment of data serves several purposes: course placement, monitoring student progress, and identifying students in need of additional support and/or interventions (Jaeger & Venezia, 2011).

A key recommendation for supporting the effective collection and use of data is to broaden outcomes to include positive development. As Weinstein (2011) states, “A focus on monitoring broader outcomes might improve our engagement of a diversity of families and reduce attrition” (p. 19). Research indicates that non-cognitive variables of student success are important in successfully retaining nontraditional students in college. These non-cognitive variables include: positive self-concept; realistic self-appraisal; ability to understand and deal with racism; preference for long-range goals over short-term of immediate needs; access to strong support person; successful leadership experience; demonstrated community service; and knowledge acquired in a field (Sedlacek, 1993).

Next Principle