A Pennsylvania School Profile Primer
Recently, the Pennsylvania Department of Education submitted a request to the United States Department of Education for a waiver from future NCLB requirements. To meet waiver request requirements, the Pennsylvania Department of Education has created a system of rating schools called the “Pennsylvania School Profile”. Originally scheduled for release in October of 2012, the Pennsylvania School Profile release has been delayed twice due to problems with inaccurate and erroneous data. It is set to be released during the 2013/2014 school year. The purpose of the PA School Profile is to synthesize dozens of academic data points for each building in a school district and assign it a single numerical grade. Each school district in Pennsylvania will receive an independent score for every school in their district.
There will be three areas of data that will be used to determine a school building’s final score. The far majority of all data will be comprised of only four tests. The four main tests that will be used are the PSSA Reading, Science, Math, and Writing assessments. Depending on the grade levels and composition of each school building, the data used to create the final school profile scores will be weighted differently. The three categories that will be used to create a school’s profile will be; “Academic Achievement”, “Academic Growth” and “Other Academic Indicators”.
“Academic Achievement”, which measures how many students pass multiple tests, will carry fifty percent of a school’s final score. This section may include, depending on the buildings grade structure, PSSA Science, Reading, Math and Writing scores. For high schools, Advanced Placement scores, NOCTI (trade certification exams taken by students that are enrolled in Career Technical Centers) results and SAT scores will be included. Unfortunately, the playing field is not level in all of these categories. For example, many small school districts do not have the luxury to offer a large catalog of AP courses. With fewer AP courses offered, fewer students will have the potential to pass the exams. Another concern that is present with both the AP and SAT exams is the dependence on parents to pay for the exams. Some districts are fortunate enough to be able to pay for students to take the AP Exams while others do not have the resources to do so and will need to rely on families to assume this cost.
“Academic Growth” for each building will count for forty percent of the buildings final score. The Pennsylvania Department of Education has hired a company from North Carolina, EVAAS, to perform the statistical analysis of state assessment scores to determine the buildings educational effect on student growth. Although an oversimplification of the process, the system is designed to compare current student results against previous results. For example, if a student were to receive a B on a fifth grade exam, they would then be expected to achieve a B on the sixth grade exam. If this is achieved, the student would be viewed as having achieved one year of academic growth. Buildings will receive points towards their final school profile score based on the aggregate growth of all students. The stronger the statistical data points towards advanced student growth, the more points a school building will receive.
The final category, “Other Growth Indicators”, will contain the final ten percent of a school buildings total score. Included in this category are: student attendance rates, cohort graduation rates and PSAT participation. Because of the weighting of the criteria in this category, high schools will be at a disadvantage compared to elementary schools.
Scores from Academic Achievement, Academic Growth and Other Growth Indicators will be combined to create a final score that will range from 50 to 100 total points.
One should note that what is not included in the Pennsylvania School Profile Score is the quality of district music programs, arts programs, athletic programs, technology programs, class size, per pupil expenditure, and district financial health. All of these are important facts that should not be ignored.
Across the Commonwealth, districts that spend $17,000 per student will be compared to those that spend $8,500. And districts that offer world class technology and arts classes will receive no recognition for doing so.
If the state desires to create a system to score and rate school districts in Pennsylvania, it should be a complete and well thought out system that includes all factors. Our kids and communities deserve it.
Value Added TeacherEvaluations: The Right Road or the Wrong Path?
One of the major educational initiatives currently being proposedby the Pennsylvania Department of Education and Governor Corbett is a teacherevaluation system based on data. The current teacher evaluation system involvesa process of rating teachers as either satisfactory or unsatisfactory based ondata collected by the building administration team. Depending on the schooldistrict, various forms of data are used to make this determination. The formsof data that are most prevalently collected include information from directobservation in the teacher's classroom, professional portfolios, achievement ofprofessional goals and participation in differentiated supervision models. Fundamentally this process has remainedunchanged for many years.
The evaluation system currently being proposed inPennsylvania would be a combination of direct classroom observation and teacher“value-added” scores. In a “value-addedmethod” of evaluating teachers, a mathematical analysis is performed on variousstudent assessments to determine the “educational value” or effect a teacherhas on his or her specific students. Thevalue added models of evaluating teachers for effectiveness are currently only inuse in a few states across the nation but, are quickly gaining popularity.Currently there are value-added models of teacher effectiveness in Tennessee, Ohioand New York. The Pittsburgh School District is in the first year of using a“value added model” to evaluate their teaching staff. The Governor has committed approximately $31million dollars of Federal “Race to the Top” grant money as well as $3.7million dollars of state revenue to this initiative.
The proposed new teacher and principal evaluation systemwill be comprised of two components. The first half of a teacher’s evaluationwill be based on data directly gathered from building administrators throughthe observation process. The State ofPennsylvania has chosen the Charlotte Danielson “Framework for Teaching” as thetool to gather observational data to complete the observation process. Thisteaching framework was first introduced in the late 90’s and has beenrecognized as a valuable tool to assist teachers in improving their instruction.This framework is based on the concept that teacher observation process shouldbe divided into four specific domains. The four domains of effectiveinstruction include:
1. Planning and Preparation
2. The Classroom Environment
4. Professional Responsibilities
Within the four domains are twenty-two teaching competenciesthat will be used to evaluate teachers. Each competency has four levels ofeffectiveness to create a rubric for observation.
The second half of a teacher or principal evaluation will bebased specifically on student data. Thedata used would be a combination of building level data, classroom data fromindividual teachers on various state assessments and data gathered by thedistrict from individual local assessments.
Building level data would include factors such asattendance, graduation rate, academic growth of subgroup population and overallstudent success on state and national assessments. Teacher specific data wouldinclude only the assessment results from the students that have direct instructionalcontact with that teacher. And finally, districts would be allowed to determinewhat additional assessments they would also like to include in the evaluationprocess.
Although on the surface this proposal may seem simple andstraightforward, research tells us that it is actually a very complicatedprocess. In fact research tells us thatthe value added method is fraught with inaccuracies. The first concern is theoversimplification of the teaching process. The belief that by measuring thegains of the students of a specific teacher would reflect that teachers “effectiveness”is not documented by research. The useof the value added method presupposes that student growth can be measured byspecific tests, is influenced only by the teacher, and is totally unrelated tothe classroom context. None of thesepresuppositions actually exist in practice. According to a recent article in PhiDelta Kappan, other factors that have been identified to affect student growthinclude: class-size, home supports, community supports, peer culture, priorteachers, and varying rates of summer learning loss. In fact, a teacher’s value added rating ismore likely to be affected by the teachers a student had in previous years thanby what their current teacher may or may not do in the classroom.
Another concern with the value-added model is a lack ofconsistency in results. A study done by the Bill and Melinda Gates Foundationsubstantiated that by varying the test used within the same subject area,significant differences in teacher effectiveness ratings were produced.
One of the fundamental flaws of thevalue added model involves the theory of randomness. A teachers relativecontribution to student learning is equivalent to making a causalinterpretation of a statistical estimate. (Braun) These relationships areaccurate most often when students are randomly assigned to classes. Forexample, in the medical field, random testing of new drug products is theindustry-standard. In this type of testing, groups of patients are randomlyassigned to a limited number of treatment programs. The optional treatmentprograms are well defined and patients are dispersed equally among thetreatment plans. The whole of the datais then studied to determine which medical treatment was most effective. Inschools, the placing of students at random into teachers classrooms is not thenorm. Very often students are placed into specific programs that may lead toconsequential tracking. For example, ateacher that may provide instruction in Advanced Placement Calculus would neverbe able to make the assertion that the students were random students, randomlyassigned to their classroom. In thepredominant number of teacher value-added models, the most inaccurate resultsare typically the results placing teachers in the highest level ofeffectiveness and the results placing teachers in the lowest level ofeffectiveness. This, combined with alack of random student placement, creates the potential of teachers beingrewarded for effectiveness they have not achieved or punished for a lack ofeffectiveness that is simply not true.
Furthermore, a study done in 2008 bySass titled, “The Stability of Value-Added Measures of Teacher Compensation Policy” and quoted by Phi Delta Kappan determined,“After controlling for prior student testscores and student characteristics, the study still found significantcorrelations between teacher ratings and students race/ethnicity, income,language background, and parent education.”
As the educational community continuesto move down the path of creating a better system of teacher and principalevaluation, we must not turn our backs on current educational research insearch of a quick fix. Our students andteachers deserve better.
Anderson, J. (February 19, 2012) States Try to Fix Quirks in TeacherEvaluations.
The New York Times. Retrieved from
Braun, H. (2005). Using StudentProgress to Evaluate Teachers: A
Primer on Value AddedModels. ETS Policy Information Center,
, 2-16. Retrieved fromhttp://www.isbe.net/peac/pdf/primer.pdf
Darling-Hammond, L.,Amrein-Beardsley, A., Haertel, E., Rothstein, J.
(212). Evaluating TeacherEvaluation. Phi Delta Kappan,
Many states are quickly jumping onto the “teacher evaluation by data” bandwagon. The ultimate reality may be that we are compelled into a system that has not been validated as being fair and more importantly, a system that has not been vetted as being beneficial to the students that we serve. New York City recently released the teacher effectiveness ratings for the teachers in their district. Below is a link to an article that explains many of the issues that New York is facing. For example, some teachers were rated on as few as 10 students’ performance. Another area of concern is the wide margin of error. On average, a teacher’s math score could be 35 percentage points off, or 53 points on the English exam.
Another article that appeared in the New York Times responding to the New York City Evaluation system reports the opinion of Diane Ravitch. Ms. Ravitch was one of the authors of “No Child Left Behind” and for much of her career has been a staunch supporter of student assessment, the AYP process and sanctions based on test results. She has now changed her stance and believes that the current educational process is in fact harming students. Her comments on the New York City Evaluation process are contained at the link below.
I hope you will read these articles and share your opinion.
It strikes me that in the nation that values education; we would place so little value on the education of educators. Alternative certification programs are not something new to the educational horizon. Research shows that a higher percentage of teachers that achieve certification through alternative paths leave the occupation than teachers trained through traditional paths.
What are your thoughts about the position that Bill Gates takes in his article "Grading the Teachers?" Do you believe that the field of education knows very little about effective teaching? Do you believe, as the responders to his survey believe, that tenure is granted too early and too easily? Do you believe that our current evaluation system is a complete fraud? How can we improve the current system? I am anxious to hear your thoughts!
The Pennsylvania Senate recently voted to approve Senate Bill 1 which includes vouchers, called “Opportunity Scholarships”, expanding the EITC program and reforming the Charter School Amendment. Below is a synopsis of the elements of each of these areas.
Opportunity Scholarships (Vouchers)
Following very closely to the Cleveland Public School Voucher Program which has already passed the federal Constitutionality challenge, the Pennsylvania Opportunity Scholarship Program will focus on a few key issues to also meet Federal Constitutionality. First, the vouchers will be able to be used for public as well as private schools and will target low income students in schools performing in the lowest five percent in the state (in the beginning years of the program). This legislation has expanded student eligibility from previously discussed versions by increasing the poverty threshold from a family of four with an income of $29,055 to an income of $41,348.00. To avoid conflict with the establishment clause, voucher checks will be made out to parents/guardians. An additional feature of the legislation is that if parents fail to endorse the check they face a fine of 300% the value of the voucher. Districts will be able to decide if they would like to accept students with Opportunity Scholarships. Using Opportunity Scholarships for the purpose of athletic recruiting is prohibited.
This program allows businesses to make a donation to a voucher scholarship program instead of paying state income taxes. By 2014-2015, there is a potential of 125 Million dollars of Pennsylvania Tax revenue being diverted to EITC scholarship funds.
Charter School Reform
Although we often think of reform as a way to improve a current system, this reform actually expands the potential to create charter schools and decreases the requirements to create a cyber school. One of the major changes involves school districts and the ability of a school district to convert a building in their district into a charter school. In past legislation, the conversion of a building to a charter school required consensus of parents, staff and school board. The current legislation requires only school board approval. How might this change be exploited? School Boards would have the leverage to convert a school at any time. When the conversion takes place, the school is then run by a private corporation and one would assume no longer under the collective bargaining agreement.
Do vouchers have a track record of working? Cleveland actually has one of the longest running voucher programs in existence. This program has been providing vouchers since 1995. What are the results? According to the national School Board Association:
“Research has indicated no overall differences in the academic achievement of public school students and voucher students, though recent evaluations have indicated that public school students made larger test score gains than voucher students who started out ahead of their public school counterparts. The research also has shown that African-American students and low-income students are underrepresented in the program compared to the city's public school population. Despite no credible evidence of academic achievement because of vouchers in Cleveland, the Ohio Legislature, in 2005, approved a statewide voucher program (EdChoice) that began in the 2006-07 school year.”
For those that may believe that vouchers are about helping kids escape “failing schools” I believe the information above shows vouchers have very little to do with helping kids. And if you believe that vouchers are for only “poor” students in Pennsylvania, you would be correct…for now.
As the debate moves to the House, please make your opinion known or live with the consequences.
If you would like additional information, please feel free to contact me.