Testing concerns call for disclosure of tests. NYS interests in “test integrity” runs afoul of sound education policy

transparency

New York State’s Testing Program is an epic fail.

2 years ago NYSEDs reputation was tarnished as a result of the Pineapple Hare debacle. http://www.nytimes.com/interactive/2012/04/20/nyregion/21pineapple-document.html

Last year, students test scores were exorbitantly low resulting in fear, upset and sheer panic among students, teachers and administrators State wide. http://www.nytimes.com/2013/08/08/nyregion/under-new-standards-students-see-sharp-decline-in-test-scores.html?pagewanted=all

The results were sobering.  Fewer than one-third of students in the entire state were “proficient,” according to the new assessments.  The results in traditionally underserved communities were even lower. Even Commissioner King admitted that students had “struggled” with the test. ehttp://www.engageny.org/sites/default/files/resource/attachments/test_score_release_letter_to_parents.pdf

It should be noted, that although King acknowledged student’s struggled, he could not prove or show to a reasonable degree of certainty why students had done poorly on the exam. Indeed, King’s explanation was conclusory, and lacked conviction because the tests were not released so the public had no way of knowing whether mistakes were the student’s fault or due to errors and/or deficiencies contained within the test itself.

The 2014 testing season has not been exempt from drama either. The ELAs were wrought with irregularities and problems. Math? Questions were missing, pages stuck together and NYS 5th grade Math Assessment Reference Chart had error in conversion of Liters to Cubic Centimeter among other things. http://lohud.us/1jm9AYr 

In fact, things have been so bad since CC became a reality in the past 3 years that NYSED released a script of “talking points” a veritable guide for parents to prepare their children for these top secret, ridiculous standardized tests. http://www.engageny.org/resource/families-to-students

You may ask yourself how do these tests even measure college and career readiness?

Well, simply put, they don’t.

Parents and educators are starting to catch on.

Hence, students are refusing to take the tests and some educators are refusing to administer them. http://www.washingtonpost.com/blogs/answer-sheet/wp/2014/04/04/teachers-refuse-to-administer-standardized-tests/

Many districts saw major jump in refusals from ELA to Math. http://lohud.us/1kuJbXQ 

But are students struggling because the test standards have improved, they are valid and the test is of better quality or are the test questions failing students because they are simply malignant in and of themselves?

The public has no way of knowing the truth either way because NYSED refuses to disclose the test questions not even after the exam has been administered and/or scored.

Why?

The nefarious Pineapple Hare gaffe certainly put NYSED on notice of Q&A concerns – the question was deemed ambiguous by NYSEDs suits after the fact and ultimately that question was invalidated from the test and was retired.

But, isn’t it fair to say that incidents like Pineapple Hare are reasonably forseeable and could be prevented if NYSED released tests so that they could be examined for Q&A purposes?

NYSEDs lack of transparency, among other things, has caused a great deal of unnecessary stress and anguish to NYS students and teachers. This is unacceptable.

For the 3rd time this year (I have lost count, to be honest) parent leaders from across the State have been put in the untenable position of having to call for the resignation of Commissioner King due to serious gaffes on his watch. http://www.nysape.org/nys-3rd-grade-test-invalid-chancellor-merryl-tisch–education-commissioner-john-king-must-resign.html

The request that King resign is with good reason, rest assured. But, I digress.

I recently composed a piece entitled: Opt In Parents Permitted to Examine Their Child’s Response to Certain Test Questions and May Be Able to Copy the Same http://wp.me/p44iDJ-ET

The issue is that CCSS does not amount to a developmentally or educationally sound policy. The testing regimen tied to student performance and teacher evaluations is unduly burdensome, not a valid indicator of student achievement/outcome, not a sound way to evaluate teachers and does not carry enough reliability to justify NYS continued investment of money, time and resources.

NYSED is quick to advance threats of sanctions and rhetoric to protect “test integrity”, but NYSED slow in being held accountable for the devastating effects that the NYS testing program is having on students and teachers across the State.

The test content is not disclosed and there is no quality assurance mechanism to assure that the material is appropriate, valid, reliable and/or proper which magnifies an already contentious problem.

NYSEDs test development process may look good on paper but in reality the  process is rife with confusion, weak links and there is virtually no quality control. http://www.p12.nysed.gov/assessment/teacher/test-development-process.pdf

The bottom line is that students and teachers are getting hurt, money is being wasted, precious time is being spent on testing at the expense of all other instruction and the system surrounding the NYS testing program is dysfunctional to say the least.

Despite the same: “We are not retreating. New York is moving forward with a common belief in the power of great teachers to make a difference in the lives of children,” said Commissioner John B. King, Jr. in a policy address at the Wagner School, New York University. http://www.engageny.org/resource/commissioner-king-were-poised-to-lead-the-country

CCSS is a mess, HST is out of control and student privacy concerns have taken priority and have dominated advocacy, legislators and State calendars throughout the entire year.

Parent-educator groups have shown great perseverence and strength in leading efforts to challenge these flawed policies on behalf of students and teachers.

On the other hand, the only thing Commissioner King has demonstrated he is capable of leading is a circus!

Even education researchers feel that NYSED has been less than transparent regarding testing and assessments. http://www.edvistas.com/sites/default/files/13/11/Testing%20and%20the%20Common%20Core%20-%20Dr.%20Bruce%20H.%20Crowder.pdf

The public clearly has a right and an interest the release of the test questions. NYS past and recent history demonstrates significant quality control concerns that ought to be acknowledged and addressed. Moreover, refusing to disclose he tests is anything but transparent to say the least.

Parent advocates like myself are dedicated to finding a way around the red tape in order to force NYSED to release test questions to the public. NYSEDs failure and refusal to do so has culminated in serious public policy concerns that undermine the best interests of students, sabotage educational programs in public schools and is tantamount to jeopardizing the livelihood of NYS teachers.

So what is the basis for refusing to release the test questions and, if there is a legal basis, isn’t it time to reconsider withholding disclosure of tests in the name of sound and transparent public policy?

I researched some of the possibilities.

The issue was addressed by the Committee on Open Governemnt much to my chagrin. COOG issued an advisory opinion about release of the test questions:

From: Camille S. Jobin-Davis, Assistant Director Sent: Wednesday, June 02, 2010 10:00 AM To: Subject: Freedom of Information Law – test questions/answers/scores

As promised, I reviewed the statutes that apply to access to test scores, questions and answers, and I also reached out to Licensing to learn a bit more about their process.

Pursuant to the Freedom of Information Law here in NY, because the agency utilizes the test questions repeatedly, although they allow applicants to inspect the test questions for a limited period of time after the test, the agency has the legal authority, pursuant to section 87(2)(h), to deny access to copies of the test question themselves.

I also learned more about the review process, including the following:

  • He can be sent his failing score. This is simply the raw score (for ex. 60 out of 100).
  • He can request a review. These are scheduled at a DOS exam site. He would be provided a copy of his answer sheet with incorrect questions indicated (we do not provide the correct answer) and the test booklet. He would be allowed one hour to review the exam. He can bring any type of reference material with him, but is not allowed to take any notes or ask questions of the proctor. At the end of the review session, all materials must be relinquished back to the proctor. This review does allow applicants to determine which questions/types of questions they did not answer correctly.
  • He would not be allowed to discussion (sic) the exam during a review or at any other time. However, he can protest a question if he believes that his answer to a question/s is the correct one for that question/s or that a question is defective. He can do this at the time of his review and the protest is forwarded back to our office for response.
  • He cannot get a copy of the exam as we re-use them on a rotating basis. It is not the type of exam where it is given once or twice a year and each time the exam is retired.

 You may want to contact the person who shared the above information with me. His contact information is as follows:

 Keith A. Simon Department of State Division of Licensing Services Supervisor, Examination Unit 518-473-2731 Keith.Simon@dos.ny.gov

Should you have any further questions, please let me know.

http://docs.dos.ny.gov/coog/ftext/f18130.html

So, according to COOG, the State has a right to withhold tests from a statutory perspective if the questions contained in the test have not been formally retired. Then, the State is permitted to further narrow the public’s right to records based on  the license NYSED shares with its testing vendor.

Is the State acting above and beyond the scope at the expense of the public’s right to access such information?

Lets balance those competing interests, shall we?

Parents are calling for the tests to be released:
“Publishing ELA and math tests was standard practice for New York until just a few years ago. In fact, ELA and math tests from 2010 and before were (and still are) available at nysedregents.org. Other high-stakes standardized tests, such as elementary and middle school science exams, as well as high school Regents exams, are likewise published on this site.

Transparency regarding any high-stakes test is critical for several reasons. By releasing tests, independent experts can determine whether the tests are accurate, grade-level appropriate, unbiased and reasonable in length given the time allotted.”

http://www.lohud.com/…/ela-math-tests-common…/8582131/

From Diane Ravitch’s blog, an educator opined that:

“…the best use of tests is for diagnostic purposes, to help teachers learn what students got wrong and where they need more instruction. Students learn too from their errors. But if the results take months to score, they arrive too late to matter. And if the questions and answers are never released, and students never see their errors, then the tests are used only for ranking, not for helping kids.

This NY teacher calls out state officials for their failure to release the tests:

“Why we will never be permitted to see those tests? I always tell my students and even my own child, “go over the question – look at what you got wrong and try to understand why you got it wrong.” And when my own students do poorly on a test I created, I take a closer look at the test items and try to understand why they got the questions wrong – perhaps I made a bogus test – it’s happened to every educator out there. We won’t be able to do that here. Could it be that these kids didn’t really get all that much wrong? Or is it that the construction of the test items were so riddled with ambiguity and multiple correct responses that they don’t want us to see what a poorly crafted instrument it was? Or, perhaps it is because they tested 4th graders with 7th grade materials? Or that the Commissioner of Education in the state of New York doesn’t have any experience teaching (I’m not sure many of us in the trenches would consider 3 years in a private charter school without open-enrollment “teaching experience”) OR at all as an administrator. Or…or…or… #Want2CtheTEST’

http://dianeravitch.net/2013/08/14/ny-regents-release-the-test-questions/

Another educator bolsters the argument that test questions ought to be released to the public:

“These exams are so deeply flawed, and now so incredibly high stakes. The idea that teachers may lose their jobs and schools (at least in New York City) may be closed based on how children do on these problematic exams is incredibly upsetting and demoralizing to educators. The fact that the state has decided that these exams can never be made public just exacerbates the problem, as the general public will never know how silly the exams are. And, to use an “added value” measure on tests that are not consistently more difficult from year to year is another serious problem.

I understand that you are very busy, but given the importance of the state tests at this time, it is absolutely critical that you analyze them carefully. If you agree with my assessment, I hope that you will consider recommending to the State Legislature that given the flaws in the tests, we are not yet ready to use them for high stakes decision making. I also hope you will consider making these exams public after the test scoring is completed. It is ironic that teachers’ individual ratings are made public while the actual test that determines those ratings is not. I know that the state already has a long-term contract with Pearson, but there is something seriously wrong with a testing company that has such inappropriate questions and passages on such a high stakes test. “
http://www.washingtonpost.com/blogs/answer-sheet/post/principal-says-many-questions-flawed-on-state-standardized-tests/2012/04/25/gIQALIpthT_blog.html

Per Valerie Strauus, here’s another great argument about why it matters when state education departments refuse to release sample questions on state standardized accountability tests. This was written by Aaron Pallas, professor of sociology and education at Teachers College, Columbia University.

“First Rule of High-Stakes Assessments: Talk about High-Stakes Assessments

Second Rule of High-Stakes Assessments: TALK about High-Stakes Assessments.”

http://www.washingtonpost.com/blogs/answer-sheet/wp/2013/04/26/why-the-public-should-see-questions-on-new-standardized-tests/

Several esteemed Principals in New York issued a letter condemning the State’s assessments and warning parents about the questions in testing point, value and purpose. They wrote:

Here’s what we know:

1)    NYS Testing Has Increased Dramatically: We know that our students are spending more time taking State tests than ever before. Since 2010, the amount of time spent on average taking the 3-8 ELA and Math tests has increased by a whopping 128%! The increase has been particularly hard on our younger students, with third graders seeing an increase of 163%!

2)    The Tests were Too Long: We know that many students were unable to complete the tests in the allotted time. Not only were the tests lengthy and challenging, but embedded field test questions extended the length of the tests and caused mental exhaustion, often before students reached the questions that counted toward their scores. For our Special Education students who receive additional time, these tests have become more a measure of endurance than anything else.

3)    Ambiguous Questions Appeared throughout the Exams: We know that many teachers and principals could not agree on the correct answers to ambiguous questions in both ELA and Math. In some schools, identical passages and questions appeared on more than one test and at more than one grade level. One school reported that on one day of the ELA Assessment, the same passage with identical questions was included in the third, fourth AND fifth grade ELA Assessments.

4)    Children have Reacted Viscerally to the Tests: We know that many children cried during or after testing, and others vomited or lost control of their bowels or bladders. Others simply gave up. One teacher reported that a student kept banging his head on the desk, and wrote, “This is too hard,” and “I can’t do this,” throughout his test booklet.

5)    The Low Passing Rate was Predicted: We know that in his “Implementation of the Common Core Learning Standards” memo of March 2013, Deputy Commissioner Slentz stated that proficiency scores (i.e., passing rate) on the new assessments would range between 30%-37% statewide. When scores were released in August 2013, the statewide proficiency rate was announced as 31%.

6)    The College Readiness Benchmark is Irresponsibly Inflated: We know that the New York State Education Department used SAT scores of 560 in Reading, 540 in Writing and 530 in mathematics, as the college readiness benchmarks to help set the “passing” cut scores on the 3-8 New York State exams. These NYSED scores, totaling 1630, are far higher than the College Board’s own college readiness benchmark score of 1550. By doing this, NYSED has carelessly inflated the “college readiness” proficiency cut scores for students as young as nine years of age.

7)    State Measures are Contradictory: We know that many children are receiving scores that are not commensurate with the abilities they demonstrate on other measures, particularly the New York State Integrated Algebra Regents examination. Across New York, many accelerated eighth-graders scored below proficiency on the eighth grade test only to go on and excel on the Regents examination one month later. One district reports that 58% of the students who scored below proficiency on the NYS Math 8 examination earned a mastery score on the Integrated Algebra Regents.

8)    Students Labeled as Failures are Forced Out of Classes: We know that many students who never needed Academic Intervention Services (AIS) in the past, are now receiving mandated AIS as a result of the failing scores. As a result, these students are forced to forgo enrichment classes. For example, in one district, some middle school students had to give up instrumental music, computer or other special classes in order to fit AIS into their schedules.

9)    The Achievement Gap is Widening: We know that the tests have caused the achievement gap to widen as the scores of economically disadvantaged students plummeted, and that parents are reporting that low-scoring children feel like failures.

10) The Tests are Putting Financial Strains on Schools: We know that many schools are spending precious dollars on test prep materials, and that instructional time formerly dedicated to field trips, special projects, the arts and enrichment, has been reallocated to test prep, testing, and AIS services.

11) The Tests are Threatening Other State Initiatives: Without a doubt, the emphasis on testing is threatening other important State initiatives, most notably the implementation of the Common Core State Standards (CCSS). Parents who see the impact of the testing on their children are blaming the CCSS, rather than the unwise decision to implement high stakes testing before proper capacity had been developed. As long as these tests remain, it will be nearly impossible to have honest conversations about the impact of the CCSS on our schools.

Here’s what we do not know:

1)    How these Tests will Help our Students: With the exception of select questions released by the state, we do not have access to the test questions. Without access to the questions, it is nearly impossible to use the tests to help improve student learning.

2)    How to Use these Tests to Improve Student Skills or Understanding: Tests should serve as a tool for assessing student skills and understanding. Since we are not informed of the make-up of the tests, we do not know, with any level of specificity, the content or skills for which children require additional support. We do not even know how many points were allotted for each question.

3)    The Underlying Cause of Low Test Scores: We do not know if children’s low test scores are actually due to lack of skills in that area or simply a case of not finishing the test — a problem that plagued many students.

4)    What to Expect Next Year: We do not know what to expect for next year. Our students are overwhelmed by rapidly changing standards, curriculum and assessments. It is nearly impossible to serve and protect the students in our care when expectations are in constant flux and put in place rapidly in a manner that is not reflective of sound educational practice.

5)    How Much this is Costing Already-Strained Taxpayers: We don’t know how much public money is being paid to vendors and corporations that the NYSED contracts to design assessments, nor do we know if the actual designers are educationally qualified.

http://www.newyorkprincipals.org/letter-to-parents-about-testing

Despite some of the sound and legitimate concerns raised above, NYSED still refuses to release the test questions to the public. Private companies, profit motive and myopic competing interests seem to prevail in NYS over the best interests of students, teachers and the public’s right of access to information as prescribed by law thereby undermining the legislative intent behind public policy records access measures.

Parent advocacy organizations are seeking to acquire the release of test questions after administration of ELA and MATH standardized tests from NYSED in light of inter alia, several instance of exam error and quality assurance concerns. These concerns trump private business and/or any profit interests, in my opinion.

NYFOIL can be found in the Public Officers Law, Sections 84-90. NYFOIL “makes all government records ‘presumptively available to the public for inspection’ unless they fall under a limited number of exceptions. Capital Newspapers Div. of Hearst Corp. v. Burns, 67 N.Y.2d 562, 566, 505 N.Y.S.2d 576 (1986). The general philosophy underpinning the statute is full agency disclosure in order to ‘achieve maximum public access to government documents.’ Encore Coll. Bookstores v. Auxiliary Serv. Corp. of State Univ. of NY at Farmingdale, 87 N.Y.2d 410, 416, 639 N.Y.S.2d 990 (1995).

NYSED refused to release some test questions. Although I have not found the specific reason in writing, I assume NYSED is withholding the records on the basis of 87(2)(h) of Public Officers Law, which presently permits the State to withhold records that “are examination questions or answers which are requested prior to the final administration of such questions.”

Public Officers Law Article 6, 87(2)h reads, in relevant part, as follows:

§87. Access to agency records.

———->2. Each agency shall, in accordance with its published rules, make available for public inspection and copying all records, except that such agency may deny access to records or portions thereof that:

————————>(h) are examination questions or answers which are requested prior to the final administration of such questions;

http://www.dos.ny.gov/coog/foil2.html#s87

But that was not always the case. In the recent past, the provision and legislative intent was broad in favor of the public’s right to access this info. But, it is being construed to limit the public’s right to info in recent years with alarming results.

In advisory opinions issued by COOG, it has been held that “if there is an intent or possibility that the questions used in the examination that is the subject of your inquiry will be used in the future, I believe that the questions, as well as the students’ answers, could be withheld. Disclosure of the questions or the answers in that circumstance would diminish or perhaps nullify the utility or efficacy of the exam.”

As to 87(2)(h), it has also been held that:

“The purpose of that provision is obvious. If questions used in an examination, whether it be a civil service exam or an examination given to students, are disclosed before they are finally given, the examination process and its ntegrity would be compromised. In short, if examination questions will be used in the future, the Law permits an agency to deny access to both the questions and the answers.”

If there is an intent or possibility that the questions used in the examination that is the subject of the inquiry will be used in the future, COOG has held that the questions, as well as the students’ answers, could be withheld because disclosure of the questions or the answers in that circumstance could diminish or perhaps nullify the utility or efficacy of the exam.

It should be noted that while NYFOIL does provide specific exemptions for when a government may withhold documents to the public, NYSED is not allowed under NYFOIL to provide greater protections for confidentiality than those set forth in Section 87(2).

http://www.dos.ny.gov/coog/foil2.html

But, dont let all that legal mumbo jumbo fool you. COOGs reasoning behind withholding the exams from public view rests on competing interests but the impact of NYSEDs decision should be looked at as a matter of public policy. Although NYSED may legally be permitted to withhold tests, is that ethical and is that choice truly in the best interest of student achievemnt and teacher evaluations purposes as a matter of education policy? Why  deprive the public  of information that facilitate academic and professional improvement outweighs the State’s interest in retaining these records for profit, does it not?

How much money are we wasting on  tests that contain errors and are deficient? Are the questions even sound and reliable? Is the test valid?

How can we know this if the public is denied access to the test and the very results that the State is using to make serious and expensive policy decisions on is based on this top secret test? Isnt this counter intuitive to sound public policy?

That test integrity boasts measures that have become more secure than the Pentagon is a clear indicator that the State’s priorities have shifted from “education policy” to coporate greed. NYS will do anything to protect the test at the expense of students and teachers.

While there may be a boilerplate exception as a matter of law, what has not been said is that withholding the test questions is not only unethical but it is also not in the best interest of students and teachers. Pineapple Hare, taking test results and missing questions sure do suggest lack of oversight as to quality control. The harm these top scret tests are  causing students and teachers can no longer be ignored.

There are clear public policy concerns at play that warrant consideration and further action to resolve in favor of students and teachers. It is time that NYS weighs the State’s interests in purportedly preserving the integrity of the tests against the adverse impact this practice is having on students, teachers and taxpayers as a matter of policy.

While there may be an legal exception, there is no true reason why NYSED should be  enforcing 87(2)(h). NYSED certainly could choose to disclose the tests to the public particularly since doing so would serve to foster  student achievment and lead to teacher improvement by way of feedback of the same.

Testing concerns call for disclosure of tests. NYS interests in “test integrity” runs afoul of sound education policy.

Until then, parents and students across the State continue to push back against this terribly flawed testing program and policy in the only way that they can make their voices heard: by refusing the test.

As of this date, approx 35, 000+ students refused the ELA exams. That number is expected to skyrocket once Math refusals are tallied.

Here is a link to the public’s copy of the refusal being tallied State wide by advocacy organizations and parents in real time.

Those opting to exercise their right to refuse the test speak volumes.

Is NYSED listening?

https://docs.google.com/spreadsheet/ccc?key=0AuLBonoXvLu9dFF1NmtyeWxGTmpRazYtcXoyVGFMeVE&usp=drive_web#gid=0

Related Links:

PBS on Testing

http://www.pbs.org/…/shows/schools/testing/companies.html

 

http://www.pbs.org/…/front…/shows/schools/etc/guide.html

 

2011 article that discusses Pearson lobbyist Kress (Tx) and how big business and education policy have intertwined.

http://www.commondreams.org/view/2011/09/08-9

 

A teacher on Long Island talks Common Core http://bit.ly/1rGO94O 

 

How @PearsonNorthAm profits off of the standardized tests “administered” to our school children. http://www.alternet.org/education/corporations-profit-standardized-

http://nycpublicschoolparents.blogspot.com/2011/08/nyc-test-scores-small-and-unreliable.html

http://www.highered.nysed.gov/tsei/

http://standardizedtests.procon.org/

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s