microsoft.public.cert.exams.mcse - Educational Testing Service defeats Brain Dumps

Author

Educational Testing Service defeats Brain Dumps

John Smith

2002-08-08, 9:23 am

Tests can be, and have been, changed by at least two organizations,
Educational Testing Service and the FAA, to defeat the use of "brain
dumps". The FAA's approach was to have a regularly rotating base of
over 1,000 questions that they will freely give you on request. Most
commenters agree that the shear number of possible questions on the
FAA exams made it easier to simply study for the test than to try and
memorize all the questions and answers. Read about Educational
Testing Service's approach below.

-----------------------------------------------------------------------------------------------------------

Officials Link Foreign Web Sites to Cheating on Graduate Admission
Exams

By JACQUES STEINBERG

An undetermined number of students in China, Taiwan and South Korea
were able to raise their scores substantially last year on the verbal
part of the most widely used entrance exam to American graduate
schools by logging on to Web sites in those countries that post
questions and answers memorized by previous test takers, test
administrators said yesterday.

After uncovering the Chinese- and Korean-language Web sites and
assessing their effect on scores, test administrators suspended the
electronic version of the tests, known as the Graduate Record
Examinations, which have been taken by 55,000 students annually in
those countries since the late 1990's. Because the tests were given at
testing centers six days a week, the questions were regularly reused,
making the tests susceptible to such cheating.

Now, the test will be given in those three countries only two days a
year, in November and March, and on paper, to guard the security of
the questions, which will be used only once. Testing officials said
they had not found a similar problem in any other country, including
the United States.

An investigation, by the Educational Testing Service, which designs
the exam, was prompted in part by the concerns of some American
college deans that the high verbal scores of some Asian students did
not match their English fluency.

Worldwide, nearly a half-million students a year take the exams, which
measure verbal and math skills. Testing officials said they did not
know how many of the students in China, South Korea and Taiwan had
cheated but that it would have taken a substantial number to cause the
average scores to increase as much as they did.

In China, for example, the average score on the verbal section of the
exam rose 100 points in a period examined last year, on a scale of 200
to 800, testing officials said. In South Korea and Taiwan, the scores
increased by 50 points.

Though the tests given in those countries often draw on the same pool
of questions as the exams in given in the United States, which are
also administered electronically nearly every day, testing officials
said that they had not uncovered similar Web sites or unusual gains
anywhere else. Testing officials said that the Web sites were probably
the work of students, not bootleg businesses.

Still, Carole Beere, the chairwoman of the Graduate Record
Examinations Board, which oversees the tests, said, "We are now
monitoring the Web very aggressively."

Tom Ewing, a spokesman for the Educational Testing Service, said the
company had rarely come upon evidence of such sophisticated,
coordinated efforts to disseminate its questions and answers. In 1996,
a California man was charged with recruiting people on the East Coast
to take standardized tests, including the Graduate Management
Admission Test, which most business schools require, only to relay the
answers to students on the West Coast, in part by using pencils on
which the answers were written in code.

More recently, Mr. Ewing said, the service discovered that a question
on a high school Advanced Placement exam was being circulated among
students via e-mail.

Though Dr. Beere said she remained confident in the validity of the
exam in the United States and elsewhere, some critics argued that the
breach uncovered in Asia was but one sign of the perils of
administering standardized tests in the age of the Internet — and of
putting too much stock in the results of any test.

"It's not called the World Wide Web for no reason," said Robert A.
Schaeffer, the public education director for the National Center for
Fair and Open Testing, an advocacy group. "Those sites, even if they
exist in another language, are accessible to students in the U.S. as
well as in China. And there are many students in the U.S. who speak
Chinese."

Dr. Beere said that this year her board had ordered a far-ranging
investigation of the validity of test results in 40 countries,
including the United States, and had found evidence of cheating in
only those three.

Testing officials said the computerized exams drew on a rotating
series of questions, often different for students taking the exam next
to each other. After a certain period — which the testing service
refused to disclose, but is believed to be at least several weeks — a
question is removed from the pool permanently.

Testing officials said they were able to confirm the cheating by
noting that the scores on certain questions had risen significantly
the longer the question was in circulation.

Dr. Beere reported the results of the investigation to deans at more
than 1,000 graduate schools in a letter mailed late last week. Its
contents were first described yesterday in an online edition of The
Chronicle of Higher Education.

In the letter, Dr. Beere, who is also the associate provost for
graduate studies and outreach at Northern Kentucky University, said
the board would work to "restore the confidence and trust that you
have in the scores from applicants in the affected regions."

She also reminded her colleagues to view the scores of any applicant
in the context of the entire application.

Microsoft Networks

2002-08-08, 8:23 pm

The FAA's approach was to have a regularly rotating base of
> over 1,000 questions that they will freely give you on request. Most
> commenters agree that the shear number of possible questions on the
> FAA exams made it easier to simply study for the test than to try and
> memorize all the questions and answers.

They obviously don't know my memory, because I would EASILY remember all
1000 questions. This is just a fact, I'm not trying to brag.

Look, I feel it boils down to having set times once or twice a year when
people can go do exams for certifications and where entirely new questions
are presented. This poses big, big problems of logistics however.

Keep posting this stuff because it is interesting. These exam violations are
an extreme concern which are not properly addressed at this time.

It does indeed have everything to do with the internet and it sure is not
called that for nothing.

"John Smith" <JSmith@hotmale.com> wrote in message
news:9105luge1jego8rqvdkb2o0hn
rcsuv778p@4ax.com...
> Tests can be, and have been, changed by at least two organizations,
> Educational Testing Service and the FAA, to defeat the use of "brain
> dumps". The FAA's approach was to have a regularly rotating base of
> over 1,000 questions that they will freely give you on request. Most
> commenters agree that the shear number of possible questions on the
> FAA exams made it easier to simply study for the test than to try and
> memorize all the questions and answers. Read about Educational
> Testing Service's approach below.
>
> --------------------------------------------------------------------------
---------------------------------
>
> Officials Link Foreign Web Sites to Cheating on Graduate Admission
> Exams
>
> By JACQUES STEINBERG
>
>
> An undetermined number of students in China, Taiwan and South Korea
> were able to raise their scores substantially last year on the verbal
> part of the most widely used entrance exam to American graduate
> schools by logging on to Web sites in those countries that post
> questions and answers memorized by previous test takers, test
> administrators said yesterday.
>
> After uncovering the Chinese- and Korean-language Web sites and
> assessing their effect on scores, test administrators suspended the
> electronic version of the tests, known as the Graduate Record
> Examinations, which have been taken by 55,000 students annually in
> those countries since the late 1990's. Because the tests were given at
> testing centers six days a week, the questions were regularly reused,
> making the tests susceptible to such cheating.
>
> Now, the test will be given in those three countries only two days a
> year, in November and March, and on paper, to guard the security of
> the questions, which will be used only once. Testing officials said
> they had not found a similar problem in any other country, including
> the United States.
>
> An investigation, by the Educational Testing Service, which designs
> the exam, was prompted in part by the concerns of some American
> college deans that the high verbal scores of some Asian students did
> not match their English fluency.
>
> Worldwide, nearly a half-million students a year take the exams, which
> measure verbal and math skills. Testing officials said they did not
> know how many of the students in China, South Korea and Taiwan had
> cheated but that it would have taken a substantial number to cause the
> average scores to increase as much as they did.
>
> In China, for example, the average score on the verbal section of the
> exam rose 100 points in a period examined last year, on a scale of 200
> to 800, testing officials said. In South Korea and Taiwan, the scores
> increased by 50 points.
>
> Though the tests given in those countries often draw on the same pool
> of questions as the exams in given in the United States, which are
> also administered electronically nearly every day, testing officials
> said that they had not uncovered similar Web sites or unusual gains
> anywhere else. Testing officials said that the Web sites were probably
> the work of students, not bootleg businesses.
>
> Still, Carole Beere, the chairwoman of the Graduate Record
> Examinations Board, which oversees the tests, said, "We are now
> monitoring the Web very aggressively."
>
> Tom Ewing, a spokesman for the Educational Testing Service, said the
> company had rarely come upon evidence of such sophisticated,
> coordinated efforts to disseminate its questions and answers. In 1996,
> a California man was charged with recruiting people on the East Coast
> to take standardized tests, including the Graduate Management
> Admission Test, which most business schools require, only to relay the
> answers to students on the West Coast, in part by using pencils on
> which the answers were written in code.
>
> More recently, Mr. Ewing said, the service discovered that a question
> on a high school Advanced Placement exam was being circulated among
> students via e-mail.
>
> Though Dr. Beere said she remained confident in the validity of the
> exam in the United States and elsewhere, some critics argued that the
> breach uncovered in Asia was but one sign of the perils of
> administering standardized tests in the age of the Internet - and of
> putting too much stock in the results of any test.
>
> "It's not called the World Wide Web for no reason," said Robert A.
> Schaeffer, the public education director for the National Center for
> Fair and Open Testing, an advocacy group. "Those sites, even if they
> exist in another language, are accessible to students in the U.S. as
> well as in China. And there are many students in the U.S. who speak
> Chinese."
>
> Dr. Beere said that this year her board had ordered a far-ranging
> investigation of the validity of test results in 40 countries,
> including the United States, and had found evidence of cheating in
> only those three.
>
> Testing officials said the computerized exams drew on a rotating
> series of questions, often different for students taking the exam next
> to each other. After a certain period - which the testing service
> refused to disclose, but is believed to be at least several weeks - a
> question is removed from the pool permanently.
>
> Testing officials said they were able to confirm the cheating by
> noting that the scores on certain questions had risen significantly
> the longer the question was in circulation.
>
> Dr. Beere reported the results of the investigation to deans at more
> than 1,000 graduate schools in a letter mailed late last week. Its
> contents were first described yesterday in an online edition of The
> Chronicle of Higher Education.
>
> In the letter, Dr. Beere, who is also the associate provost for
> graduate studies and outreach at Northern Kentucky University, said
> the board would work to "restore the confidence and trust that you
> have in the scores from applicants in the affected regions."
>
> She also reminded her colleagues to view the scores of any applicant
> in the context of the entire application.
>

John Smith

2002-08-09, 3:23 pm

On Fri, 9 Aug 2002 03:00:39 +0100, "Microsoft Networks"
<richard.warwick@blueyonder.co.uk> wrote:

> The FAA's approach was to have a regularly rotating base of
>> over 1,000 questions that they will freely give you on request. Most
>> commenters agree that the shear number of possible questions on the
>> FAA exams made it easier to simply study for the test than to try and
>> memorize all the questions and answers.
>
> They obviously don't know my memory, because I would EASILY remember all
>1000 questions. This is just a fact, I'm not trying to brag.

lol, even if you're not trying to brag. I'm impressed. Someone with
that good a memory would likely be able to manage under most
circumstances. I'd say for MCSD & MCSE exams, the question pool might
need to be larger than for the FAA exam.

>Look, I feel it boils down to having set times once or twice a year when
>people can go do exams for certifications and where entirely new questions
>are presented. This poses big, big problems of logistics however.

That's why I posted the article on the apporach taken by Edu.Testing
Serv. That's what they did, and it seems to be working for them.

>
>Keep posting this stuff because it is interesting. These exam violations are
>an extreme concern which are not properly addressed at this time.

Thanks for the civil and constructive comments!

>
>It does indeed have everything to do with the internet and it sure is not
>called that for nothing.
>
>"John Smith" <JSmith@hotmale.com> wrote in message
> news:9105luge1jego8rqvdkb2o0hn
rcsuv778p@4ax.com...
>> Tests can be, and have been, changed by at least two organizations,
>> Educational Testing Service and the FAA, to defeat the use of "brain
>> dumps". The FAA's approach was to have a regularly rotating base of
>> over 1,000 questions that they will freely give you on request. Most
>> commenters agree that the shear number of possible questions on the
>> FAA exams made it easier to simply study for the test than to try and
>> memorize all the questions and answers. Read about Educational
>> Testing Service's approach below.
>>
>> --------------------------------------------------------------------------
>---------------------------------
>>
>> Officials Link Foreign Web Sites to Cheating on Graduate Admission
>> Exams
>>
>> By JACQUES STEINBERG
>>
>>
>> An undetermined number of students in China, Taiwan and South Korea
>> were able to raise their scores substantially last year on the verbal
>> part of the most widely used entrance exam to American graduate
>> schools by logging on to Web sites in those countries that post
>> questions and answers memorized by previous test takers, test
>> administrators said yesterday.
>>
>> After uncovering the Chinese- and Korean-language Web sites and
>> assessing their effect on scores, test administrators suspended the
>> electronic version of the tests, known as the Graduate Record
>> Examinations, which have been taken by 55,000 students annually in
>> those countries since the late 1990's. Because the tests were given at
>> testing centers six days a week, the questions were regularly reused,
>> making the tests susceptible to such cheating.
>>
>> Now, the test will be given in those three countries only two days a
>> year, in November and March, and on paper, to guard the security of
>> the questions, which will be used only once. Testing officials said
>> they had not found a similar problem in any other country, including
>> the United States.
>>
>> An investigation, by the Educational Testing Service, which designs
>> the exam, was prompted in part by the concerns of some American
>> college deans that the high verbal scores of some Asian students did
>> not match their English fluency.
>>
>> Worldwide, nearly a half-million students a year take the exams, which
>> measure verbal and math skills. Testing officials said they did not
>> know how many of the students in China, South Korea and Taiwan had
>> cheated but that it would have taken a substantial number to cause the
>> average scores to increase as much as they did.
>>
>> In China, for example, the average score on the verbal section of the
>> exam rose 100 points in a period examined last year, on a scale of 200
>> to 800, testing officials said. In South Korea and Taiwan, the scores
>> increased by 50 points.
>>
>> Though the tests given in those countries often draw on the same pool
>> of questions as the exams in given in the United States, which are
>> also administered electronically nearly every day, testing officials
>> said that they had not uncovered similar Web sites or unusual gains
>> anywhere else. Testing officials said that the Web sites were probably
>> the work of students, not bootleg businesses.
>>
>> Still, Carole Beere, the chairwoman of the Graduate Record
>> Examinations Board, which oversees the tests, said, "We are now
>> monitoring the Web very aggressively."
>>
>> Tom Ewing, a spokesman for the Educational Testing Service, said the
>> company had rarely come upon evidence of such sophisticated,
>> coordinated efforts to disseminate its questions and answers. In 1996,
>> a California man was charged with recruiting people on the East Coast
>> to take standardized tests, including the Graduate Management
>> Admission Test, which most business schools require, only to relay the
>> answers to students on the West Coast, in part by using pencils on
>> which the answers were written in code.
>>
>> More recently, Mr. Ewing said, the service discovered that a question
>> on a high school Advanced Placement exam was being circulated among
>> students via e-mail.
>>
>> Though Dr. Beere said she remained confident in the validity of the
>> exam in the United States and elsewhere, some critics argued that the
>> breach uncovered in Asia was but one sign of the perils of
>> administering standardized tests in the age of the Internet - and of
>> putting too much stock in the results of any test.
>>
>> "It's not called the World Wide Web for no reason," said Robert A.
>> Schaeffer, the public education director for the National Center for
>> Fair and Open Testing, an advocacy group. "Those sites, even if they
>> exist in another language, are accessible to students in the U.S. as
>> well as in China. And there are many students in the U.S. who speak
>> Chinese."
>>
>> Dr. Beere said that this year her board had ordered a far-ranging
>> investigation of the validity of test results in 40 countries,
>> including the United States, and had found evidence of cheating in
>> only those three.
>>
>> Testing officials said the computerized exams drew on a rotating
>> series of questions, often different for students taking the exam next
>> to each other. After a certain period - which the testing service
>> refused to disclose, but is believed to be at least several weeks - a
>> question is removed from the pool permanently.
>>
>> Testing officials said they were able to confirm the cheating by
>> noting that the scores on certain questions had risen significantly
>> the longer the question was in circulation.
>>
>> Dr. Beere reported the results of the investigation to deans at more
>> than 1,000 graduate schools in a letter mailed late last week. Its
>> contents were first described yesterday in an online edition of The
>> Chronicle of Higher Education.
>>
>> In the letter, Dr. Beere, who is also the associate provost for
>> graduate studies and outreach at Northern Kentucky University, said
>> the board would work to "restore the confidence and trust that you
>> have in the scores from applicants in the affected regions."
>>
>> She also reminded her colleagues to view the scores of any applicant
>> in the context of the entire application.
>>
>

Richard Warwick

2002-08-09, 7:23 pm

> lol, even if you're not trying to brag. I'm impressed. Someone with
> that good a memory would likely be able to manage under most
> circumstances. I'd say for MCSD & MCSE exams, the question pool might
> need to be larger than for the FAA exam.

I take it then that my memory is good

There are also specific techniques
by which people demonstrate incredible memory feats. The effective ones are
mostly based on association. My visual memory is good, though I would
refrain from calling it "photographic."

In short, the "large pool" of questions is better than nothing but not by
itself sufficient.

Rearranging the answer order wouldn't phase me in the slightest.

Adaptive testing is a good initiative and not without its merits. Again I
feel that as a technique in isolation it is insufficient.

Maybe we should all sign a petition and hand it in, to the delight of
Microsoft?

"John Smith" <JSmith@hotmale.com> wrote in message
news:sg98lu4lvir46s64jisb41adi
o64g63ijh@4ax.com...
> On Fri, 9 Aug 2002 03:00:39 +0100, "Microsoft Networks"
> <richard.warwick@blueyonder.co.uk> wrote:
>
> > The FAA's approach was to have a regularly rotating base of
> >> over 1,000 questions that they will freely give you on request. Most
> >> commenters agree that the shear number of possible questions on the
> >> FAA exams made it easier to simply study for the test than to try and
> >> memorize all the questions and answers.
> >
> > They obviously don't know my memory, because I would EASILY remember all
> >1000 questions. This is just a fact, I'm not trying to brag.
>
> lol, even if you're not trying to brag. I'm impressed. Someone with
> that good a memory would likely be able to manage under most
> circumstances. I'd say for MCSD & MCSE exams, the question pool might
> need to be larger than for the FAA exam.
>
> >Look, I feel it boils down to having set times once or twice a year when
> >people can go do exams for certifications and where entirely new
questions
> >are presented. This poses big, big problems of logistics however.
>
> That's why I posted the article on the apporach taken by Edu.Testing
> Serv. That's what they did, and it seems to be working for them.
>
> >
> >Keep posting this stuff because it is interesting. These exam violations
are
> >an extreme concern which are not properly addressed at this time.
>
> Thanks for the civil and constructive comments!
>
> >
> >It does indeed have everything to do with the internet and it sure is not
> >called that for nothing.
> >
> >"John Smith" <JSmith@hotmale.com> wrote in message
> > news:9105luge1jego8rqvdkb2o0hn
rcsuv778p@4ax.com...
> >> Tests can be, and have been, changed by at least two organizations,
> >> Educational Testing Service and the FAA, to defeat the use of "brain
> >> dumps". The FAA's approach was to have a regularly rotating base of
> >> over 1,000 questions that they will freely give you on request. Most
> >> commenters agree that the shear number of possible questions on the
> >> FAA exams made it easier to simply study for the test than to try and
> >> memorize all the questions and answers. Read about Educational
> >> Testing Service's approach below.
> >>
>
>> -------------------------------------------------------------------------
-
> >---------------------------------
> >>
> >> Officials Link Foreign Web Sites to Cheating on Graduate Admission
> >> Exams
> >>
> >> By JACQUES STEINBERG
> >>
> >>
> >> An undetermined number of students in China, Taiwan and South Korea
> >> were able to raise their scores substantially last year on the verbal
> >> part of the most widely used entrance exam to American graduate
> >> schools by logging on to Web sites in those countries that post
> >> questions and answers memorized by previous test takers, test
> >> administrators said yesterday.
> >>
> >> After uncovering the Chinese- and Korean-language Web sites and
> >> assessing their effect on scores, test administrators suspended the
> >> electronic version of the tests, known as the Graduate Record
> >> Examinations, which have been taken by 55,000 students annually in
> >> those countries since the late 1990's. Because the tests were given at
> >> testing centers six days a week, the questions were regularly reused,
> >> making the tests susceptible to such cheating.
> >>
> >> Now, the test will be given in those three countries only two days a
> >> year, in November and March, and on paper, to guard the security of
> >> the questions, which will be used only once. Testing officials said
> >> they had not found a similar problem in any other country, including
> >> the United States.
> >>
> >> An investigation, by the Educational Testing Service, which designs
> >> the exam, was prompted in part by the concerns of some American
> >> college deans that the high verbal scores of some Asian students did
> >> not match their English fluency.
> >>
> >> Worldwide, nearly a half-million students a year take the exams, which
> >> measure verbal and math skills. Testing officials said they did not
> >> know how many of the students in China, South Korea and Taiwan had
> >> cheated but that it would have taken a substantial number to cause the
> >> average scores to increase as much as they did.
> >>
> >> In China, for example, the average score on the verbal section of the
> >> exam rose 100 points in a period examined last year, on a scale of 200
> >> to 800, testing officials said. In South Korea and Taiwan, the scores
> >> increased by 50 points.
> >>
> >> Though the tests given in those countries often draw on the same pool
> >> of questions as the exams in given in the United States, which are
> >> also administered electronically nearly every day, testing officials
> >> said that they had not uncovered similar Web sites or unusual gains
> >> anywhere else. Testing officials said that the Web sites were probably
> >> the work of students, not bootleg businesses.
> >>
> >> Still, Carole Beere, the chairwoman of the Graduate Record
> >> Examinations Board, which oversees the tests, said, "We are now
> >> monitoring the Web very aggressively."
> >>
> >> Tom Ewing, a spokesman for the Educational Testing Service, said the
> >> company had rarely come upon evidence of such sophisticated,
> >> coordinated efforts to disseminate its questions and answers. In 1996,
> >> a California man was charged with recruiting people on the East Coast
> >> to take standardized tests, including the Graduate Management
> >> Admission Test, which most business schools require, only to relay the
> >> answers to students on the West Coast, in part by using pencils on
> >> which the answers were written in code.
> >>
> >> More recently, Mr. Ewing said, the service discovered that a question
> >> on a high school Advanced Placement exam was being circulated among
> >> students via e-mail.
> >>
> >> Though Dr. Beere said she remained confident in the validity of the
> >> exam in the United States and elsewhere, some critics argued that the
> >> breach uncovered in Asia was but one sign of the perils of
> >> administering standardized tests in the age of the Internet - and of
> >> putting too much stock in the results of any test.
> >>
> >> "It's not called the World Wide Web for no reason," said Robert A.
> >> Schaeffer, the public education director for the National Center for
> >> Fair and Open Testing, an advocacy group. "Those sites, even if they
> >> exist in another language, are accessible to students in the U.S. as
> >> well as in China. And there are many students in the U.S. who speak
> >> Chinese."
> >>
> >> Dr. Beere said that this year her board had ordered a far-ranging
> >> investigation of the validity of test results in 40 countries,
> >> including the United States, and had found evidence of cheating in
> >> only those three.
> >>
> >> Testing officials said the computerized exams drew on a rotating
> >> series of questions, often different for students taking the exam next
> >> to each other. After a certain period - which the testing service
> >> refused to disclose, but is believed to be at least several weeks - a
> >> question is removed from the pool permanently.
> >>
> >> Testing officials said they were able to confirm the cheating by
> >> noting that the scores on certain questions had risen significantly
> >> the longer the question was in circulation.
> >>
> >> Dr. Beere reported the results of the investigation to deans at more
> >> than 1,000 graduate schools in a letter mailed late last week. Its
> >> contents were first described yesterday in an online edition of The
> >> Chronicle of Higher Education.
> >>
> >> In the letter, Dr. Beere, who is also the associate provost for
> >> graduate studies and outreach at Northern Kentucky University, said
> >> the board would work to "restore the confidence and trust that you
> >> have in the scores from applicants in the affected regions."
> >>
> >> She also reminded her colleagues to view the scores of any applicant
> >> in the context of the entire application.
> >>
> >
>

Sponsored Links