ponder=on vs ponder=off comparision

[Volker Pittlik]

Geschrieben von: / Posted by: Volker Pittlik at 15 July 2000 13:31:18:
Programs: AnMon506, Arasanx_53, Amyf_000420, Bringer16, Comet_B21, Crafty1710, EXchess314, Gromit_30, LambChop_710, Phalanx22, SOS_991103, TCB0045

round robin, 20 games each match, time control 60+2
System: 2*520 MHz Celeron, NT4.0
Setting: ~32 MB hash each program, learning on, books coming with the program, all 4- and some 5-piece TBs
All games can be found in my gamesArchive
Ponder=on

No. Name Win Draw Loss Unf. Score Games % SOP
----------------------------------------------------------------
1 Crafty1710 +140 =39 -41 *0 159.5 220 72.5% 5820.0
2 AnMon506 +120 =36 -64 *0 138.0 220 62.7% 5470.0
3 Amyf_000420 +111 =35 -74 *0 128.5 220 58.4% 4855.0
4 Gromit_30 +106 =36 -78 *0 124.0 220 56.4% 6285.0
5 TCB0045 +107 =34 -79 *0 124.0 220 56.4% 4520.0
6 Comet_B21 +98 =42 -80 *0 119.0 220 54.1% 6080.0
7 Phalanx22 +93 =38 -89 *0 112.0 220 50.9% 7140.0
8 SOS_991103 +92 =36 -92 *0 110.0 220 50.0% 6925.0
9 Bringer16 +85 =36 -99 *0 103.0 220 46.8% 5140.0
10 EXchess314 +49 =45 -126 *0 71.5 220 32.5% 5140.0
11 Arasanx_53 +48 =46 -126 *0 71.0 220 32.3% 3860.0
12 LambChop_710 +43 =33 -144 *0 59.5 220 27.0% 4765.0
Total Games: 1320
White Wins: 596 (45.2%)
Black Wins: 496 (37.6%)
Draws: 228 (17.3%)
Unfinished: 0 (0.0%)
Ponder=off

Ponder=off
No. Name Win Draw Loss Unf. Score Games % SOP
----------------------------------------------------------------
1 Crafty1710 +158 =36 -26 *0 176.0 220 80.0% 6105.0
2 Gromit_30 +110 =42 -68 *0 131.0 220 59.5% 6460.0
3 Amyf_000420 +106 =46 -68 *0 129.0 220 58.6% 4520.0
4 AnMon506 +110 =37 -73 *0 128.5 220 58.4% 5475.0
5 TCB0045 +100 =48 -72 *0 124.0 220 56.4% 4365.0
6 Comet_B21 +94 =49 -77 *0 118.5 220 53.9% 6275.0
7 Phalanx22 +96 =43 -81 *0 117.5 220 53.4% 7275.0
8 SOS_991103 +96 =35 -89 *0 113.5 220 51.6% 6910.0
9 Bringer16 +64 =43 -113 *0 85.5 220 38.9% 4945.0
10 Arasanx_53 +69 =33 -118 *0 85.5 220 38.9% 3460.0
11 EXchess314 +42 =30 -148 *0 57.0 220 25.9% 5500.0
12 LambChop_710 +36 =36 -148 *0 54.0 220 24.5% 4710.0
Total Games: 1320
White Wins: 564 (42.7%)
Black Wins: 517 (39.2%)
Draws: 239 (18.1%)
Unfinished: 0 (0.0%)
Conclusion
The results of both tournaments are very similar. Tournaments with ponder=off and ponder=on seem to give comparable results.
Volker

[Frank Quisinsky]

Geschrieben von: / Posted by: Frank Quisinsky at 15 July 2000 13:58:05:
Als Antwort auf: / As an answer to: ponder=on vs ponder=off comparision geschrieben von: / posted by: Volker Pittlik at 15 July 2000 13:31:18:
Hallo Volker,
Super, ein Anhaltspunkt (steckt eine Menge Arbeit in diesem Test).
Super weil die Frage viele Personen interessiert und nun auch mal dahingehend ein Ergebnis (mit Aussagekraft) vorliegt.
Gruß
Frank
PS:
Wie dem auch ist, Crafty ist immer noch die Nummer 1 unter den freien Engines und das nun schon viele Jahre !

[Mogens Larsen]

Geschrieben von: / Posted by: Mogens Larsen at 15 July 2000 14:40:48:
Als Antwort auf: / As an answer to: ponder=on vs ponder=off comparision geschrieben von: / posted by: Volker Pittlik at 15 July 2000 13:31:18:

Programs: AnMon506, Arasanx_53, Amyf_000420, Bringer16, Comet_B21, Crafty1710, EXchess314, Gromit_30, LambChop_710, Phalanx22, SOS_991103, TCB0045

round robin, 20 games each match, time control 60+2
System: 2*520 MHz Celeron, NT4.0
Setting: ~32 MB hash each program, learning on, books coming with the program, all 4- and some 5-piece TBs
All games can be found in my gamesArchive
The results of both tournaments are very similar. Tournaments with ponder=off and ponder=on seem to give comparable results.

Volker, thank you for the effort. Very interesting data.
The data does seem to drive a stick through the misconception of an ultimate way to implement ponder that is reserved for the few. Not a surprise result as far as I'm concerned, but it's nice to see data that confirm my thoughts. Even though it's by no means proved that ponder doesn't influence the comparative strength between engines significantly.
Best wishes...
Mogens

MCL HoMePaGe

[pete]

Geschrieben von: / Posted by: pete at 15 July 2000 16:18:35:
Als Antwort auf: / As an answer to: ponder=on vs ponder=off comparision geschrieben von: / posted by: Volker Pittlik at 15 July 2000 13:31:18:

The results of both tournaments are very similar. Tournaments with ponder=off and ponder=on seem to give comparable results.
Volker

At last someone decided to really _do_ this time-consuming test . The result leaves little doubt ; enough data for significance.
Thanks Volker.
pete

[Christian]

Geschrieben von: / Posted by: Christian at 16 July 2000 11:46:38:
Als Antwort auf: / As an answer to: ponder=on vs ponder=off comparision geschrieben von: / posted by: Volker Pittlik at 15 July 2000 13:31:18:
Auch wenn viele Programmierer immer wieder darauf hinweisen
wie sehr ihr Programm vom Pondern abhängt (zB Hyatt),
bestätigen die Ergebnisse in der Praxis das nicht bzw. sind die Konsequenzen
für das prakt. Spiel geringer.
Turnieregebnisse bzw. Elolisten ermittelt ohne Pondern decken sich
tendeziell zumeist mit jenen mit Pondern, so auch meine Erfahrung.
Christian