26 January 2015

Chess Improvement Programs: Directed vs. Eclectic

Although people's learning styles vary significantly - meaning that there is no "one true path" to mastery that will apply to everyone for any skill - it seems obvious that possessing some type of chess improvement program is important for eventual success in gaining playing strength.  ("Do study techniques matter in chess?" previously addressed this idea in a broader sense, as did "Reflections on Training".)

Having a program - in other words, a structured approach to training that contains achievable goals - makes it more much more likely that you will in fact stay on your chosen path, rather than simply studying on a whim at random times.  "Whimsical" training can be done for fun and in some cases may make for a useful intellectual break from other tasks, but it is unlikely to produce significant results over time.

That leads to the question of how to construct a training program.  Here I would like to highlight some different types of resources available that are aimed at adult chess improvers, though they also could apply to up-and-coming juniors.  I break them down generally into "directed" programs, which are intended to be all-inclusive from a single source, and "eclectic" ones, which rely on independently gathering together chess resources.  Directed programs are by nature more highly structured (and demanding), which has its pluses and minuses.  I'll also discuss my own eclectic program as an example of that type.

Directed programs

A directed program, whether with a personal chess coach or following a set lesson plan and materials, is explicitly designed to achieve a set level of skill mastery.  We are all familiar with the "directed" educational programs involved in secondary education (typically through age 18), which aim (with varying results) to provide all students with a grasp of mathematics, science, literature, reading/writing, arts, etc.  The time commitment and need to follow a set plan week after week are what sets these programs apart - and make them more challenging to keep up with.
  • The most recent example of a full, amateur-to-master improvement course is that offered by Chess.com University through their "Prodigy Program".  Despite the name, it is open to adult students and the motto is "Master chess at any age, within 5 years".  Core features include: 7-page PDF weekly study plans; monthly online correspondence tournament; monthly master-level simul; and analysis of students' games.
  • The International Chess School's "Grandmaster Package" has been available for several years now, delivered online with PDF workbooks and video lessons; samples are available at the linked website.  An unofficial forum about the course was set up by former blogger Blue Devil Knight, although it is only active periodically.
  • Another example of a "comprehensive" packaged program is at GM Igor Smirnov's "Remote Chess Academy" which features his "Self-Taught Grandmaster" software along with some more specialized packages on openings and subjects like "How to Beat Titled Players".
  • Tiger Chess by GM Nigel Davies uses a subscription model, with currently 160 weekly lessons available, a monthly clinic with GM Davies, access to resources on openings, and guidance on outside books and internet resources.  A reduced membership level offers some introductory video courses.  A "Chess Improver" blog post talks more about the breakdown of time for various study topics.
  • The granddaddy of comprehensive courses, before online versions existed, was GM Lev Alburt's Comprehensive Chess Course.  Its series of instructional books is not fully integrated, but does run from beginner to advanced material and covers openings, tactics, middlegame strategy, and endings.
  • The modern version of beginner to master chess books is the three-set, nine-volume series by GM Artur Yusupov, under the set titles "Build Up Your Chess", "Boost Your Chess" and "Chess Evolution".  In this case, he breaks up the different topics (calculation, tactics, strategy, positional play, endgame, openings) by strength levels, so each of the three book sets is designed to be comprehensive for its particular level.
Eclectic programs

The line between "directed" and "eclectic" is not necessarily a hard one.  For example, if you follow Yusupov's book series in order, that is certainly structured learning, but not quite at the same level as receiving weekly lessons and exercises to complete.  Eclectic programs by nature rely on your interests and choices to drive the improvement process, which may be more fun and better match your own desires and personality.  The downside is that you will need to put more effort into determining what to study and in evaluating materials beforehand, rather than having it packaged up for you.

"Eclectic" does not necessarily mean disorganized.  Here are some resources designed to help the chess improver get organized for training:
  • IM Igor Khemeltsky's Chess Exam and Training Guide is a book for those looking to diagnose their strengths and weaknesses and receive follow-up guidance, this may be a good place to start.  He also offers an exam book on tactics.
  • Chess.com has "study plans" available on the site which link in an organized way to articles and videos available there.  Chess Mentor courses contain everything from basic tactics to advanced endgames, while the Study Plan Directory provides comprehensive links for training from beginners to intermediate (Class B) level.
  • Chess Training for Budding Champions by Jesper Hall, despite the title, is a rather advanced book on training designed for the ambitious amateur.  It does well in providing concrete ideas for setting up training programs, illustrates topics well with annotated games, and was the primary inspiration for my own focus on analyzing your own games.
  • More recently, Axel Smith's Pump Up Your Rating is another good single-volume work, with perhaps more focus on teaching the concepts rather than illustrating them, yet with training in mind.  I believe it's worth quoting an excerpt from the introduction, which echoes some of the themes I've addressed in this blog:
"[Serious chess training] can be summarized in five words: active learning through structured training...The four pillars of chess training described in this book, in decreasing order of importance, are:
  •  The List of Mistakes - analyzing your own games and categorizing the mistakes.
  • The Woodpecker Method - learning the tactical motifs and solving simple exercises to internalize them into your intuition.
  • Openings - studying them in such a way that you also learn middlegame positions and standard moves.
  • Theoretical Endgames - studying them only once."
My own eclectic program has a better chance of being followed this year, as I expect to have more time to concentrate on my chess.  What I would like to do is come up with a rotating list of chess resources (books, DVDs, significant videos/articles, etc.) that I complete fully before re-populating the list.  The list should be broad and contain opening, middle and endgame works, along with other general improvement topics.  Regular tournament-level play (online and OTB) and game analysis (both my own and master-level commentaries) will continue in parallel, along with tactics training via Chess.com's Tactics Trainer and the Chess Tactics Server.  I'll put up the next list once I have it together, as reference and to keep me honest by having a public goal.

No quick fix

Any improvement program requires significant time, dedicated effort and serious study.  The chess world is blessed with a large volume of books, software programs and online resources that allow us to make progress; in the end, however, it's up to us to do the necessary work (and play).  I would like to finish with the following quote from Grandmaster Versus Amateur, however, which gives a useful perspective:
"One happy aspect of being an amateur is that one can remain focused on the artistic side of the game, by not being in a permanent rush to win prizes and rating points."
So let's keep in mind the positives about being amateurs!

10 January 2015

Chess computing resources for 2015 - Part III: Game databases and e-books

In this last part of the series - following Part I on database analysis programs and Part II on chessplaying software - I'd like to look at some of the database resources available and discuss their importance for the improving player.  In addition to standard database files, typically containing very large collections of games in a proprietary or PGN format, there are now online, browser-accessible database references and a large number of e-books which are in reality database files with associated text and annotations.  We'll look at some examples from each category.

Database files

The first indispensable database for any player is one that contains your own games.  Unless you've been keeping separate online records of your games - something many chessplaying sites will now do automatically for you - you will need to manually enter them on your database program board.  This in itself can be a learning experience, so even if you have a stack of old tournament scoresheets lying around, that can be viewed as an opportunity rather than a chore.

I keep separate databases for my over-the-board (OTB) tournament games and slow online games completed at Chess.com (mostly from the Slow Chess League), since I prefer to split the different types of games.  As needed, though, I could easily combine them into a single database by copying and pasting the games into a new one.  The importance of having a personal games database is hard to underestimate, although I think many amateurs often do just that.
  • For the improving chessplayer, at minimum all of your tournament games should be recorded and analyzed to determine what you did right, as well as where you went wrong, and what general and specific lessons you can take away from each game.  Aside from games decided by rare one-move blunders in the opening - which may also be worth remembering for the future - each game then becomes an opportunity for in-depth chess study and research.  See "Analyzing your own games is more than analyzing your own games" for more on this approach to training.
  • Personal databases allow you to generate statistics on your performance at the push of a button.  Even basic stats such as your won-loss record with each color, score and performance rating by opening, and score against different rating categories can lead to new insights and identify unexpected weaknesses (or strengths).
  • Your personal database should be one of the centers of your preparation for future tournaments/games, especially if you have regularly recurring opponents, opening lines, or middlegame and endgame themes.  Understanding and identifying commonalities in past games, including solutions that you reached in subsequent analysis, should guide your expectations for future play.  This helps ensure that the lessons you learn one day are not simply forgotten the next.
Next we have what most people automatically think about when they see the word "database", the very large comprehensive databases with millions of games.  These types of reference databases are fundamental for analysis, research and comparison purposes.  Along with quantity of games, however, quality of games is just as important, and having annotated games included is a major benefit.
  • For ChessBase users, the yearly "Mega" database is the gold standard, primarily because it includes a significant number of games annotated by world-class players.  You can easily search on just annotated games, for example, to get the most out of this feature.  The ChessBase "Big" database version is similarly large, but lacks the annotated games.
  • The other main commercial option, Chess Assistant, comes with a database with a similarly high number of games (over 6 million).
  • Free large-scale databases for download can be found via internet search.  I can't vouch for their quality, but one good place to start for resources is Chessopolis.  To a large extent the freeware function for reference databases has been taken over by browser-accessible databases (see next section).
  • Updating your main reference database can be done either commercially - both of the commercial packages mentioned above typically come with one year of free downloaded updates - or via The Week in Chess (TWIC), an outstanding freely available service to the international chess community.  TWIC has other news and features that are of interest.
The last category of databases I want to discuss is specific to a purpose and typically is created by a player interested in a particular topic, using a search on a reference database.  Some other prepackaged databases also fall into this category, including ones specializing in a particular opening, for example.
  • One of the most obvious "purpose" databases to create is one containing the games of a known future opponent.  Even for amateurs this can be easily done in online communities, where individuals' games are accessible and often downloadable in PGN.  I do this at Chess.com when preparing for Slow Chess League games, to see what openings my opponents use and to see something of their style of play.  Professionals of course do this on a much more intensive level.
  • One opening study method is to look at the games of a model player, someone who frequently played the opening with success on a professional level.  A database search, typically by ECO code, can easily create such a roster of games for review and analysis, as I did with the "Stonewall Hero" exercise.  IM Jeremy Silman also discusses the use of databases for opening preparation in a recent Chess.com article.
  • Specific databases for study purposes can be created based on almost anything that you can think of, including specific middlegame and endgame structures, subject to the limitations of the search functions in your database analysis software.
  • This blog includes PGN database collections of my published annotated commentary games (master-level) and personal games; download links are in the sidebar.
  • Blunder Prone has some recent comments on deliberate practice using databases.

Online databases

Browser-accessible, large online databases have proliferated and evolved to the point where they have become a genuine alternative to prepackaged, local database files.  Often they will allow you to download your search results in PGN format, which allows you to then annotate and manipulate the databases for study.  Because of the large number of database resources now available, I won't try to provide a comprehensive list, but rather a few selected places to start.
  • Chess Tempo is a site whose primary function is to serve as a training resource, with both basic (free) and commercial features.  For example, it has tactics training, endgame training, and a range of options for computer opponents (the topic of Part II).  However, it also has an online, updated databases of 2+ million games with advanced search and filter options.  Laurent S mentioned this in his comment on a previous post on computer resources.  
  • 365Chess.com is another site featuring a large (3.5+ million) searchable database that also has some training functions.  It is set up similarly with basic (free) and advanced (commercial) features, with PGN downloads and database creation among the latter.
  • On a simpler level, the Shredder site has opening and endgame online databases that also allow you to paste in a position (in FEN) for searching.
  • While Chess.com is not primarily a database site, its "Game Explorer" feature allows for basic database exploration and PGN download of games.
It's worth noting that the main commercial database products also now have "live" components, with ChessBase's LiveBase accessible to its customers.


The more sophisticated e-book products are, in essence, "purpose" databases where the main selling points are the narratives/text and annotations included with the games, rather than the games database per se.  This is not to say that simpler e-books, which are essentially digital copies of print editions, aren't useful; they significantly enhance both the portability and availability of chess products.  However, e-books that contain databases have a significantly greater potential for your own investigation and analysis, as you can more easily explore related ideas via included links and your own searches.  They also don't require you to have a physical chess set with you.
  • Everyman Chess is a leading e-book publisher and their extensive free sample downloads are an excellent way to "try before you buy".
  • ChessBase has a similarly wide variety of "Fritztrainer" training products that include video narration along with annotated databases.  The DVDs can be used either with a commercial ChessBase product or with a free viewer.  One of the first I purchased was Viktor Kortchnoi's autobiographical "My Life for Chess" which was both entertaining and useful.
  • While opening study is an obviously useful subject for e-books, they are also taking advantage of the format's applicability to everything from game collections of top players to classic training works like Capablanca's Chess Fundamentals.

What about training software?

While I've included references to a number of training resources and methods in this post and in the earlier parts of this series, I haven't looked at computer resources whose specific function is training.  I will, however, include some discussion of them in an upcoming post - "Chess Improvement Programs: Directed vs. Eclectic".

03 January 2015

Chess computing resources for 2015 - Part II: Chessplaying software and computer opponents

In contrast with Part I of this series, which looked at more specialized database analysis programs, Part II covers a greater variety of computer chess tools that include chessplaying software and computer opponents.  Many software packages include all functions - database, playing, analysis, and training - but typically focus on one category more than others.  Because of the great number of differences in features between different software packages, in this post I'll focus on common chessplaying functions and how an improving player can take advantage of them.  After that discussion is a review of some computer resources, including freeware and online.  A brief overview of popular engines is also included.

As a side note, it's been interesting for me to see how chess software has evolved away from dedicated chessplaying programs - an approach exemplified by the old Chessmaster series - to a more modular approach, one which splits the GUI (graphical user interface) and the chess engine.  Users now can easily mix-and-match, using their favorite engine(s) along with whatever engine is bundled with the software they use.  This splitting of functions has its pluses and minuses, however, as I think some of the focus on the actual playing experience has been sacrificed.  The performance differences between different engines also are not nearly as significant as they used to be, especially for chessplaying purposes. Personally, I've found it difficult to take computer opponents seriously, as I have trouble caring about the outcome of a game when it is not at least somewhat similar to tournament conditions.

Computer opponents - what are they good for?

All chessplaying software packages offer different options for handicaps, in which the engine's effective playing strength is reduced.  Different mechanisms are used to accomplish this, with varying results.  The simplest method is to use a fast time limit for the computer to move (1 second, 5 seconds, etc.)  Another simple weakening method is to not permit the computer to "think" on its opponent's time.  For the more basic engines and those used for online play in browsers, these are popular choices (sometimes the only choices) to help calibrate the computer opponent's strength.  For an improving player, I think using the quick time method can be detrimental for training, since it psychologically encourages you to think and move fast as well, rather than develop and employ a comprehensive and coherent thinking process.  However, if what you want is a casual game within a quick time limit, or are using blitz games for a specific training purposes such as opening familiarization, then fast time controls are certainly appropriate.

This brings us to the question of what computer opponents are good for, within the context of a training/improvement program.  I think that they are best viewed as "sparring partners" - a concept familiar from other individual combat sports, boxing being the most well known, but also including fencing and martial arts.  Good sparring partners should offer you several things:
  • The chance to get a workout - a mental one in the case of chess - allowing the player to practice and reinforce their skills and thinking habits under conditions similar to that of a real match.  This usually requires a "slow chess" time control (45 45, 60 5, or higher).  Since computer sparring partners will not care what the time control is set at, having nothing else to do, this is a bonus - as long as you have the necessary focus and patience yourself.
  • A level of opposition that is at or (preferably) somewhat above your own playing strength.  You should have a legitimate chance to beat your sparring partner, albeit with significant effort.  Running an engine at top strength would prove a frustrating and insurmountable hurdle, while beating up an overly handicapped opponent will teach you nothing.  This concept is related to effortful study, a process in which you constantly push your boundaries, bit without taking on tasks completely beyond your strength.  In the context of a computer opponent, once you start beating them more than 50 percent of the time, you should increase their strength.
  • Opposition that has different "styles" or combinations of strengths and weaknesses.  It takes different skills to face and defeat an opponent who always goes for aggressive attacks, versus one who is content to play quietly and set up a solid position.  Opening variety is also important, to become familiar with different position-types and avoid being surprised.
How you set up your "sparring partner" therefore will greatly affect your experience.  Your chessplaying software should have a variety of slower time controls (often which you can input yourself), handicap methods that you can understand and easily adjust, and ways of varying the computer opponent's play.  These latter two points deserve some additional attention, especially in the context of their training usefulness.  
  • Despite the wide variety of handicap settings available, computer opponents often will play in an unrealistic manner that detracts from the experience and makes training less effective.  For example, they may randomly decide to drop material, make obviously pointless moves, and the like.  This is unfortunately hard to avoid from the programmer's point of view when trying to find practical ways to make a program play weaker than it can.
  • Computer opponents without randomizing functions will play the exact same game every time, which is of very limited use for training (and not very enjoyable, either).  Make the computer opponent behave too randomly, however, and it becomes difficult to learn via iterative training and amass useful experience, especially with mainline openings and key common position-types.
Each person will have their own particular preference for what they are looking for in a computer opponent.  That said, I'll offer some practical guidelines:
  • If the software package has automated functions that address the above issues, use them.  For example, some offer a specific Elo rating setting for opponent strength.  In that case, you can set the opponent to be 100-200 points higher than your own rating and see how you do against it over a few games, then adjust the strength in future games as necessary.
  • The Fritz GUI in addition offers a "Sparring" mode for new games, with five level settings (from "Very Easy" to "Really Hard").  The program calibrates its play accordingly and will deliberately play a certain number of moves, based on the level setting, for which it can calculate a tactical refutation, but otherwise play a strong game.  I find this mode of play to be more realistic in simulating a "real" game, as the program does not simply give material away (or allow itself to be mated) without requiring significant effort from its opponent.
  • When a default opening book is included with a software package, there are usually settings that will force the computer to play "best" or "optimal" lines, or alternatively force more variety in the opening choices.  It takes additional work, but can be even better for training purposes, when you edit or assemble your own opening book for the program to use.  Naturally using your own repertoire database is a possibility for this.
  • If you are doing tactical or visualization drills, then using extra features in a GUI such as displaying hints, threatened squares, etc. may be useful.  Naturally none of these aids are available during tournament conditions, so I would suggest not using any of these for serious training games.
Computer resources and packages

The following list is not meant to be comprehensive, but rather to give a sampling of the major programs available and types of resources, including online freeware, that can be found.  Comments on these, and on resources not originally included, would be useful to see; I'll incorporate them as appropriate in future edits.

The Deep Fritz 14 GUI is the latest version of the long-running Fritz series from ChessBase.  It has a full range of features for serious play, including a large number of different 2D and 3D virtual chess sets, as well as advanced analysis functions and some database functions.  For players looking for a high-end sparring partner and superior analysis capabilities, along with basic database resources, this GUI would be a good starting package.  For more advanced database functions and software, see Part I.  It's worth noting that ChessBase sells the GUI under the name of the engine bundled with it, so for example the latest Komodo 8 product is simply the Deep Fritz 14 GUI with Komodo 8 included as the primary engine.  Other UCI engines (see below) can be added easily to the GUI.

The Aquarium program - the latest release being Aquarium 2015 (and an additional version with Houdini 4 bundled) - is another chess package that includes a full range of analysis and playing functions, although its GUI and functionality is built primarily around the analysis functions.  Published by ChessOK, its database also uses the same proprietary format as their Chess Assistant database software.  Aquarium 2015 currently appears to have some issues with its deep analysis (IDeA) functions.

Shredder - as mentioned by Tommyg in his comment on Part I - remains a commercial alternative GUI, notably being available across mobile platforms as well, and has some online browser-accessible features on the site.  The playing community seems to be a strength of the product. Hiarcs Chess Explorer is another alternative all-in-one software package (thanks also to Tommyg for the comment on it).

The Chessmaster series, as mentioned above, was focused primarily on the playing experience, wth Chessmaster XI (also known as Chessmaster: Grandmaster Edition) being the last edition.  I previously recommended it as a sparring partner, since it had by far the best range of customizable opponents, as well as an outstanding video tutorial series included with it.  Unfortunately this product (from 2007) is no longer supported by software publisher Ubisoft, so I can't recommend it anymore.  I was originally able to install it on my Windows 7 machine with some difficulty, but eventually after one Windows Update cycle the program refused to work properly, even after reinstalling it multiple times.  If you have an older machine, it may still function.

The SCID vs PC freeware database analysis package was mentioned in Part I and it also includes some basic chessplaying functions.  There are other free GUIs available that are more developed around the playing and analysis functions, for example the more complex Arena Chess GUI 3.5 or the simpler Tarrasch Chess GUI.

Chess.com has a browser opponent available, or you can play against different bot opponents in its live section.


I've saved the section on engines for last, in part because for playing purposes they are now so strong that differences among top engines are essentially irrelevant.  Your choice of handicap options will be much more important in determining the type of playing experience you have, rather than which Elo 3000+ engine your computer is running.

For analysis purposes, engines may still show some significant if not large differences between them, at least when there are no concrete tactical advantages.  So it is from the analysis use perspective that I'll offer a few comments, along with the top engine home page links.  All of them can be downloaded directly from the links in their UCI versions, which you can then plug into your preferred GUI.  As an alternative, some intentionally weak engines have been developed for playing purposes - they may be included in some software packages - and may prove useful as sparring partners without having to handicap them.
  • Komodo 8 is what I currently use as an analysis engine.  It has an interesting history and appears to incorporate a greater degree of human positional judgment than its peers, as discussed in this ChessBase news article.  It is this latter quality that was the deciding factor for me and it seems legitimate.  In using Komodo 8, I've noticed that it gives (slightly) greater evaluation weight to compensation for sacrificed material and other positional factors than other engines; computers historically have been too materialistic, although that's changed to a great extent.
  • Houdini 4 is the latest version of the Houdini engine series.  I used the (freeware) Houdini 1.5a and (commercial) Houdini 2.0c previously for analysis engines.  Houdini appears to be somewhat faster that Komodo 8 in its evaluation functions, which remain top-notch.
  • Stockfish (currently version 5) is the top freeware UCI engine and can hold its own at the Elo 3000+ level.  Using this as a freeware option means you are sacrificing little (if any) power and functionality and it has downloads for a variety of platforms, including mobile apps.
In Part I, I discussed the practical value of using engines for game analysis, as part of an integrated process involving database analysis software (and of course your own brain).  This is a legitimate use of a tool to make progress.  What should be avoided is the practice of using software to replace (rather than augment) your brain, as sometimes occurs when people simply pop their games into a "full analysis" software feature, let it run, briefly see what the analysis results are, then move on.  A more detailed look at the pros and cons of computer analysis can be found in "Pitfalls of Computer Analysis" on this blog.  A more exaggerated (if entertaining) view of the harm relying on computer engines can be found in IM Jeremy Silman's recent article on Chess.com.  Coincidentally, a recent article on the "Rise of the Machines" at The Chess Improver also touches on these and some other related issues.

The future

All of these programs will no doubt continue to evolve over time, with new entries in each subcategory.  I think from a functional perspective the logical path would be to combine top-level database, analysis and gameplaying functions in a single software suite.  ChessBase could do this now with relative ease, I suspect, but keeping ChessBase and Fritz products separate is a way of segmenting the market for additional revenue.  ChessOK has a similar type of split with its Chess Assistant and Aquarium products.

All of the major packages (even freeware) now include database analysis functions, so deliberately keeping major playing/analyzing functionality separate from database functions appears more and more artificial.  I'm still hoping, even after a poor experience with the first version of the Chess King program, that a single and economical software package will eventually emerge that has an aesthetic GUI, engaging and varied computer opponents, good analysis functions, and powerful database options.  With the engines now separated from the GUIs and at a very high strength level, perhaps programmers can turn more attention to the gameplaying experience and thinking through how best to present and display analysis options and results.

Given the quality of the freeware available today, I recommend that people try out different available programs before purchasing a new commercial package.  Even if you do decide to go the commercial route, you'll have (for free) a better idea of what features you desire most, and/or what you really don't like, something which I think is central to effectively using computer tools in a training program.

The next post in this series is Part III, which looks at database and e-book resources.

29 December 2014

Chess computing resources for 2015 - Part I: Database and Analysis Software

This starts a series of posts that will be, in part, an update of previous ones on computer-related resources.  In this post, I intend to review chess database and analysis software functionality in a more in-depth manner, from the point of view of an improving club player.  My primary aim is to provide a reasonably comprehensive guide on how improving players can use these computer tools in a practical manner.  I therefore focus on how they can assist game analysis and related studies, which I've made the centerpiece of my own improvement program.

Online chess game publishing is a separate but related topic that I think deserves its own future update, although the last post on the topic from 2013 remains useful as a starting point.

Companion posts in this series on other types of computing resources can be found here:
Part II - Chessplaying software and computer opponents
Part III - Game databases and e-books

Database and Analysis Software

This category of computing tools, which use game databases for training, study, research and analysis purposes, has evolved to the point where I believe it can and should be employed by all serious chessplayers, not just the pros.  The broad range of functions now included in software packages cover just about everything you would want to do in chess analysis and preparation.

Although some database programs offer options for computer opponents along with analysis functions, that is normally not their primary focus, so I've decided to break that category out (along with a more detailed discussion of computer engines) into a separate post on chessplaying software / computer opponents.  Finally, in a third post I'll include a list of some game database resources and discuss the use of chess e-books (that in reality are annotated databases) and related software.  Naturally, individual software packages can contain all of the above categories of chess resources.

Below we will explore various types of database functions and highlight their possible utility for the improving player.  In large part, I'm doing this in order to help expand my own training tools for the new year by making myself smarter on the available options.  That said, I hope readers may also find the walkthrough and function review helpful.

Because ChessBase 13 (CB13) was just recently released and is popularly viewed as the "gold standard" for database software, I'm using that as the primary reference point for examples.  Other packages (including the two listed below) may provide similar functionality in many respects and for less money (or even for free, in the case of SCID vs PC).  Because of the quality freeware available, there is now no barrier to entry for any serious player (of whatever grade) to have, at minimum, a database of their own collected games available for analysis, along with a large database of master-level games for study, comparison and research.
  • Chess Assistant 15 - this package from ChessOK is the main commercial database alternative to CB13, with its own proprietary format, although both it and CB13 can also work with PGN files.
  • SCID vs PC - this is a full-featured freeware database analysis package (including the Stockfish engine and pre-loaded opening books) that also has some extra features like a FICS client for online play.  Latest release version as of this post is October 2014, from the Sourceforge site linked above.
For organizational purposes, below I'll examine different database functions under the headings of opening, middlegame and endgame study, along with a final section on game analysis, which serves to put it all together.

A.  Opening Study

One of the fundamental tasks for a chessplayer is to keep a record of their opening repertoire, as it is created and evolved.  Although ChessBase has offered designated "repertoire" databases with the most recent versions, including two individual ones for White and Black with CB13, this type of database organization and use is not terribly intuitive, including for more advanced players.  The simple openings database system I use - SORDS - organizes different major variations into games, similar to the way openings books are divided into different chapters. This is something any database program can do for you.
  • Collect Openings is a CB13 function (see PDF manual p. 227) that is designed to be used on a database of your own games.  It will automatically create a new database that groups and merges games into different opening classifications, in effect showing you what your repertoire has been.  This seems like a powerful tool and could, for example, be used to generate your first repertoire database automatically.
  • Opening "books" are included in many database programs, can be generated from them, or downloaded as separate files.  CB13 allows you to generate them from a reference database (see PDF manual p. 191) or if you are online to access the "Live Book" which is always up-to-date.  A "book" gives you a tree of opening moves to follow, similar to printed references such as Modern Chess Openings (MCO), the Encyclopedia of Chess Openings (ECO, whose codes are now standard), Nunn's Chess Openings (NCO), etc.
Statistics are normally included along with book displays. For example, here is a screenshot of the LiveBook from CB13.  It shows the move (the opening move for White, in this case), number of games in the book with that move, the score (result) in percentage terms, average Elo of players playing that move, the date, and the book evaluation.  This is typical across database programs.  The CB13 Livebook also has "visits" and a percentage next to it, which shows what moves people have been looking at in the online book.

The "Reference" tab in CB13 provides a deeper look at the position on the board and is primarily used for study of the opening phase, as it automatically retrieves and compares all games in the database with the same position.  As an example, below shows Gelfand-Tomashevsky from the 2014 Baku Grand Prix commentary after Black's 6th move.  As with the book format, we can see the move list (starting with White's move 7), number of games in the database and score (from White's perspective).  It also includes the last time it was played in the database by a titled player, the "Hot" indicator (a bar that reflects more recent play by top players), and a list of the best players who have played the move.  (move list, % result from White's perspective, "Hot" games, best players)

In the second window, the database program provides the most popular and highest-scoring lines (showing number of games and percentage score) along with the line fragment.  This shows at a glance how the game can evolve from the position and points of departure.  The third window contains all of the database games (obviously with a scroll bar) and can be sorted by all of the column headers - White player, White Elo rating, Black player, Black Elo rating, result, date, and notation.  These functions are all quite helpful and will be discussed further in the Game Analysis section below.  All of the games shown can be brought up in a separate window by double-clicking them.

"Report - Opening Report" is a CB13 function that scans the reference database and sorts information into a new database with HTML text and links, essentially providing an automatically-generated document detailing the reference pane information.  While this can be quite helpful to have all laid out for you in once place, as the main moves and continuations are included in the output, along with links to high-level sample games, the function is not fully reliable.  For example, running the opening report on the game position above, from October 2014, does not include it or any of the other 2014 games in the database output.  I believe this is a quirk in how CB13 is querying the date field of the games, as I've noticed the 2014 games in the database (Mega 2014) have included a month in that field; the software appears not to be able to recognize that with this particular function.  This same quirk can affect column sorting by date, although the games themselves at least appear.

Search functions allow you to look up games by ECO code and whatever other criteria you would like to add.  For example, as part of my "Stonewall Hero" exercise, I pulled all of Artur Yusupov's games in the Stonewall (ECO code A90) up via the search function and copied them into a new database.
  • In the reverse process, most databases should allow you to look up the ECO code of the game - in CB13 this can be done via "Report - Opening Classification".  ECO codes can also be searched online at various chess sites (or even just via standard internet search such as Google, for example "ECO code Classical Caro-Kann").
Identifying the game novelty (the move when it first goes out of the database) can be done various ways, including simply stepping through the game moves until there are no other database games showing in the Reference section.  The shortcut for this in CB13 is to use the Report - Novelty annotation function, which accesses the Live database for comparison purposes.  In the example above, Gelfand-Tomashevsky, the novelty is flagged as 12...c5 and the software automatically includes a predecessor game as a variation in the annotations, as well as a game in what it considered a relevant major variation for White on move 9.  More on the role of variations will be discussed in the below Game Analysis section.

B.  Middlegame Study

While openings are an obvious choice for database research, since there are normally so many other examples to draw on for each game analyzed, the middlegame should not be ignored either.  Looking at similar structures, tactics and maneuvers across different games is an excellent way to identify and better understand position-types that are common in your games.

In CB13 the user has to manually activate the "themes" for database sorting, but once that is done (File - Options - Misc - Activate theme keys), they will automatically appear for databases in ChessBase format.  There are four tabs with different theme categories: a general one, tactics, strategy and endgames.  What appears there will depend on how (or if) the games have been modified or marked.
  • "Themes" tab contains a number of different miscellaneous categories, the most useful of which (if you have an annotated database like Mega 2014) is probably "Commented games", which contains games with comment symbols and/or text broken down into a number of subcategories such as "Initiative", "Attack", etc.  Individual historically important players have their own subcategories, for example in Mega 2014 Fischer has 162 commented games.
  • "Tactics" tab provides broad categories ("Attack", "Defense", etc.) rather than examples of specific tactics - see below for a more specific search function - but also includes games with training questions.  In Mega 2014 there are 3732 games with training questions (2088 commented) included.  Because CB13 allows you to edit any game to include training questions, this could be a very useful tool for assembling your own computer-based "flashcards" for tactics, using your own games or those of others as a base.  Training questions can also be used in an opening repertoire database, as a means of testing your recall and understanding of positions.  The questions can also be toggled on or off in the main menu, so if you prefer to see the whole game at once it's not an obstacle to have the questions included in database games.
  • "Maneuver search" (PDF manual p. 88) is the way you can look for a particular tactic across different games using CB13.  You need to follow the manual's instructions for the search, but it allows you to do some powerful things, for example look for all bishop sacrifices on h7 while automatically mirroring tactics for the other side (in this case Black bishop sacrifices on h2).
  • "Strategy" tab contains a large number of categories and is much more developed than the corresponding tactical themes.  For example, there is a detailed breakdown of pawn structure types and different material imbalances.  This offers an easy way of researching both classic and recent games with particular middlegame features, for example an isolated White pawn on d4.
  •  "Report - similar move" (PDF manual p.168) - "With one click the program finds games with similar pawn structures where similar moves were made."  When analyzing games (see below), this is a particularly interesting feature.
  • "Report - similar structures" (PDF manual p.167) - "With one click the program loads all the games with similar pawn structures from the database and sorts them by similarity to the current game. The program also takes account of the positioning of the pieces (rooks on open rows, queen and bishop on the same diagonal, the position of the kings)."  This is also a particularly powerful comparative feature when analyzing games and can benefit a serious student, one who wants to emulate Benedict of Amber and observe how similar battles are fought.  Both this and "similar move" do appear to work properly, in my limited testing.

C.  Endgame Study

Endgame themes lend themselves well to database analysis and comparison.  The "Endgames" tab contains a lot of different subcategories, for the most part broken down by material, already populated.  Below I've expanded the Pawn Endings and Rook vs. Other Pieces endings as examples.

  • "Report - Similar endgames" is a CB13 function that can be called from a specific game (PDF manual p.167) - "The games are sorted by similarity. The program takes account of pawn structures (passed pawns, blocked pawns, connected pawns, chains, isolanis, backward pawns) and the relevant positioning of the pieces (rooks behind passed pawns, rook is cutting the king off, king in a square, wrong bishop, etc)."  Again, this type of function is useful for your own game analysis, especially if you want to take it to a deeper level or to get a quick idea of how other players have handled similar situations.  This can, among other things, help identify winning ideas and plans.
  • Searching for material distribution in an endgame is a more basic function, but if your database software lacks automatic report generation, then it can serve a similar purpose.

D.  Game Analysis: putting it all together

For the type of game analysis I normally do for training purposes, I take a balanced approach between doing only a superficial review and a deep probing of my games and selected master-level ones of interest.  For a more detailed discussion of the benefits of the analysis process, you can follow the above link; here I'll focus on the utility of using database and analysis software to enable the process.
  • Commentary / "your thoughts here" - any software that allows you to create and maintain a database of your own commented games is an invaluable study tool.  Putting your thoughts down (in terms of text commentary, symbols, and variations) shortly after each serious game you play is fundamental to better understanding your strengths and your areas needing improvement.  Normally I do this on a fairly superficial level (taking 15-20 minutes) as a first pass on analysis of one of my games, to identify my thinking process, evaluations during the game, and key variations that were considered but not played.
  • Openings / "what would Grandmasters do?" - the next step in analysis is stepping through the opening moves with the "Reference" tab open.  This tells me how far the game follows the main line (as indicated by the majority of database games at the master level), if there are major move branches or only one overwhelming preference by professional players, and helps capture transpositions that only looking at the move sequence would ignore.  Some of the most useful additional functions are looking at what the strongest players for your side do (sorting by Elo) and also sorting by date, to see what is currently considered best (or at least most popular).  Other games can easily be copied into your game as variations, either for temporary or permanent reference.
  • Variations / "choices choices choices" - using the software, you have a free hand to put down all of the options that occur to you at each point and then explore them as much as you want.  While the same type of thing can be done using a physical board - and this has its uses for training - the virtual board gives you much greater efficiency in moving between the stem position and all of the various branches, as well as recording your thoughts.  It's important to be curious about what works and what doesn't in a position (and why).  Something that made an impression on me was a quote from GM Nigel Davies, as blogger Robert Pearson used to cite, "...move the pieces around...Grandmasters do this, amateurs don't."  In other words, set things up for yourself and work it out.
  • Engine analysis / "evaluate" - there has been a good deal of silliness written about the horrible evils of using chess engines in training, on the one hand, and their infallibility, on the other hand.  "Pitfalls of Computer Analysis" contains some more detailed thoughts on this topic.  To my mind, the best use of engines, when loaded into database software, is to provide a very strong objective evaluation of the position in front of you.  One trick even top grandmasters like Kramnik use is to load an engine and have it display its current evaluation, but not the moves under consideration, so they can work it out for themselves.  The major caveat for improving players using an engine for assistance is that, if you cannot understand its evaluation in the absence of concrete tactics, it will not be of much use to you from a learning perspective.  However, if you can at least describe the major positional factors at play, a top engine can help guide you in better understanding things like the value of compensation, space advantages and structural strengths/weaknesses, especially as you step through variations to see their final outcomes.  Engine analysis normally is of significant assistance in identifying key turning points in a game, along with highlighting missed opportunities - especially ones which would simply never occur to you.
For more in-depth computer analysis, CB13 includes the "deep analysis" function, which is best left running for a lengthy period of time on a game and then saved as a different game from the original one.  It also goes a step further with programmed multiple "analysis jobs" which let you queue up position analyses with various advanced parameters for your engine(s).  This type of approach was first included a few years ago in the Aquarium software package (which will be referenced in the next post in the series) as "IDeA" projects.  These types of functions in practical terms will be of interest more for the professional or correspondence player, or people who are more interested in very deep analysis of individual games; I have not done more than give the functions a cursory look.

In Part II of this series, I will take a closer look at chessplaying software and engines, including some additional thoughts on the practical value of computer opponents and analysis for the improving player.

Below are some additional references for CB13 functions:
All of the above are from the ChessBase site and of course one must keep in mind that they are designed to help sell the product.  The individual authors' assessments of the product nevertheless seem to be valid; I would consider that the first article is the most candid in that respect.

20 December 2014

Last Chess.com tournament of 2014: A cautionary tale in the Caro-Kann; a patient victory in the English

I recently finished the Slow Swiss #19 tournament (90 30 time control) run by the Dan Heisman Learning Center over at Chess.com.  The following two games I think are the most instructive from the event, which was quite popular.  Although games played via the computer can't fully replicate the over-the-board experience, I find the slow time control games offered at the Slow Chess League to be the next best thing, especially for training purposes.

The first game below is a rather quick loss for me in a Caro-Kann, the mainline Classical variation; I think it can serve as a cautionary tale for other Caro-Kann players on what to avoid.  My opponent rattled off all of the moves through around move 13 without any thought, showing a strong familiarity with the opening, and rather quickly played the best database scoring moves through move 16.  At that point, I made a serious error and my opponent then played all of the strongest engine moves through move 26 - at which point White had over a +10 engine evaluation - showing precisely how to exploit my weaknesses on the kingside.  For the Black side, this game is primarily a one-move lesson (that Black needs to play ...Qd5 at a certain point), but it also serves an excellent illustration for the White side on how to conduct an attack in this situation.

The second game, presented below, is a patient win for me in a slow-developing English Opening, in which my opponent starts with "The Sniper" formation then goes for a double fianchetto.  The game is quite even for a long time, with a few small imbalances, and earlier in my chess career I would have been impatient to reach a draw, or perhaps try too hard for a win.  Here I was able to think in terms of prophylaxis and in improving my own pieces until my opponent made a relatively minor error, which then allowed a critical breakthrough on the queenside leading to a won endgame.  Patience is, I think, a feature of improved mental toughness.  (One shortcut to this, I think, is also keeping in mind Bobby Fischer's mantra of "no draws!")