The Heijden database (version 6)

HHdbVI endgame study database is a collection of more than 93000 endgame studies in PGN format. We recommend this database for use with CQL (or, for that matter, to anyone interested in endgame studies).

The HHdbVI encodes metadata about the study in the PGN header and comments. The meaning of the metadata is described in HHdbVI reference manual.

hhdb introduction

The hhdb filter is an interface to HHdbVI (the hhdb filter will not work with earlier versions of the Heijden database).

An hhdb filter consists of the keyword hhdb followed by one or more hhdb keywords .

  hhdb sound //match sound studies
  hhdb twin //match twin studies
  hhdb mainline // use the <main> tag in the PGN to determine if current position is a main line

These hhdb keywords are not keywords in the usual sense: unlike ordinary keywords, hhdb keywords can be reused as variable names. These hhdb keywords are called parameters or arguments to the hhdb filter.

Some hhdb keywords have synonyms. One of these synonyms will derive directly from the HHdbVI specification, and the others are mnemonic. For example, the following two lines are equivalent:

  hhdb AN
  hhdb anticipation

hhdb keywords do not need to be enclosed in quotation marks except if the hhdb keyword contains a non-alphanumeric character (a parenthesis or angle bracket).

  hhdb "(c)" //study is a correction
  hhdb correction //"(c)"
  hhdb "<main>" //alternate main line

The full list of allowed hhdb keywords follows. You can display them with

  cql -hhdb help

Table of hhdb filters

hhdb keyword/filter	Meaning
<cook>	previous move cooks the study
<eg>	solution ended with previous move, the rest is only analytical proof and may have alternate moves
<main>	previous move begins an alternate main line
<or>	alternate move similar to main line; minor dual
<minor_dual>	tagged minor dual
(c) correction	correction of earlier study
(m) modification	modification
(s) corrected_solution	corrected solution
(v) version	version (perhaps a correction or modification)
AN anticipation	study is fully anticipated by someone else
CR colors_reversed	colors reversed from original study
MC too_many_composers	some composers not in player field but rather in the first comment
PH posthumous	posthumously published
TE theoretical_ending	a theoretical ending, may not be a study
TT theme_tourney theme_tournament	study is from theme tournament
TW twin	twin study
U1 dual_at_move_1	second solution at move 1
U2 dual_after_move_1	extra solution after move 1
U3 white_fails	incorrect; white cannot meet the stipulation
U4 white_wins_in_draw	white wins in a draw study
U5 unreachable	initial position is illegal
unsound	study is unsound
composer	the composers of the study as a string
mainline	current position is a mainline, accounting for <main> tags
sound	study is sound
variation	current position is not a mainline, accounting for <main> tags
prize	prize, including special prizes
award	any award
hm	honorable mention
commendation	commendation
special	a special award
sort	sort the awards following
stipulation	stipulation specification as a string
max	the final place in a shared award
gbr	GBR code of the study, as a string
gbr kings gbr material gbr pieces gbr pawns	specified substring of the GBR code of the study
search	the Black, White, Event tag values in the PGN header, and the first comment; returned one-per-line in a string
firstcomment	comment before move 1, if any
egdiagram	numeric value of the diagram number in EG of the study, if any
diagram	diagram number of the study, as a string

An hhdb filter followed by a quoted string means search

If an hhdb filter F has a value that is a string, then the filter F can be followed by a string literal:

  hhdb composer "Kasparyan"
  hhdb firstcomment "U1"
  hhdb stipulation "mate"
  hhdb gbr kings "e4"
  hhdb diagram "XII"

These are implicit search filter: the quoted string is searched for literally within the value of F :

  hhdb composer "Kasparyan"
  ≡
  "Kasparyan" in hhdb composer

Use the ~~ operator to perform regex searches

To search using regular expressions, preface the regular expression with ~~:

  hhdb composer ~~ "Gri.*k"
  hhdb composer ~~ "Tro[itszk]+i"
  hhdb firstcomment ~~ "U[1-5]"

All hhdb filters could change any of the capture groups, like \1 or \2, so these should not be relied on after an hhdb filter is evaluated.

Specific hhdb filters

This section discusses in more detail some specific hhdb filters

Awards: hhdb prize, hhdb hm, hhdb commendation, hhdb special

Study awards are of three major categories: prize, honorable mention, and commendation. Within each major category, an award can be special or not special. Each award has an associated rank or range of ranks (with a minimum rank and a maximum rank). If no rank is specifed, the minimum rank is set to 1 and the maximum rank to 10000.

For example, a clear second prize is written "2.p" in the database. This award has major category of prize, without a special designation, whose rank is 2, as are its minimum rank and maximum rank.

A shared 2nd through 5th special commendation would be denoted "2/5.sp.c" in the database. It has major category of commendation; is special; and would have minimum rank 2, maximum rank 5.

hhdb supports the major categories with keywords prize, hm and commendation. If x is a major category, then hhdb x matches the position if the study has an award whose major category is x:

  hhdb hm //study was awarded an honorable mention
  hhdb prize //study was awarded a prize
  hhdb commendation //study was awarded a commendation

The above three filters include awards with a special designation: hhdb hm would match a special honorable mention.

The filter hhdb award is true if the study was awarded any award.

If the special hhdb keyword is included with the major category x, then only awards of the major category x that are also designated special are considered:

  hhdb special commendation // special commendations
  hhdb special prize // special prize
  hhdb special award // any special award
  hhdb special // same as hhdb special award

Awards considered by hhdb

The hhdb award and similar filters only consider awards mentioned in the Event field of the pgn of the current study. This corresponds to awards that appeared with the first publication of the study.

Some awards appear only in the comment preceding the first move of the game, and these awards are not considered by hhdb award. These can be searched for using regular expressions or string search:

  "1.p" in hhdb firstcomment

is true if there is a first prize appearing in the first comment.

  hhdb firstcomment ~~ "\.[phc]"

will match an award in the first comment.

You can also use hhdb search, which includes both the Event and the first comment:

  "1.p" in hhdb search
  hhdb seach ~~ "\.[phc]"

These filters might have a few false positives, but will generally match any first prizes or any award respectively.

Numeric value of an award filter

Each of the following six hhdb award filters has an associated numeric value:

  hhdb prize
  hhdb special prize
  hhdb hm
  hhdb special hm
  hhdb commendation
  hhdb special commendation

The value of the numeric filter is the minimum rank of the matching award. If there is no such matching award then the filter does not match and consequently has no numeric value.

If special is not specified, then special awards are not considered when the numeric value of the hhdb filter is used.

For example,

  x=hhdb prize

will set the numeric variable x to the minimum rank of the study's award, but will fail to match if the study was awarded no prize or was awarded a special prize.

To obtain the rank of a special award, use the special keyword:

  y=hhdb special commendation

This will set y to the minimum rank of the study's award, if that award was a special commendation. If the award was not a special commendation, the filter will not match the position.

Getting the maximum rank of an award

To use the maximum rank instead of the minimum rank in a numeric hhdb award filter, use the max keyword:

  hhdb prize max<=1

will match all studies with clear, unshared, nonspecial, first prizes explicitly denoted as such. This would not match a prize 1/2.p because its maximum rank is 2. Likewise, it would not match a prize .p because that maximum rank is set to 10000.

sorting studies by award

If the sort keyword immediately follows hhdb and precedes award categories and special designators, then the matching studies are sorted in descending order as follows: prizes, special prizes, honorable mentions, special honorable mentions, commendations, special commendations. Within of these six categories, awards are sorted first by minimum rank, then by maximum rank:

  hhdb sort award // sort all awards

and

  hhdb sort prize //sort all prizes

Do not use more than one hhdb sort filter in the same cql file.

The following CQL sorts all non-special Matous awards:

  hhdb composer "Matous"
  not hhdb special
  hhdb sort award

Note that sort hhdb prize is different from hhdb sort prize. sort hhdb prize is a standard sort of the numeric filter hhdb prize. As a numeric filter, hhdb prize will not match special prizes. The differences are summarized in the table below, although these all follow from earlier definitions.

hhdb sort prize	sort hhdb prize
matches special prizes	does not match special prizes
sorts by the minimum rank then the maximum rank	only sorts by the minimum rank
sorts in ascending rank order	sorts in descending rank order

This finds the earliest sound studies before 1900 that have won a first prize:

  hhdb prize == 1 //first prize
  hhdb sound
  sort min "Year of study" year<1900 

hhdb keywords inside the Black player field

Each term x, like TW or U1, which would arise in the Black field in the PGN header corresponds to the CQL

player black ""x""

(Technically, this filter would be preceded by notransform, since it is unaffected by transforms).

For example, hhdb PH is equivalent to

player black "PH"

hhdb sound

hhdb sound is true of a study if the <cook> tag does not appear in the comments, and if the study is not designated U3, U4 or U5 by the database. This will include studies with minor duals, even on move 1, because CQL tries to avoid false negatives, and the significance of a minor dual is sometimes subjective.

hhdb unsound

hhdb unsound is true of a study if the study is designated either U1, U2, U3, U4, or U5 by the database. This includes all studies with <cook> as a comment. In Note that studies with only minor duals will be considered unsound, even if not everyone would consider these to be unsound, because CQL tries to avoid false negatives. Thus, to only see studies without any 'U' indications in the database, use

  not hhdb unsound

hhdb cooked

hhdb cooked is true of a study if the <cook> tag appears in the comments to any move. This filter searches moves in variations even if the variations tag is not set in the CQL header.

hhdb variations; hhdb mainline

A position arises from a move that starts a new (ordinary) variation if

move previous secondary

is true in the position (See secondary ). Therefore, a position is in an ordinary variation if

find <-- move previous secondary

is true.

But positions tagged <main> are never considered to start a new variation when following the HHdbVI conventions. Therefore, a position starts a variation under the HHdbVI conventions when the following filter matches the position:

  move previous secondary and not hascomment "<main>"

Hence, hhdb variation is equivalent to

  find <-- move previous secondary and not hascomment "<main>"

In other words, the position has an ancestor that is (a) an ordinary variation and (b) not commented with the tag <main>.

Finally, hhdb mainline is equivalent to not hhdb variation.

hhdb composer

hhdb composer is the composers as listed in the database, their names separated by spaces. This filter by default uses the information in the White PGN header field. If MC occurs in the Black PGN header field, which indicates additional composers, then this filter uses the information in the first comment to the game to extract all the composers.

For example, a study composed solely by Gady Costeff will have

  hhdb composer == "Costeff=G"

That is, the value of the hhdb composer filter will be the string "Costeff=G". To find studies by Costeff, use

  hhdb composer "Costeff"

This is a shorthand, because of the hhdb implicit search rules, for

  "Costeff" in hhdb composer

to seach for composers whose names begin with C and end in ff, use

  hhdb composer ~~ "C.*ff"

Guy-Blandford-Roycroft codes: hhdb gbr

The value of the GBR code of a study is hhdb gbr:

  hhdb gbr == "+0723.12f5g7" // example

Here, the first character indicates win or draw; the next 4 digits indicate the queens, rooks, bishops and knights (1 for a white piece; 3 for a black piece). Following the period, the next two digits indicate the numbers of white and black pawns; and finally two squares denoting the white and black king positions.

Since hhdb gbr is a string, it can searched for using hhdb implicit search:

  hhdb gbr "0723"

or regex search

  hhdb gbr ~~ "07\d[3-9]"

hhdb gbr material; sorting by GBR code

If hhdb gbr is followed by the hhdb keyword material then only the material portion of the GBR code is returned (everything except the result indication and the king positions:)

  hhdb gbr == "+0723.12f5g7"
  hhdb gbr material == "0723.12"

If you want to sort by GBR code, you would normally sort by hhdb gbr material:

  sort "GBR material" hhdb gbr material

This performs a lexicographic sort, of the material portion of the GBR code.

hhdb gbr pieces

  hhdb gbr pieces
  ≡
  hhdb gbr[1:5]

It holds the 4 character string denoting the material of the non-King, non-pawn pieces:

  hhdb gbr pieces == "0723"

hhdb gbr pawns

This is the two-character string indicating the number of pawns for white and black:

  hhdb gbr pawns
  ≡
  hhdb gbr[6:8]

For example

  hhdb gbr pawns == "12" //1 white pawn, 2 black pawns

To get say the number of black pawns, use

  int hhdb gbr pawns[1]

hhdb gbr kings

The two king squares as a string are in the four character string

  hhdb gbr kings
  ≡
  hhdb gbr [8:]

For example

  hhdb gbr kings == "f5g7"

To get the white king square, use

  makesquare hhdb gbr kings[:2]

the black king square is:

  makesquare hhdb gbr kings[2:]

hhdb stipulation

Some studies have additional stipulations listed in the comment before the first move. The stipulation types are: "mate", "draw (50 move rule)", "shortest mate", "quickest win", and "ult".

The stipulation, as a string, is the value of hhdb stipulation:

  x=hhdb stipulation
  x=="mate in 50" // example

To get the numeric value of direct mates, use

  hhdb stipulation ~~ "mate in (\d+)"
  int \1

For example, to sort by longest mates, use:

  hhdb stipulation ~~ "mate in (\d+)"
  sort "Mate stipulation" int \1

The hhdb command line interface

All hhdb filters can be invoked from the command line. Quotation marks are not needed around arguments.

When invoked from the command line, hhdb filters that are strings must be followed by a command line argument which is interpreted as the hhdb implicit search argument to that file

  cql -hhdb composer Minski -hhdb sort award -input hhdbvi.pgn
  cql -hhdb sound foo.cql

The option -vi is short for -input hhdbvi.pgn.

Use -hhdb help for a list of valid hhdb keywords.