Programming from the CQL command line

The basic CQL invocation from the command line looks like this:

cql foo.cql

or sometimes say

cql -i bar.pgn -o output.pgn -g 1 100 foo.cql

But you can use the command line to do simple searches and to modify existing CQL files. Doing this is sometimes quicker and easier than creating a whole new CQL file.

You can omit the .cql file and search directly for positions or position fragments:

cql -cql "Ra3 Kb1 pd3" -i data.pgn

This will find all positions in the file data.pgn with a position with a white rook an a3, a white king on b1, and a black pawn on d3.

The quotation marks here are used for shell metacharacter quoting: it makes sure that the "Ra3 Kb1" is read as one command line argument.

To avoid having to worry about shell metacharacters, you can also use multiple -cql options:

cql -cql Ra3 -cql Kb1 -cql pd3 -i data.pgn

You can search the file data.pgn for a particular position FEN notation via:

cql -fen 8/5kp1/p5N1/P4Pp1/6P1/PKP5/1B1r4/8 -i data.pgn

You can preface any of these filters by a transform. For example, to look for positions that match either

Ra3 Kb1 pd3

or the switching colors:

ra6 kb8 Pd6

use

cql -flipcolor -cql "Ra3 Kb1 pd3" -i data.pgn

The default CQL file

The most important CQL file you should know does not have a name but has these contents:

cql().

This is the default CQL file. When you run cql and do not specify, cql will use this file instead. Note that this file simply matches every position (because . matches any position).

How command line filters are handled

Ordinarily CQL reads the filters in the body of the CQL file and evaluates these filters at every position.

When there are command line filters (a command line filters is introduced either via the -cql option or the -fen option) then all the command line filters are collected into a new compound filter called the preamble. This preamble is inserted before the first filter of the body of the CQL file to form the new body.

Any command line transforms, such as -reversecolor and -flipcolor are applied to all subsequent filters, whether those filters are from the command line or from the body of the CQL file.

Thus, the CQL command line

cql -cql "Ra3 Kb1 pd3" -i data.pgn 

prepends the preamble

{Ra3 Kb1 pd3}

to the body of the default CQL file to get a new body:

{Ra3 Kb1 pd3}
	    .

Since the . matches every position, that body is equivalent to

{Ra3 Kb1 pd3}

Similar considerations apply to FEN and transforms.

Use of the default CQL file

A common situation to use command line arguments is when you see an interesting position, say in a web browser. You can select its fen and use the -fen option to search for the position in a database:

cql -fen 8/5kp1/p5N1/P4Pp1/6P1/PKP5/1B1r4/8 -i foo.pgn

If it is an endgame position, you can of course preface this with -flipcolor to all for colors to be swapped.

Transforms

The name of any transform, preceded by - is an option that transforms all following filters (specified either on the command line via -cql or -fen or in the CQL file):

	cql -flipcolor foo.cql
	cql -reversecolor foo.cql

Recall that the differences between flipcolor and reversecolor is that the former allows the original filter, and the latter requires a color transform. That is, the following two lines are identical:

flipcolor Ra2
Ra2 or ra7

as are these two lines

reversecolor Ra2
ra7

A common situation where one would want to use command line transforms is to quickly find piece configurations that are flipped or shifted in a database.

For example,

cql -cql "Bb1 Kb2 Pa2 Pc2" -i foo.pgn

will find the specified configuration of pieces. To find it shifted, and for either color, in any configuration, try say

cql -flipcolor -flipvertical -shift -cql "Bb1 Kb2 Pa2 Pc2" -i foo.pgn

Note here the output will go to cqldefault-out.pgn as usual.

As a tip, generally when pawns are on the board, it's not a good idea to use -flip, because that includes diagonal flips, which drastically change the pawn configuration. In this example, the -flipcolor allows for color interchange (which flips the board horizontally and swaps the piece colors); the -flipvertical allows for reflections about the vertical bisector of the board; and the shift allows the configuration to be shifted in any direction.

-matchcount

You can specify the matchcount as well at the command line using -matchcount followed by a range. In particular, you can specify the negation of a CQL file - matching all games that are NOT matched by the CQL file itself using matchcount 0.

For example, suppose you are experimenting with CQL definitions of pure mate.

You have a short and simple definition in puremate-simple.cql . You also have a more precise definition in puremate.cql . The file puremate-simple.cql allows certain mates as pure that puremate.cql does not.

Suppose you want to find out all games that reach pure mate according to the definition in puremate-simple.cql but not in puremate.cql.

You could write a separate CQL file for this, but a simpler way is as follows.

First, generate a file of the simple pure mates, without any annotations:

cql -silent -i mydatabase.pgn -o puremate-simple-out.pgn examples/puremate-simple

This will read games from mydatabase.pgn and output (to the file puremate-simple-out.pgn) those that conclude in a puremate according to the definition in puremate-simple.cql.

Now how many of these are not pure mates according to puremate.cql? Use

cql -i puremate-out.pgn -matchcount 0 -o non-puremate-out.pgn examples/puremate 

This reads in the games in the file puremate-out.pgn, the list of pure mates, and outputs the ones that are not also pure mates according to puremate.cql.

-mainline, -variations and -virtualmainline

You can determine on the command line whether CQL looks only in the mainline or in variations. This feature isn't useful for analyzing databases of competitive chess games, but it can be useful in studies. You can also look in so-called virtual mainlines using the -virtualmainline option, which is an experimental feature and only applies to studies, not competitive games.

Extracting a game with -gamenumber

This simple option is useful in debugging and other situations too.

Sometimes after running a CQL file, you will find that a game was matched that you didn't expect. You can read the output of the original CQL file to get the game number of that game in the original database, say 32773. You can then select just that game using

cql -g 32773 -o game32773.pgn -i originaldatabase.pgn -silent

This puts just game 32773 from the original database into its own file. You can then more easily and quickly run multiple versions of the CQL file you are trying to debug directly on that smaller pgn file. Note the -silent in the command line: that makes sure that just the original game, without any CQL added comments, is output. (In fact, if you omitted the -silent option, CQL would append CQL after every position, because the default CQL body is . ).

Another common use of -g is in the early stages of writing a CQL file. You might not be sure if there are 100 matches or 100000 matches in the database of some file, or you might be experimenting with a feature. You can easily just run your file on the first 10 games, or 100 games of the database:

cql -g 1 100 mycqlfile.cql