PGN parser in PowerShell

In order that Scid vs PC generates the Scid database of TCEC 2019 superfinal 14 STOCKFISH vs Leela Chess zero, let us introduce the PGN parser in PowerShell language for Windows 10.

The 5 MB of the TCEC PGN of the Superfinal 14 could be downloaded from the legacy TCEC on Chessdom:

« Season 14 – 11.12.2018″ > « Superfinal« : click the Save icon with the tooltip « Download PGN ».

This large PGN file is in CuteChess format at level of the comment. The goal of the PGN parser is to convert this PGN in Scid format (4 MB) so you can import the PGN and save it in standard .si4, .sn4, .sg4 (1 MB).

The PGN tags between « […] » and the Game Terminator « 1-0 » are unchanged (the output = the input).

Let us forget the UCI comment plenty of parameters for the engines before the chess moves.

Let us focus first on the Book Moves:

We use the PowerShell cmdlet ConvertFrom-String in the old fashion way just to split the chess move in two lines in order to separate the White move from the Black move.

Run the Windows PowerShell ISE (Integrated Scripting Environment) from the Start button of Windows 10: try « P » as PowerShell or the full name of the ISE.
After the prompt (for example « PS C:\Users\Chess> » with a dark blue background), enter one time by session the following policy:

Set-ExecutionPolicy -ExecutionPolicy RemoteSigned -Scope Process

To split the full chess move in two parts, we use the delimiter of the end of the first comment « } »:

ConvertFrom-String -InputObject "1. e4 {book} e5 {book}" -Delimiter "}" -OutVariable PgnOpening -PropertyNames WhiteMove, BlackMove

The Windows PowerShell automatically presents the answer as a table without to code anything more:

WhiteMove   BlackMove
---------   ---------
e4 {book    e5 {book} 

Pay attention of the missing delimiter « } » in the comment of the first column « WhiteMove » because « } » served as delimiter.

PowerShell help

Consult the help of the cmdlet ConvertFrom-String. You can dot it Inside the ISE but it is recommended to start another PowerShell prompt with administrator privilege:

Right-click the Windows Start button > contextual menu « Windows PowerShell (admin) » then enter your admin password.

Update your PowerShell help once:

Update-Help -Verbose -Force -ErrorAction SilentlyContinue

This command requires internet access and needs some seconds or minutes… Then you are able to ask for the wished help:

get-help ConvertFrom-String

Minimal PGN reader

The following minimal parser is able to read each PGN line and dispatch it in the appropriate section:

# pgnReader.ps1 reads a TCEC PGN in CuteChess format and displays it

$PgnInput = @'
[White "Stockfish 190203"]
[Black "LCZero v20.2-32930"]
[Result "1-0"]

{WhiteEngineOptions: Protocol=uci; Hash=16384; Threads=43; SyzygyPath=C:/Syzygy/; Move Overhead=1000; OwnBook=false; Ponder=false;}
1. d4 {book, mb=+0+0+0+0+0,} Nf6 {book, mb=+0+0+0+0+0,}
2. c4 {book, mb=+0+0+0+0+0,} e6 {book, mb=+0+0+0+0+0,}
3. g3 {book, mb=+0+0+0+0+0,} c5 {book, mb=+0+0+0+0+0,}
4. d5 {book, mb=+0+0+0+0+0,} exd5 {book, mb=-1+0+0+0+0,}
5. cxd5 {book, mb=+0+0+0+0+0,} b5 {book, mb=+0+0+0+0+0,}
6. Nf3 {d=36, sd=65, mt=583398, tl=6631602, s=51130154, n=29829281088, pv=Nf3 Bb7 e4 Nxe4 Bg2 Be7 O-O O-O Re1 f5 Nc3 Na6 Ne5 Bd6 Nxe4 fxe4 Rxe4 Nc7 Be3 Rf5 Bf4 Bxe5 Rxe5 Rxe5 Bxe5 d6 Bf4 a5 Qd2 Qf6 Re1 h6 h4 b4 g4 Re8 Rxe8+ Nxe8 Qe3 Kf8 g5 hxg5 Bxg5 Qf7 Qe6 Qxe6 dxe6 Bxg2 e7+ Kg8 Kxg2 Kf7, tb=1003, h=100.0, ph=0.0, wv=0.44, R50=49, Rd=-11, Rr=-11, mb=+0+0+0+0+0,}
Bb7 {d=24, sd=59, mt=78552, tl=7136448, s=53686, n=4214874, pv=Bb7 e4 Nxe4 Bg2 Be7 O-O O-O Re1 f5 Nc3 d6 Nxb5 Nd7 Nd2 Ndf6 Nxe4 Nxe4 Nc3 Nxc3 bxc3 Bf6 Rb1 Bc8 Bf4 g5 Bd2 Qc7 Qh5 Qg7 c4 Qg6 Qxg6+ hxg6 Rb3 Kf7 Bf1 Re8 Reb1 g4 Bf4 Be5 Bg5 Bf6 Bd2 g5 Bd3 Kg6, tb=0, h=26.1, ph=0.0, wv=0.97, R50=49, Rd=-11, Rr=-11, mb=+0+0+0+0+0,}
7. e4 {d=36, sd=66, pd=Bb7, mt=40211, tl=6606391, s=55390895, n=2227323315, pv=e4 Nxe4 Bg2 Be7 O-O O-O Re1 f5 Nc3 Na6 Nxb5 Nb4 d6 Bf6 a3 Nc6 Nd2 Nd4 Nxe4 Bxe4 Bxe4 fxe4 Rxe4 Rb8 Nxd4 Bxd4 Be3 Rxb2 Bxd4 cxd4 Ra2 Rfxf2 Rxb2 Rxb2 Qxd4 Qa8 Qc4+ Kh8 Qf7 Rb1+ Kf2 Rb2+ Kf3 Re2 Qe7 Rxe4 Qxe4 Qf8+ Qf4 Qc8 Qd4, tb=740, h=63.5, ph=100.0, wv=0.64, R50=50, Rd=-11, Rr=-11, mb=+0+0+0+0+0,}
Nxe4 {d=27, sd=58, pd=e4, mt=69857, tl=7081591, s=47744, n=7176784, pv=Nxe4 Bg2 Be7 O-O O-O Re1 f5 Nc3 d6 Nxb5 Nd7 Nd2 Ndf6 Nxe4 Nxe4 Nc3 Nxc3 bxc3 Bf6 Rb1 Bc8 Bf4 g5 Bd2 Qc7 Qh5 Qf7 Qxf7+ Kxf7 Bf1 f4 gxf4 Rg8 Bg2 Bf5 Rb7+ Kg6 fxg5 Bxg5 Bxg5 Kxg5 Rf7 Kg6 Ree7 Rae8 Rg7+ Kh6 Rxg8 Rxg8 Kh1 Rg7 Re8, tb=0, h=46.6, ph=100.0, wv=0.88, R50=50, Rd=-11, Rr=-11, mb=-1+0+0+0+0,}
1-0
'@

$PgnInput -split "`n" | ForEach-Object { # means For each line of text in the PGN where $_ represents the current line
    if ($_ -match "^\[.*\]"){ # Reproduce pairs of [tag-name "tag_value"] without any changes
        Write-output($_) # Optionnally, you can put a "#" before "Write-output" to disable the generation of the wished section as in the following:
    }
    elseif($_ -match "^{[^}]*}"){ # Skip first comment  of UCI prms starting at the first column.
                    # RegExpr : ^first col. "{" + all except "}" + "}" see: get-help about_regular_expressions
                    #                             "[^}]*"
        #Write-output ("Delete first comment : " + $_)
    }
    elseif ($_ -match "\} "){ # During the opening book, both White and Black moves are on the same line
        Write-output($_)
    }
    elseif ($_ -match "^0-1" -or $_ -match "^1-0" -or $_ -match "^1/2-1/2"){ # Game terminator?
        Write-output($_)
    }
    else # Remaining moves in the middle game and the end game including empty line
    {
        Write-output($_)
    }
}

This generates the input without the UCI comment:

C:\Users\Chess\Documents\Powershell\pgnReader.ps1
[White "Stockfish 190203"]
[Black "LCZero v20.2-32930"]
[Result "1-0"]

1. d4 {book, mb=+0+0+0+0+0,} Nf6 {book, mb=+0+0+0+0+0,}
2. c4 {book, mb=+0+0+0+0+0,} e6 {book, mb=+0+0+0+0+0,}
3. g3 {book, mb=+0+0+0+0+0,} c5 {book, mb=+0+0+0+0+0,}
4. d5 {book, mb=+0+0+0+0+0,} exd5 {book, mb=-1+0+0+0+0,}
5. cxd5 {book, mb=+0+0+0+0+0,} b5 {book, mb=+0+0+0+0+0,}
6. Nf3 {d=36, sd=65, mt=583398, tl=6631602, s=51130154, n=29829281088, pv=Nf3 Bb7 e4 Nxe4 Bg2 Be7 O-O O-O Re1 f5 Nc3 Na6 Ne5 Bd6 Nxe4 fxe4 Rxe4 Nc7 Be3 Rf5 Bf4 Bxe5 Rxe5 Rxe5 Bxe5 d6 Bf4 a5
 Qd2 Qf6 Re1 h6 h4 b4 g4 Re8 Rxe8+ Nxe8 Qe3 Kf8 g5 hxg5 Bxg5 Qf7 Qe6 Qxe6 dxe6 Bxg2 e7+ Kg8 Kxg2 Kf7, tb=1003, h=100.0, ph=0.0, wv=0.44, R50=49, Rd=-11, Rr=-11, mb=+0+0+0+0+0,}
Bb7 {d=24, sd=59, mt=78552, tl=7136448, s=53686, n=4214874, pv=Bb7 e4 Nxe4 Bg2 Be7 O-O O-O Re1 f5 Nc3 d6 Nxb5 Nd7 Nd2 Ndf6 Nxe4 Nxe4 Nc3 Nxc3 bxc3 Bf6 Rb1 Bc8 Bf4 g5 Bd2 Qc7 Qh5 Qg7 c4 Qg6 
Qxg6+ hxg6 Rb3 Kf7 Bf1 Re8 Reb1 g4 Bf4 Be5 Bg5 Bf6 Bd2 g5 Bd3 Kg6, tb=0, h=26.1, ph=0.0, wv=0.97, R50=49, Rd=-11, Rr=-11, mb=+0+0+0+0+0,}
7. e4 {d=36, sd=66, pd=Bb7, mt=40211, tl=6606391, s=55390895, n=2227323315, pv=e4 Nxe4 Bg2 Be7 O-O O-O Re1 f5 Nc3 Na6 Nxb5 Nb4 d6 Bf6 a3 Nc6 Nd2 Nd4 Nxe4 Bxe4 Bxe4 fxe4 Rxe4 Rb8 Nxd4 Bxd4 B
e3 Rxb2 Bxd4 cxd4 Ra2 Rfxf2 Rxb2 Rxb2 Qxd4 Qa8 Qc4+ Kh8 Qf7 Rb1+ Kf2 Rb2+ Kf3 Re2 Qe7 Rxe4 Qxe4 Qf8+ Qf4 Qc8 Qd4, tb=740, h=63.5, ph=100.0, wv=0.64, R50=50, Rd=-11, Rr=-11, mb=+0+0+0+0+0,}
Nxe4 {d=27, sd=58, pd=e4, mt=69857, tl=7081591, s=47744, n=7176784, pv=Nxe4 Bg2 Be7 O-O O-O Re1 f5 Nc3 d6 Nxb5 Nd7 Nd2 Ndf6 Nxe4 Nxe4 Nc3 Nxc3 bxc3 Bf6 Rb1 Bc8 Bf4 g5 Bd2 Qc7 Qh5 Qf7 Qxf7+ 
Kxf7 Bf1 f4 gxf4 Rg8 Bg2 Bf5 Rb7+ Kg6 fxg5 Bxg5 Bxg5 Kxg5 Rf7 Kg6 Ree7 Rae8 Rg7+ Kh6 Rxg8 Rxg8 Kh1 Rg7 Re8, tb=0, h=46.6, ph=100.0, wv=0.88, R50=50, Rd=-11, Rr=-11, mb=-1+0+0+0+0,}
1-0

You can put in comment the « Write-output($_) » with a « # » in the beginning of the line in a section to see the impact on the generated PGN line.

---

The second part of this PowerShell tutorial will introduce the ConvertFrom-String cmdlet from the Artificial Intelligence point of view, that is to say how machine learning could generate automatically our PGN parser using some chosen examples.