Prerequisites: 3.8a6315, Compiler (version 19)
This package adds a closure compiler to the standard Squeak image without replacing the existing compiler. A new 'general' preference called #compileBlocksAsClosures determines which compiler to use for NEW methods only. When true the closure compiler is used, when false the existing compiler is used (default). Block contexts will still be created by existing methods.
Etoy scripts/tiles do not work yet with the closure compiler so be sure to use them with the above preference set to false.
The closure compiler is a total rewrite of the Smalltalk compiler using the SmaCC parser generator and the Refactory abstract systax tree (www.refactory.com). A special thanks goes to John Brant and Don Roberts for creating both the Refactory Browser and Smacc, and to Daniel Vainsencher and Markus Gaelli for porting them, respectively, to Squeak.
For more details on the compiler organization itself see the Compiler class comment. Below is a description of how the compiler creates block closures.
Block Closures
Each non-inlined blocks has its own CompiledMethod. These block methods are held in the literals of the home method and sent the #createBlock: message at runtime to create BlockClosures. Home method temps captured by inner blocks are placed inside a ClosureEnvironment when the home method is started. This environment is supplied as the argument to each #createBlock:. When #value... is sent to a block closure, its method is executed in a new MethodContext with its closure environment as the receiver. The block method accesses its free variables (captured home temps) via this environment.
Closure environments are nested mirroring the nesting of blocks. Each environment points to its parent environment (the top method environment has no parent). However, for efficiency, environments that have no captured temps are skipped (never created). For example, an environment's parent may actually be its grand-parent. There is no special parent variable in ClosureEnvironment, it is just another named variable such as 'self' or 'parent env' (special var with space so it can't be referenced by user code), or it may not be their at all.
A block closure that returns to its home context does so by finding the thisContext sender that owns the top environment. A return inside a block forces the home environment to be created even if it has no captured temps. Each context holds its local environment (which holds its captured temps) in its #myEnv instance variable (previously the unused #receiverMap variable). Code that references captured temps goes through the #myEnv context variable.
Block closures are totally separate from their home context. They are reentrant and each activation has its own block-local temps. So except for the thisContext psuedo-variable, contexts are now LIFO (assuming we get rid of old block contexts and recompile the whole image).
While block contexts reuse the same context on each activation, block closures create a new context each time. This makes block closures slower, until we speed up general activation. For an example of its slowness (and general slowness of context activation) execute '[(1 to: 1000000) do: [:i | i]] timeToRun' under each compiler (toggle the preference). On my Pentium II its about 50% slower. This pathological case should not discourage you. It does nothing in the block, so the context creation dominates. In practice, real work will be done inside each activation, and also, blocks are not as common as straight methods. Running macro-benchmarks you will not see such a noticable difference.
Currently, block closures execute using primitive 188, which requires arguments to be packaged in an array, create an array on each activation. Primitives 186 and 187 have to be rewritten (they were written assumming the old closure form of shared temps instead of a single environment), so we can use them directly without creating the arg array. The above microbenchmark (50% slowdown) was done after primitive 186 and 187 were working, so it will be even worse until you get them in.
Since blocks access fields inside environments it would be nice to have bytecodes that can access any object's instance/index variables intead of just the receiver's (or be forced to send #at: or #instVarAt:). See PotentialNewBytecodes.txt for recommended additional bytecodes that would reduce code size and potentially improve execution speed.
Test Cases
Thanks to Boris Gaertner and Rob Withers for writing closure test cases (BlockClosuresTestCase).