14.14 Linklets and the Core Compiler

► |  | The Racket Reference

1 | Language Model
2 | Notation for Documentation
3 | Syntactic Forms
4 | Datatypes
5 | Structures
6 | Classes and Objects
7 | Units
8 | Contracts
9 | Pattern Matching
10 | Control Flow
11 | Concurrency and Parallelism
12 | Macros
13 | Input and Output
14 | Reflection and Security
15 | Operating System
16 | Memory Management
17 | Unsafe Operations
18 | Running Racket
| Bibliography
| Index

▼ | 14 | Reflection and Security

14.1 | Namespaces
14.2 | Evaluation and Compilation
14.3 | The racket/ load Language
14.4 | Module Names and Loading
14.5 | Impersonators and Chaperones
14.6 | Security Guards
14.7 | Custodians
14.8 | Thread Groups
14.9 | Structure Inspectors
14.10 | Code Inspectors
14.11 | Plumbers
14.12 | Sandboxed Evaluation
14.13 | The racket/ repl Library
14.14 | Linklets and the Core Compiler

On this page:

linklet?
compile-linklet
recompile-linklet
eval-linklet
instantiate-linklet
linklet-import-variables
linklet-export-variables
linklet-directory?
hash->linklet-directory
linklet-directory->hash
linklet-bundle?
hash->linklet-bundle
linklet-bundle->hash
instance?
make-instance
instance-name
instance-data
instance-variable-names
instance-variable-value
instance-set-variable-value!
instance-unset-variable!
instance-describe-variable!
variable-reference->instance
correlated?
correlated-source
correlated-line
correlated-column
correlated-position
correlated-span
correlated-e
correlated->datum
datum->correlated
correlated-property
correlated-property-symbol-keys

top ← prev up next →

#### 14.14`` Linklets and the Core Compiler

(require racket/linklet) | package: base

A linklet is a primitive element of compilation, bytecode marshaling, and evaluation. Racket’s implementations of modules, macros, and top-level evaluation are all built on linklets. Racket programmers generally do not encounter linklets directly, but the racket/linklet library provides access to linklet facilities.

A single Racket module (or collection of top-level forms) is typically implemented by multiple linklets. For example, each phase of evaluation that exists in a module is implemented in a separate linklet. A linklet is also used for metadata such as the module path indexes for a module’s requires. These linklets, plus some other metadata, are combined to form a linklet bundle. Information in a linklet bundle is keyed by either a symbol or a fixnum. A linklet bundle containing linklets can be marshaled to and from a byte stream by write and (with read-accept-compiled is enabled) read.

When a Racket module has submodules, the linklet bundles for the module and the submodules are grouped together in a linklet directory. A linklet directory can have nested linklet directories. Information in a linklet directory is keyed by #f or a symbol, where #f must be mapped to a linklet bundle (if anything) and each symbol must be mapped to a linklet directory. A linklet directory can be equivalently viewed as a mapping from a lists of symbols to a linklet bundle. Like linklet bundles, a linklet directory can be marshaled to and from a byte stream by write and read; the marshaled form allows individual linklet bundles to be loaded independently.

A linklet consists of a set of variable definitions and expressions, an exported subset of the defined variable names, a set of variables to export from the linklet despite having no corresponding definition, and a set of imports that provide other variables for the linklet to use. To run a linklet, it is instantiated as as linklet instance (or just instance, for short). When a linklet is instantiated, it receives other linklet instances for its imports, and it extracts a specified set of variables that are exported from each of the given instances. The newly created linklet instance provides its exported variables for use by other linklets or for direct access via instance-variable-value. A linklet instance can be synthesized directly with make-instance.

A linklet is created by compiling an enriched S-expression representation of its source. Since linklets exist below the layer of macros and syntax objects, linklet compilation does not use syntax objects. Instead, linklet compilation uses correlated objects, which are like syntax objects without lexical-context information and without the constraint that content is coerced to correlated objects. Using an S-expression or correlated object, the grammar of a linklet as recognized by compile-linklet is

> > (linklet [[imported-id/renamed ...] ...]
> > [exported-id/renamed ...]
> > defn-or-expr ...)
> > imported-id/renamed |  | = |  | imported-id
> > |  | | |  | (external-imported-id internal-imported-id)
> > exported-id/renamed |  | = |  | exported-id
> > |  | | |  | (internal-exported-id external-exported-id)

Each import set [imported-id/renamed ...] refers to a single imported instance, and each import-id/renamed corresponds to a variable from that instance. If separate external-imported-id and internal-imported-id are specified, then external-imported-id is the name of the variable as exported by the instance, and internal-imported-id is the name used to refer to the variable in the defn-or-exprs. For exports, separate internal-exported-id and external-exported-id names corresponds to the variable name as exported as referenced in the defn-or-exprs, respectively.

The grammar of an defn-or-expr is similar to the expander’s grammar of fully expanded expressions (see Fully Expanded Programs) with some exceptions: quote-syntax and #%top are not allowed; #%plain-lambda is spelled lambda; #%plain-app is omitted (i.e., application is implicit); lambda, case-lambda, let-values, and letrec-values can have only a single body expression; and numbers, booleans, strings, and byte strings are self-quoting. Primitives are accessed directly by name, and shadowing is not allowed within a linklet form for primitive names, imported variables, defined variables, or local variables.

When an exported-id/renamed has no corresponding definition among the defn-or-exprs, then the variable is effectively defined as uninitialized; referencing the variable will trigger exn:fail:contract:variable, the same as referencing a variable before it is defined. When a target instance is provided to instantiate-linklet, any existing variable with the same name will be left as-is, instead of set to undefined. This treatment of uninitialized variables provides core support for top-level evaluation where variables may be referenced and then defined in a separate element of compilation.

Added in version 6.90.0.1 of package base.

> procedure (linklet? v) → boolean?
> v : any/c

Returns #t if v is a linklet, #f otherwise.

> > procedure
> > (compile-linklet |  | form |  |  |  |
> > | [ | name |  |  |  |
> > |  | import-keys |  |  |  |
> > |  | get-import |  |  |  |
> > |  | options]) |  | → |  | linklet?
> form : (or/c correlated? any/c)
> name : any/c = #f
> import-keys : #f = #f
> get-import : #f = #f
> | options |  | : |  | (listof (or/c 'serializable 'unsafe 'static 'no-prompt))
> |  |  | = |  | '(serializable)
> (compile-linklet |  | form |  |  |  |
> |  | name |  |  |  |
> |  | import-keys |  |  |  |
> | [ | get-import |  |  |  |
> |  | options]) |  | → |  |
> linklet? |  | vector?
> form : (or/c correlated? any/c)
> name : any/c
> import-keys : vector?
> | get-import |  | : |  |
> (or/c #f (any/c . -> . (values (or/c linklet? instance? #f)
> (or/c vector? #f))))
> |  |  | = |  | #f
> | options |  | : |  | (listof (or/c 'serializable 'unsafe 'static 'no-prompt))
> |  |  | = |  | '(serializable)

Takes an S-expression or correlated object for a linklet form and produces a linklet. As long as 'serializable included in options, the resulting linklet can be marshaled to and from a byte stream when it is part of a linklet bundle (possibly in a linklet directory).

The optional name is associated to the linklet for debugging purposes and as the default name of the linklet’s instance.

The optional import-keys and get-import arguments support cross-linklet optimization. If import-keys is a vector, it must have as many elements as sets of imports in form. If the compiler becomes interested in optimizing a reference to an imported variable, it passes back to get-import (if non-#f) the element of import-keys that corresponds to the variable’s import set. The get-import function can then return a linklet or instance that represents an instance to be provided to the compiled linklet when it is eventually instantiated; ensuring consistency between reported linklet or instance and the eventual instance is up to the caller of compile-linklet. If get-import returns #f as its first value, the compiler will be prevented from making any assumptions about the imported instance. The second result from get-import is an optional vector of keys to provide transitive information on a returned linklet’s imports (and is not allowed for a returned instance); the returned vector must have the same number of elements as the linklet has imports. When vector elements are eq? and non-#f, the compiler can assume that they correspond to the same run-time instance. A #f value for get-import is equivalent to a function that always returns two #f results.

When import-keys is not #f, then the compiler is allowed to grow or shrink the set of imported instances for the linklet. The result vector specifies the keys of the imports for the returned linklet. Any key that is #f or a linklet instance must be preserved intact, however.

If 'unsafe is included in options, then the linklet is compiled in unsafe mode: uses of safe operations within the linklet can be converted to unsafe operations on the assumption that the relevant contracts are satisfied. For example, car is converted to unsafe-car. Some substituted unsafe operations may not have directly accessible names, such as the unsafe variant of in-list that can be substituted in unsafe mode. An unsafe operation is substituted only if its (unchecked) contract is subsumed by the safe operation’s contract. The fact that the linklet is compiled in unsafe mode can be exposed through variable-reference-from-unsafe? using a variable reference produced by a #%variable-reference form within the module body.

If 'static is included in options, then the linklet must be instantiated only once; if the linklet is serialized, then any individual instance read from the serialized form must be instantiated at most once. Compilation with 'static is intended to improve the performance of references within the linklet to defined and imported variables.

If 'no-prompt is included in options, then when the resulting linklet is instantiated, the use-prompt? argument to instantiate-linklet may be treated as #f.

The symbols in options must be distinct, otherwise exn:fail:contract exception is raised.

Changed in version 7.1.0.8 of package base: Added the 'no-prompt option.

> > procedure
> > (recompile-linklet |  | linklet |  |  |  |
> > | [ | name |  |  |  |
> > |  | import-keys |  |  |  |
> > |  | get-import |  |  |  |
> > |  | options]) |  | → |  | linklet?
> linklet : linklet?
> name : any/c = #f
> import-keys : #f = #f
> | get-import |  | : |  |
> (any/c . -> . (values (or/c linklet? #f)
> (or/c vector? #f)))
> |  |  | = |  | (lambda (import-key) (values #f #f))
> | options |  | : |  | (listof (or/c 'serializable 'unsafe 'static 'no-prompt))
> |  |  | = |  | '(serializable)
> (recompile-linklet |  | linklet |  |  |  |
> |  | name |  |  |  |
> |  | import-keys |  |  |  |
> | [ | get-import |  |  |  |
> |  | options]) |  | → |  |
> linklet? |  | vector?
> linklet : linklet?
> name : any/c
> import-keys : vector?
> | get-import |  | : |  |
> (any/c . -> . (values (or/c linklet? #f)
> (or/c vector? #f)))
> |  |  | = |  | (lambda (import-key) (values #f #f))
> | options |  | : |  | (listof (or/c 'serializable 'unsafe 'static 'no-prompt))
> |  |  | = |  | '(serializable)

Like compile-linklet, but takes an already-compiled linklet and potentially optimizes it further.

Changed in version 7.1.0.6 of package base: Added the options argument.
Changed in version 7.1.0.8: Added the 'no-prompt option.

> procedure (eval-linklet linklet) → linklet?
> linklet : linklet?

Returns a variant of a linklet that is prepared for JIT compilation such that every later use of the result linklet with instantiate-linklet shares the JIT-generated code. However, the result of eval-linklet cannot be marshaled to a byte stream as part of a linklet bundle, and it cannot be used with recompile-linklet.

> > procedure
> > (instantiate-linklet |  | linklet |  |  |  |
> > |  | import-instances |  |  |  |
> > | [ | target-instance? |  |  |  |
> > |  | use-prompt?]) |  | → |  | instance?
> linklet : linklet?
> import-instances : (listof instance?)
> target-instance? : #f = #f
> use-prompt? : any/c = #t
> (instantiate-linklet |  | linklet |  |  |  |
> |  | import-instances |  |  |  |
> |  | target-instance |  |  |  |
> | [ | use-prompt?]) |  | → |  | any
> linklet : linklet?
> import-instances : (listof instance?)
> target-instance : instance?
> use-prompt? : any/c = #t

Instantiates linklet by running its definitions and expressions, using the given import-instances for its imports. The number of instances in import-instances must match the number of import sets in linklet.

If target-instance is #f or not provided, the result is a fresh instance for the linklet. If target-instance is an instance, then the instance is used and modified for the linklet definitions and expressions, and the result is the value of the last expression in the linklet.

The linklet’s exported variables are accessible in the result instance or in target-instance using the linklet’s external name for each export. If target-instance is provided as non-#f, its existing variables remain intact if they are not modified by a linklet definition.

If use-prompt? is true, then the evaluation each definition and expression in the linklet is wrapped in a prompt in the same ways as an expression in a module body.

> procedure (linklet-import-variables linklet) → (listof (listof symbol?))
> linklet : linklet?

Returns a description of a linklet’s imports. Each element of the result list corresponds to an import set as satisfied by a single instance on instantiation, and each member of the set is a variable name that is used from the corresponding imported instance.

> procedure (linklet-export-variables linklet) → (listof symbol?)
> linklet : linklet?

Returns a description of a linklet’s exports. Each element of the list corresponds to a variable that is made available by the linklet in its instance.

> procedure (linklet-directory? v) → boolean?
> v : any/c

Returns #t if v is a linklet directory, #f otherwise.

> procedure (hash->linklet-directory content) → linklet-directory?
> content : (and/c hash? hash-eq? immutable? (not/c impersonator?))

Constructs a linklet directory given mappings in the form of a hash table. Each key of content must be either a symbol or #f, each symbol must be mapped to a linklet directory, and #f must be mapped to a linklet bundle or not mapped.

> procedure (linklet-directory->hash linklet-directory)
> → (and/c hash? hash-eq? immutable? (not/c impersonator?))
> linklet-directory : linklet-directory?

Extracts the content of a linklet directory into a hash table.

> procedure (linklet-bundle? v) → boolean?
> v : any/c

Returns #t if v is a linklet bundle, #f otherwise.

> procedure (hash->linklet-bundle content) → linklet-bundle?
> content : (and/c hash? hash-eq? immutable? (not/c impersonator?))

Constructs a linklet bundle given mappings in the form of a hash table. Each key of content must be either a symbol or a fixnum. Values in the hash table are unconstrained, but the intent is that they are all linklets or values that can be recovered from write output by read.

> procedure (linklet-bundle->hash linklet-bundle)
> → (and/c hash? hash-eq? immutable? (not/c impersonator?))
> linklet-bundle : linklet-bundle?

Extracts the content of a linklet bundle into a hash table.

> procedure (instance? v) → boolean?
> v : any/c

Returns #t if v is a linklet instance, #f otherwise.

> > procedure
> > (make-instance |  | name |  |  |  |
> > | [ | data |  |  |  |
> > |  | mode] |  |  |  |
> > |  | variable-name |  |  |  |
> > |  | variable-value ... |  |  |  |
> > |  | ...) |  | → |  | instance?
> name : any/c
> data : any/c = #f
> mode : (or/c #f 'constant 'consistent) = #f
> variable-name : symbol?
> variable-value : any/c

Constructs a linklet instance directly. Besides associating an arbitrary name and data value to the instance, the instance is populated with variables as specified by variable-name and variable-value.

The optional data and mode arguments must be provided if any variable-name and variable-value arguments are provided. The mode argument is used as in instance-set-variable-value! for every variable-name.

> procedure (instance-name instance) → any/c
> instance : instance?

Returns the value associated to instance as its name—either the first value provided to make-instance or the name of a linklet that was instantiated to create the instance.

> procedure (instance-data instance) → any/c
> instance : instance?

Returns the value associated to instance as its data—either the second value provided to make-instance or the default #f.

> procedure (instance-variable-names instance) → (list symbol?)
> instance : instance?

Returns a list of all names for all variables accessible from instance.

> > procedure
> > (instance-variable-value |  | instance |  |  |  |
> > |  | name |  |  |  |
> > | [ | fail-k]) |  | → |  | any
> instance : instance?
> name : symbol?
> fail-k : any/c = (lambda () (error ....))

Returns the value of the variable exported as name from instance. If no such variable is exported, then fail-k is used in the same way as by hash-ref.

> > procedure
> > (instance-set-variable-value! |  | instance |  |  |  |
> > |  | name |  |  |  |
> > |  | v |  |  |  |
> > | [ | mode]) |  | → |  | void?
> instance : instance?
> name : symbol?
> v : any/c
> mode : (or/c #f 'constant 'consistent) = #f

Sets or creates the variable exported as name in instance so that its value is v, as long as the variable does not exist already as constant. If a variable for name exists as constant, the exn:fail:contract exception is raised.

If mode is 'constant or 'consistent, then the variable is created or changed to be constant. Furthermore, when the instance is reported for a linklet’s import though a get-import callback to compile-linklet, the compiler can assume that the variable will be constant in all future instances that are used to satisfy a linklet’s imports.

If mode is 'consistent, when the instance is reported though a callback to compile-linklet, the compiler can further assume that the variable’s value will be the same for future instances. For compilation purposes, “the same” can mean that a procedure value will have the same arity and implementation details, a structure type value will have the same configuration, a marshalable constant will be equal? to the current value, and so on.

> procedure (instance-unset-variable! instance name) → void?
> instance : instance?
> name : symbol?

Changes instance so that it does not export a variable as name, as long as name does not exist as a constant variable. If a variable for name exists as constant, the exn:fail:contract exception is raised.

> > procedure
> > (instance-describe-variable! |  | instance |  |  |  |
> > |  | name |  |  |  |
> > |  | desc-v) |  | → |  | void?
> instance : instance?
> name : symbol?
> desc-v : any/c

Registers information about name in instance that may be useful for compiling linklets where the instance is return via the get-import callback to compile-linklet. The desc-v description can be any value; the recognized descriptions depend on virtual machine, but may include the following:

- `(procedure ,arity-mask) — the value is always a procedure that is not impersonated and not a structure, and its arity in the style of procedure-arity-mask is arity-mask.

- `(procedure/succeeds ,arity-mask) — like `(procedure ,arity-mask), but for a procedure that never raises an exception of otherwise captures or escapes the calling context.

- `(procedure/pure ,arity-mask) — like `(procedure/succeeds ,arity-mask), but with no observable side effects, so a call to the procedure can be reordered.

Added in version 7.1.0.8 of package base.

> > procedure
> > (variable-reference->instance |  | varref |
> > | [ | ref-site?]) |
> → (if ref-site? (or/c instance? #f symbol?) instance?)
> varref : variable-reference?
> ref-site? : any/c = #f

Extracts the instance where the variable of varref is defined if ref-site? is #f, and returns the instance where varref itself resides if ref-site? is true. This notion of variable reference is the same as at the module level and can reflect the linklet instance that implements a particular phase of a module instance.

When ref-site? is #f, the result is #f when varref is from (#%variable-reference) with no identifier. The result is a symbol if varref refers to a primitive.

> procedure (correlated? v) → boolean?
> v : any/c
> procedure (correlated-source stx) → any
> stx : correlated?
> procedure (correlated-line stx) → (or/c exact-positive-integer? #f)
> stx : correlated?
> procedure (correlated-column stx) → (or/c exact-nonnegative-integer? #f)
> stx : correlated?
> procedure (correlated-position stx) → (or/c exact-positive-integer? #f)
> stx : correlated?
> procedure (correlated-span stx) → (or/c exact-nonnegative-integer? #f)
> stx : correlated?
> procedure (correlated-e stx) → any
> stx : correlated?
> procedure (correlated->datum stx) → any
> stx : (or/c correlated? any/c)
> procedure (datum->correlated v [srcloc]) → correlated?
> v : any/c
> | srcloc |  | : |  |
> (or/c correlated? #f
> (list/c any/c
> (or/c exact-positive-integer? #f)
> (or/c exact-nonnegative-integer? #f)
> (or/c exact-positive-integer? #f)
> (or/c exact-nonnegative-integer? #f))
> (vector/c any/c
> (or/c exact-positive-integer? #f)
> (or/c exact-nonnegative-integer? #f)
> (or/c exact-positive-integer? #f)
> (or/c exact-nonnegative-integer? #f)))
> |  |  | = |  | #f
> procedure (correlated-property stx key val) → correlated?
> stx : correlated?
> key : any/c
> val : any/c
> (correlated-property stx key) → any/c
> stx : correlated?
> key : any/c
> procedure (correlated-property-symbol-keys stx) → list?
> stx : correlated?

Like syntax?, syntax-source, syntax-line, syntax-column, syntax-position, syntax-span, syntax-e, syntax->datum, datum->syntax, syntax-property, and syntax-property-symbol-keys, but for correlated objects.

Unlike datum->syntax, datum->correlated does not recur through the given S-expression and convert pieces to correlated objects. Instead, a correlated object is simply wrapped around the immediate value. In contrast, correlated->datum recurs through its argument (which is not necessarily a correlated object) to discover any correlated objects and convert them to plain S-expressions.

top ← prev up next →