1.4 API for Making Bytecode

► |  | raco: Racket Command-Line Tools

1 | raco make: Compiling Source to Bytecode
2 | raco exe: Creating Stand-Alone Executables
3 | raco distribute: Sharing Stand-Alone Executables
4 | raco planet: Automatic Package Distribution
5 | raco pkg: Package Management
6 | raco setup: Installation Management
7 | raco decompile: Decompiling Bytecode
8 | raco demod: Demodularizing Programs
9 | raco link: Library Collection Links
10 | raco pack: Packing Library Collections
11 | raco unpack: Unpacking Library Collections
12 | raco ctool: Working with C Code
13 | raco test: Run tests
14 | raco docs: Documentation Search
15 | raco expand: Macro Expansion
16 | raco read: Reading and Pretty-Printing
17 | raco scribble: Building Documentation
18 | Adding a raco Command
19 | Installation Configuration and Search Paths

▼ | 1 | raco make: Compiling Source to Bytecode

1.1 | Running raco make
1.2 | Bytecode Files
1.3 | Dependency Files
1.4 | API for Making Bytecode
1.5 | API for Parallel Builds
1.6 | Compilation Manager Hook for Syntax Transformers
1.7 | API for Simple Bytecode Creation
1.8 | API for Bytecode Paths
1.9 | Compiling to Raw Bytecode
1.10 | API for Raw Compilation
1.11 | API for Reading Compilation Dependencies

On this page:

make-compilation-manager-load/ use-compiled-handler
managed-compile-zo
managed-compiled-context-key
make-compilation-context-error-display-handler
trust-existing-zos
make-caching-managed-compile-zo
manager-compile-notify-handler
manager-trace-handler
manager-skip-file-handler
current-path->mode
file-stamp-in-collection
file-stamp-in-paths
get-file-sha1
get-compiled-file-sha1
with-compile-output
parallel-lock-client
compile-lock->parallel-lock-client
make-compile-lock
install-module-hashes!

top ← prev up next →

#### 1.4`` API for Making Bytecode

(require compiler/cm) | package: base

The compiler/cm module implements the compilation and dependency management used by raco make and raco setup.

> > procedure
> > (make-compilation-manager-load/use-compiled-handler |
> > | [ | delete-zos-when-rkt-file-does-not-exist? |
> > |  | #:security-guard security-guard]) |
> → (path? (or/c symbol? false/c) . -> . any)
> delete-zos-when-rkt-file-does-not-exist? : any/c = #f
> security-guard : (or/c security-guard? #f) = #f

Returns a procedure suitable as a value for the current-load/use-compiled parameter. The returned procedure passes its arguments on to the current-load/use-compiled procedure that is installed when make-compilation-manager-load/use-compiled-handler is called, but first it automatically compiles a source file to a ".zo" file if

- the file is expected to contain a module (i.e., the second argument to the handler is a symbol);

- the value of each of (current-eval), (current-load), and (namespace-module-registry (current-namespace)) is the same as when make-compilation-manager-load/use-compiled-handler was called;

- the value of use-compiled-file-paths contains the first path that was present when make-compilation-manager-load/use-compiled-handler was called;

- the value of current-load/use-compiled is the result of this procedure; and

- one of the following holds:

  - the source file is newer than the ".zo" file in the first sub-directory listed in use-compiled-file-paths (at the time that make-compilation-manager-load/use-compiled-handler was called), and either no ".dep" file exists or it records a source-file SHA-1 hash that differs from the current version and source-file SHA-1 hash;

  - no ".dep" file exists next to the ".zo" file;

  - the version recorded in the ".dep" file does not match the result of (version);

  - the target machine recorded in the ".dep" file does not match the result of (current-compile-target-machine);

  - the source hash recorded in the ".dep" file does not match the current source hash;

  - one of the files listed in the ".dep" file has a ".zo" timestamp newer than the target ".zo" and use-compiled-file-check is set to 'modify-seconds;

  - the combined hashes of the dependencies recorded in the ".dep" file does not match the combined hash recorded in the ".dep" file.

If SHA-1 hashes override a timestamp-based decision to recompile the file, then the target ".zo" file’s timestamp is updated to the current time, unless the use-compiled-file-check parameter is not set to 'modify-seconds.

After the handler procedure compiles a ".zo" file, it creates a corresponding ".dep" file that lists the current version and the identification of every file that is directly required by the module in the compiled file. Additional dependencies can be installed during compilation via compiler/cm-accomplice. The ".dep" file also records the SHA-1 hash of the module’s source, and it records a combined SHA-1 hash of all of the dependencies that includes their recursive dependencies. If a bytecode file is generated by recompiling a bytecode file that was formerly compiled as machine-independent, then the ".dep" file also records the SHA-1 hash of the machine-independent form, since the recompiled module’s behavior should be exactly the same.

The special combination of (cross-installation?) as #t, (current-compile-target-machine) as #f, and (current-compiled-file-roots) having two or more elements triggers a special compilation mode. Bytecode specific to the running Racket is written to the directory determined by the first element of (current-compiled-file-roots). Bytecode specific to the cross-compilation target is written to the directory determined by the first element of (current-compiled-file-roots). By configuring (current-compiled-file-roots) so that the first element is outside a build tree and the second element is inside the build tree, cross-compilation can create a build tree suitable for the target machine while building and loading bytecode (for macro expansion, etc.) that is usable on the current machine. This mode works correctly for a build directory that starts with only source code and machine-independent bytecode.

The handler caches timestamps when it checks ".dep" files, and the cache is maintained across calls to the same handler. The cache is not consulted to compare the immediate source file to its ".zo" file, which means that the caching behavior is consistent with the caching of the default module name resolver (see current-module-name-resolver).

If use-compiled-file-paths contains an empty list when make-compilation-manager-load/use-compiled-handler is called, then an exn:fail:contract exception is raised.

If the delete-zos-when-rkt-file-does-not-exist? argument is a true value, then the returned handler will delete ".zo" files when there is no corresponding original source file.

If the security-guard argument is supplied, it is used when creating ".zo" files, ".dep" files, and "compiled/" directories, and when it adjusts the timestamps for existing files. If it is #f, then the security guard in the current-security-guard when the files are created is used (not the security guard at the point make-compilation-manager-load/use-compiled-handler is called).

The continuation of the compilation of a module is marked with a managed-compiled-context-key and the module’s source path.

Do not install the result of make-compilation-manager-load/use-compiled-handler when the current namespace contains already-loaded versions of modules that may need to be recompiled—unless the already-loaded modules are never referenced by not-yet-loaded modules. References to already-loaded modules may produce compiled files with inconsistent timestamps and/or ".dep" files with incorrect information.

The handler logs messages to the topic 'compiler/cm at the level 'info. These messages are instances of a compile-event prefab structure:

> (struct compile-event (timestamp path type) #:prefab)

The timestamp field is the time at which the event occurred in milliseconds since the epoch. The path field is the path of a file being compiled for which the event is about. The type field is a symbol which describes the action the event corresponds to. The currently logged values are 'locking, 'start-compile, 'finish-compile, and 'already-done.

Changed in version 6.1.1.8 of package base: Added identification of the compilation context via managed-compiled-context-key.
Changed in version 6.6.0.3: added check on a source’s SHA1 hash to complement the timestamp check, where the latter can be disabled via use-compile-file-check.

> > procedure
> > (managed-compile-zo |  | file |  |  |  |
> > | [ | read-src-syntax |  |  |  |
> > |  | #:security-guard security-guard]) |  | → |  | void?
> file : path-string?
> | read-src-syntax |  | : |  | (any/c input-port? . -> . syntax?)
> |  |  | = |  | read-syntax
> security-guard : (or/c security-guard? #f) = #f

Compiles the given module source file to a ".zo", installing a compilation-manager handler while the file is compiled (so that required modules are also compiled), and creating a ".dep" file to record the timestamps of immediate files used to compile the source (i.e., files required in the source).

If file is compiled from source, then read-src-syntax is used in the same way as read-syntax to read the source module. The normal read-syntax is used for any required files, however.

If security-guard is not #f, then the provided security guard is used when creating the "compiled/" directories, ".dep" and ".zo" files, and when it adjusts the timestamps of existing files. If it is #f, then the security guard in the current-security-guard when the files are created is used (not the security guard at the point managed-compile-zo is called).

While compiling file, the error-display-handler parameter is set to (make-compilation-context-error-display-handler (error-display-handler)), so that errors from uncaught exceptions will report the compilation context.

Changed in version 6.1.1.8 of package base: Added error-display-handler configuration.

> value managed-compiled-context-key : any/c

A key used as a continuation mark key by make-compilation-manager-load/use-compiled-handler for the continuation of a module compilation. The associated value is a path to the module’s source.

Added in version 6.1.1.8 of package base.

> procedure (make-compilation-context-error-display-handler orig-handlers)
> → (string? any/c . -> . void?)
> orig-handlers : (string? any/c . -> . void?)

Produces a handler suitable for use as an error-display-handler value, given an existing such value. The generated handler shows information about the compilation context when the handler’s second argument is an exception whose continuation marks include managed-compiled-context-key keys.

Added in version 6.1.1.8 of package base.

> parameter (trust-existing-zos) → boolean?
> (trust-existing-zos trust?) → void?
> trust? : any/c

A parameter that is intended for use by raco setup when installing with pre-built ".zo" files. It causes a compilation-manager load/use-compiled handler to “touch” out-of-date ".zo" files instead of re-compiling from source.

> > procedure
> > (make-caching-managed-compile-zo |
> > | [ | read-src-syntax |
> > |  | #:security-guard security-guard]) |
> → (path-string? . -> . void?)
> | read-src-syntax |  | : |  | (any/c input-port? . -> . syntax?)
> |  |  | = |  | read-syntax
> security-guard : (or/c security-guard? #f) = #f

Returns a procedure that behaves like managed-compile-zo (providing the same read-src-syntax each time), but a cache of timestamp information is preserved across calls to the procedure.

> parameter (manager-compile-notify-handler) → (path? . -> . any)
> (manager-compile-notify-handler notify) → void?
> notify : (path? . -> . any)

A parameter for a procedure of one argument that is called whenever a compilation starts. The argument to the procedure is the file’s path.

> parameter (manager-trace-handler) → (string? . -> . any)
> (manager-trace-handler notify) → void?
> notify : (string? . -> . any)

A parameter for a procedure of one argument that is called to report compilation-manager actions, such as checking a file. The argument to the procedure is a string.

The default value of the parameter logs the argument, along with current-inexact-milliseconds, to a logger named 'compiler/cm at the 'debug level.

> parameter (manager-skip-file-handler)
> → (-> path? (or/c (cons/c number? promise?) #f))
> (manager-skip-file-handler proc) → void?
> proc : (-> path? (or/c (cons/c number? promise?) #f))

A parameter whose value is called for each file that is loaded and needs recompilation. If the procedure returns a pair, then the file is skipped (i.e., not compiled); the number in the pair is used as the timestamp for the file’s bytecode, and the promise may be forced to obtain a string that is used as hash of the compiled file plus its dependencies. If the procedure returns #f, then the file is compiled as usual. The default is (lambda (x) #f).

> parameter (current-path->mode)
> → (or/c #f (-> path? (and/c path? relative-path?)))
> (current-path->mode path->mode) → void?
> path->mode : (or/c #f (-> path? (and/c path? relative-path?)))
> = | #f

Used by make-compilation-manager-load/use-compiled-handler and make-caching-managed-compile-zo to override use-compiled-file-paths for deciding where to write compiled ".zo" files. If it is #f, then the first element of use-compiled-file-paths is used. If it isn’t #f, then it is called with the original source file’s location and its result is treated the same as if it had been the first element of use-compiled-file-paths.

Note that this parameter is not used by current-load/use-compiled. So if the parameter causes ".zo" files to be placed in different directories, then the correct ".zo" file must still be communicated via use-compiled-file-paths, and one way to do that is to override current-load/use-compiled to delete ".zo" files that would cause the wrong one to be chosen right before they are loaded.

Added in version 6.4.0.14 of package base.

> procedure (file-stamp-in-collection p)
> → (or/c (cons/c number? promise?) #f)
> p : path?

Calls file-stamp-in-paths with p and (current-library-collection-paths).

> procedure (file-stamp-in-paths p paths)
> → (or/c (cons/c number? promise?) #f)
> p : path?
> paths : (listof path?)

Returns the file-modification date and delayed hash of p or its bytecode form (i.e., ".zo" file), whichever exists and is newer, if p is an extension of any path in paths (i.e., exists in the directory, a subdirectory, etc.). Otherwise, the result is #f.

This function is intended for use with manager-skip-file-handler.

> procedure (get-file-sha1 p) → (or/c string? #f)
> p : path?

Computes a SHA-1 hash for the file p; the result is #f if p cannot be opened.

> procedure (get-compiled-file-sha1 p) → (or/c string? #f)
> p : path?

Computes a SHA-1 hash for the bytecode file p, appending any dependency-describing hash available from a ".dep" file when available (i.e., the suffix on p is replaced by ".dep" to locate dependency information). The result is #f if p cannot be opened.

> procedure (with-compile-output p proc) → any
> p : path-string?
> proc : ([port input-port?] [tmp-path path?] . -> . any)

A wrapper on call-with-atomic-output-file that passes along any security guard put in place by make-compilation-manager-load/use-compiled-handler, etc.

> parameter (parallel-lock-client)
> | → |  |
> (or/c #f
> (->i ([command (or/c 'lock 'unlock)]
> [file bytes?])
> [res (command) (if (eq? command 'lock)
> boolean?
> void?)]))
> (parallel-lock-client proc) → void?
> | proc |  | : |  |
> (or/c #f
> (->i ([command (or/c 'lock 'unlock)]
> [file bytes?])
> [res (command) (if (eq? command 'lock)
> boolean?
> void?)]))

Holds the parallel compilation lock client, which is used by the result of make-compilation-manager-load/use-compiled-handler to prevent compilation races between parallel builders.

When proc is #f (the default), no checking for parallel compilation is done (and thus multiple threads or places running compilations via make-compilation-manager-load/use-compiled-handler will potentially corrupt each other’s ".zo" files).

When proc is a function, its first argument is a command, indicating if it wants to lock or unlock the path specified in the second argument.

When the proc 'lock command returns #t, the current builder has obtained the lock for zo-path. Once compilation of zo-path is complete, the builder process must release the lock by calling proc 'unlock with the exact same zo-path.

When the proc 'lock command returns #f, another parallel builder obtained the lock first and has already compiled the zo. The parallel builder should continue without compiling zo-path. (In this case, make-compilation-manager-load/use-compiled-handler’s result will not call proc with 'unlock.)

Example:

> > (let* ([lc (parallel-lock-client)]
> [zo-name #"collects/racket/compiled/draw_rkt.zo"]
> [locked? (and lc (lc 'lock zo-name))]
> [ok-to-compile? (or (not lc) locked?)])
> (dynamic-wind
> (lambda () (void))
> (lambda ()
> (when ok-to-compile?
> (printf "Do compile here ...\n")))
> (lambda ()
> (when locked?
> (lc 'unlock zo-name)))))
> Do compile here ...

> > procedure
> > (compile-lock->parallel-lock-client |  | pc |
> > | [ | cust]) |
> → (-> (or/c 'lock 'unlock) bytes? boolean?)
> pc : place-channel?
> cust : (or/c #f custodian?) = #f

Returns a function that follows the parallel-lock-client by communicating over pc. The argument must be the result of make-compile-lock.

This communication protocol implementation is not kill safe. To make it kill safe, it needs a sufficiently powerful custodian, i.e., one that is not subject to termination (unless all of the participants in the compilation are also terminated). It uses this custodian to create a thread that monitors the threads that are doing the compilation. If one of them is terminated, the presence of the custodian lets another one continue. (The custodian is also used to create a thread that manages a thread safe table.)

> procedure (make-compile-lock) → place-channel?

Creates a place-channel? that can be used with compile-lock->parallel-lock-client to avoid concurrent compilations of the same Racket source files in multiple places.

> procedure (install-module-hashes! bstr [start end]) → void?
> bstr : bytes?
> start : exact-nonnegative-integer? = 0
> end : exact-nonnegative-integer? = (bytes-length bstr)

Adjusts the bytecode representation in bstr (from bytes start to end) to install a hash code, including any submodules within the region. The existing representation should have zero bytes in place of each hash string, which is what write produces for a compiled form.

Added in version 6.3 of package base.

top ← prev up next →