2 Workflows

A project’s workflow reads files from the project’s assets directory and writes files to the project’s distribution directory.

A workflow’s implementation is a subclass of unlike-compiler%, which allows any asset to depend on other assets. Any workflow class is a valid value for polyglot+% in .polyglotrc.rkt.

2.1 Built-In Workflows

To save time for new users, Polyglot ships with three built-in workflows: polyglot/base%, polyglot/imperative%, and polyglot/functional%. Each may be extended, overridden, or ignored entirely.

polyglot/imperative% and polyglot/functional% both operate under the assumption that they must produce HTML5 documents from Markdown files. In addition, each Markdown file may contain application elements and library elements. Application elements are of form <script type="application/racket">...</script>, and library elements are of form <script type="text/racket">...</script>. The text content of these elements are copied to Racket modules contained in (polyglot-temp-directory). The name of each Racket module is equal to ID.rkt, where ID is the value of the id attribute of the <script> element. If the id attribute is not present or empty, a temporary name will be used instead.

The Racket modules created on disk from application elements are loaded using dynamic-require in the order they are encountered in Markdown. The intended behavior and responsibility of application element’s Racket module depends on the workflow executing it. Library elements, on the other hand, are saved to disk as-is and used according to the whims of the Racket modules produced from application elements.

2.1.1 polyglot/base

The base workflow sets rules shared by polyglot/functional% and polyglot/imperative% by specializing the behavior of unlike-compiler%.

Specifically, add-dependencies! maps the output of (discover-dependencies txexpr/expanded) to clear names using compiler and adds them to the build using (send compiler add!). As a special case, Markdown dependencies are added to the compilation but without a dependency relationship on clear. This is to avoid a circular dependency locking up a build when two pages link to each other.

The returned advance/c procedure must be used as the next step for the asset named by clear. That procedure will resolve the dependencies in txexpr/expanded, write a finished HTML5 document to a distribution, and fulfill the asset using the path of the HTML5 document.

You will likely not call this yourself, but you can use it for custom workflows derived from polyglot/base% if you aim to create HTML5 documents from Markdown using a different set of rules. The dependency resolution step is tedious, and this will take care of it for you.

For an example, see the functional workflow’s delegate implementation.

2.1.2 polyglot/imperative

The Imperative Workflow seeks application elements within Markdown files and runs them under the expectation that they will produce content as a side-effect. In that sense, app elements behave similarly to <?php echo ...; ?> in PHP.

To use module-level contracts, require the safe submodule.

The transformation does not mutate target, but does depend on side-effects:

1. Remember initial-layout as the page layout.

2. Save all Racket modules from application elements and library elements found in target-prime to disk. Remove all library elements from target-prime.

3. For each Racket module M written to disk, in order matching app elements encountered:

   1. Instantiate the module using (dynamic-require M #f)

   2. read all tagged X-expressions produced by write calls in the module as a side-effect.

   3. Replace the page layout with (dynamic-require M 'layout), if possible. Otherwise keep the existing layout.

   4. Replace the app element that sourced M with the content written by that element.

4. Return (layout target-prime), where layout is bound to the layout procedure after Step 3.

Side-effects:

• Unique and short-lived directories and Racket modules will appear in (system-temp-rel). They are deleted by the time control leaves this procedure.

• Events will appear on unlike-assets-logger. Info-level events will report summarized content fragments created by your application elements. Any output on (current-error-port) caused by script elements will apperar as error-level events.

2.1.3 polyglot/functional

To use module-level contracts, require the safe submodule.

The transformation is a fold on target into a new page target-prime that repeats these steps until no substitutions occur:

1. Save all Racket modules from application elements and library elements found in target-prime to disk.

2. Evaluate (dynamic-require path 'replace-page (lambda () (lambda (x) x))) for each path derived from an app element, in order.

3. For each replace-page procedure, functionally replace target-prime with

   (parameterize ([current-replace-element-predicate F])

   (replace-page target-prime))

   where F is a predicate that matches the application element that sourced the replace-page procedure.

4. Remove from target-prime all app and lib elements discovered in Step 1.

If this process repeats more than max-passes times, the procedure will raise exn:fail.

In addition to all side-effects produced by app or library elements, status events will appear on unlike-assets-logger. Temporary Racket modules are deleted by the time control leaves this procedure.

2.2 polyglot/txexpr: Workflows from Scratch

This module provides all bindings from the txexpr and xml modules, plus the below.

To use module-level contracts, require the safe submodule.

polyglot/txexpr offers workflow-independent tools to define where programs exist within annotated documents, and to create new documents according to how you process those programs. Combining this module with your own subclass of unlike-compiler% allows you to write static site generators that evolve independently of built-in workflows.

2.2.1 Analysis

If an element matches, the search will not descend into the child elements.

If a single element contains multiple eligible attribute values, they will all appear in the returned output.

Relative paths will not be made complete. It’s up to you to decide a base directory. This frees you from needing to use (assets-rel).

Values that appear on parent elements will come before values that appear on child elements in the output. In the event multiple dependency values appear on a single element, they will appear in the order respecting the attribute list on that element.

2.2.2 Replacing Elements

The following procedures find and replace elements using callbacks. The callback that defines replacements elements is called replace for the sake of this documentation.

The procedures are either passive or aggressive. The aggressive procedures will replace all matching elements, including the ones that appear in the replacements it already made. The passive procedures will replace all matching elements, but if the replacement produces more matching elements, they will simply leave them in the output.

Consider the following replacement rules:

In this case you want a passive procedure, because it will replace all <p> elements in a document with <section> elements, but will leave the <p> elements from the replacement as part of the intended output.

An aggressive procedure would not terminate because the substitution rules would disallow any <p> elements, even in the replacement.

This is not to say that aggressive procedures are generally unhelpful. They are meant for use cases where replacements may vary and eventually stop producing matching elements.

2.2.2.1 Passive Replacement

e.g. (tx-replace-tagged tx 'h2 (lamdba (x) `((h3 unquote (get-elements x)))))

Normally you do not need to call this directly, but it is helpful to understand how it works. This is useful if you want to build a "stepper" to inspect replacements in a broader pipeline.

A matching element can be replaced by at least zero elements, so the replace procedure must return a list of txexpr.

Return an empty list to remove the element outright (possibly leaving an empty parent element).

As a special case, if (replace? tx) is true, then the return values will be (values (replace tx) (list tx)). This is the only case where the first returned value matches (listof txexpr-element?) and not txexpr? in the range contract.

Take care to understand that while all elements with at least one matching child are reconstructed, the substitution will not account for nested children. This avoids the risk of infinite loops in the event replacement elements always includes other matching elements.

To guarentee full replacement of elements, use substitute-many-in-txexpr/loop.

Pair this with discover-dependencies to set up a workflow where discovered build-time assets are replaced with production-ready assets.

(define page (run-txexpr! (parse-markdown md-file) layout))

(define optimized (foldl (λ (dep res)

(dict-set res dep (write-optimized-to-disk! dep)))

#hash()

(discover-dependencies page)))

; Replace things like <img src="logo.png" /> with <img src="809a2d.png" />

(define production-ready (apply-manifest page optimized))

(with-output-to-file "page.html"

#:exists 'truncate

(λ ()

(displayln "<!DOCTYPE html>")

(displayln (xexpr->html page))))

2.2.2.2 Aggressive Replacement

Acts as a shorthand for substitute-many-in-txexpr/loop, except it only returns the first value.

(substitute-many-in-txexpr/loop '(p)

(λ (x) (tag-equal? x 'p))

(λ (x) '((p))))

substitute-many-in-txexpr/loop raises exn:fail if it iterates once more after performing max-replacements.

The return values are like those returned from substitute-many-in-txexpr.

Unlike the other substitution procedures, interlace-txexprs accepts multiple pairings of replace? and replace. If replace?/list-or-proc or replace/list-or-proc are not lists, they will be treated as if they were lists containing the original value as the only element. The lists must have the same number of elements, just like if you had provided them to map or foldl.

For each pass, the following happens:

1. For each replace? and replace procedure, do this:

   (substitute-many-in-txexpr/loop (cons (gensym) tx-expressions)

   replace?

   replace

   #:max-replacements max-replacements)

2. If any replacements occurred, repeat.

interlace-txexprs returns only the transformed list of tagged X-expressions, or raises exn:fail if it would exceed max-passes.

This is the procedure you would likely use to write more flexible workflows. Here is an example program that parses a Markdown file, and defines a pass to remove all script and style elements, then all elements with no children. Because the procedure will continue until no substitutions are possible, only the heading will remain.

2.2.3 Content Generation

2.3 polyglot/elements

polyglot/elements integrates the Racket module system with tagged X-expressions via dynamic-require.

Writes the text children of script to a file in dir and returns a path to the created file.

If script contains multiple separate strings as children, then they will be separated by newline characters in the output file.

Side-effect: Sends "Wrote script: ~a" info-level event to unlike-assets-logger, where ~a is the displayed form of the returned path.

Like (dynamic-require path #f), except any use of current-output-port, current-input-port, or current-error-port are reflected in the returned ports. load-script will apply make-input to an output port before loading the module to populate a buffer. That buffer may be consumed via current-input-port in the module’s top-level forms.

(define-values (readable-stdout writeable-stdin readable-stderr)

(load-script (write-script '(script ((id "blah"))

"#lang racket"

"(displayln \"What's your name?\")"

"(define name (read-line))"

"(printf \"Hi, ~a!\" name)")

(current-directory))

(λ (to-module)

(displayln "Sage" to-module))))

(displayln (read-line readable-stdout))

(displayln (read-line readable-stdout))

Take care to note that if the module found at path waits for input, you will need to provide it via make-input or else control will not leave load-script.