The Visitor Pattern
...or how I learned to stop worrying and love architectural patterns. Sometimes.
If you'd like to have your Pharo snippets in your html export styled exactly the same way as you see them inside GT, check out the package below. Hoping these changes can get upstreamed. If/when that happens I will update this page.
[ EpMonitor current disableDuring: [ Metacello new repository: 'github://botwhytho/GtHtmlExportPharoStyling:main/src'; baseline: 'GtHtmlExportPharoStyling'; load ] ] asAsyncFuture await: AsyncFutureExecutionConfiguration default lowPriority
For a long time I had known about the visitor pattern but had never used it. That changed around 2020 when I started using Pharo and Glamorous Toolkit and saw this pattern in many places in those environments.
I can't claim to remember when I eventually started grokking it but I did at some point and have used it a couple times since then for various use cases. The main use has been when dealing with tree data structures and I've encountered those the most when parsing source code into Abstract Syntax Trees (ASTs). Below I'll share an example that I used to build this blog.
This blog is written in and published from GT and that means that you can interleave code
theAnswer := 41 + 1
with text (and other types of snippets) easily. As of the writing of the other blog post linked above, the html export of knowledge bases in GT would export the above code without syntax highlighting as a dull gray color. I explored a couple of options to try and get this working.
Since the html export already bundles JavaScript assets with the final generated pages, I thought I'd search how various sites (Github, Discord, etc.) highlight Smalltalk (and other language) code in code blocks. This led to multiple solutions both client side and server side, way too many solutions to choose from, as well as having to bundle in more dependencies. I figured this could be done with good old html + css. Also, It just didn't seem right that I'd have to use another language to style Pharo when I knew Pharo knows how to style itself. Couldn't I re-use some of that machinery that already existed?
After thinking about the problem, I remembered that GT has a way to create custom 'stylers': basically a way to add custom highlighting or even arbitrary icons/buttons inline with your code, tied to a specific class, specific AST pattern, etc. By the way, I have never seen such a powerful and easily customizable mechanism in any other IDE. Anyway, I had used this functionality before and figured that there should also be non-custom stylers that shipped with GT by default. Maybe syntax highlighting lived here?
Well it does. the class responsible for syntax highlighting in Pharo method coders and Pharo snippets is GtPharoStyler
. To have a better idea of all the different stylers my current GT image has, take a look at the class hierarchy under GtGenericPharoStyler
(going further up the class hierarchy yields even more results, but further away from this use case).
"Code used to generate image below" aTree := BrSimpleTree new nodeStencil: [ BrLabel new aptitude: BrGlamorousLabelAptitude ]; nodeDataBinder: [ :aNodeElement :aClass | aNodeElement text: aClass name ]; items: {GtGenericPharoStyler} lazy: [ :eachClass | eachClass subclasses ]. BrFrame new matchParent addChild: aTree expandAll
The GtGenericPharoStyler
class is a user of the TGtPharoProgramNodeVisitor
Trait which employs the visitor pattern to walk/visit a Pharo AST. Our subclass of interest, GtPharoStyler
styles the different parts of Pharo code accordingly as it walks the AST. An example would be the GtPharoStyler>>#visitBooleanLiteral:
method which styles boolean literal values (true/false) with a specific shade of red.
The various visitXXX: methods in that class have all the necessary information that can be re-used to style an html export with css. As an added advantage, any time a new html export happens, syntax highlighting would stay in sync with any changes that happen in GT.
So how would this work in practice? Basically, we subclass GtPharoStyler
, override most of the visiting methods to call super visitEntity: and after that we can check all the attributes (color, font weight, etc.) attached to a specific substring and output the equivalent style to a ZnHtmlOutputStream
that knows how to write html.
We can do all this manually, but that would be NO FUN! Let's take a relevant side quest into Code Generation and Meta Programming in Pharo to understand a bit better how to automate such a tedious task.
So the conclusion from the code generation link above is that unfortunately, the visitor pattern was actually NOT a good fit for this problem because GtPharoStyler
walks an AST and mutates a text with some attributes. This actions are imperative while html is declarative. The best approach I could think for this problem was to iterate through the already styled BlRunRopedText
and figure out where attributes change and act accordingly. Luckily there exists a method BlRunRopedTextIterator>>#nextSpan
that does return the next 'span' in the formatted text with the same format. Below is a code sample that generates the styled html output of the code sample itself. Go ahead, run this in GT. This is the first step in modifying GT's html export functionality so that code snippets, method definitions, etc. have the same syntax highlighting we see inside GT.
aCollection := OrderedCollection new. anHtmlStream := ZnHtmlOutputStream on: (WriteStream on: String new). aTempFile := '.' asFileReference / UUID new asInteger asString , 'html'. aCoder := GtPharoSnippetCoder forSource: thisSnippet code. anAst := aCoder astSync. aSourceText := aCoder currentSourceText copy unstyled. aStyler := GtPharoStyler new coderViewModel: aCoder asCoderViewModel; style: aSourceText ast: anAst. aSourceText readStream iterator in: [ :anIterator | [ anIterator hasNext ] whileTrue: [ anIterator nextSpan in: [ :aSpan | aCollection add: aSpan text -> aSpan attributes ] ] ]. anHtmlStream tag: #pre attributes: {#style. 'white-space: pre-wrap; word-wrap: break-word'} do: [ aCollection do: [ :anAssociation | | attributes | attributes := anAssociation value select: [ :each | {BlTextForegroundAttribute. BlFontWeightAttribute} includes: each class ] thenCollect: [ :each | each asCssStyle ]. attributes := attributes ifEmpty: [ #() ] ifNotEmpty: [ :notEmpty | {#style. notEmpty joinUsing: ';'} ]. anHtmlStream tag: #span attributes: attributes with: anAssociation key ] ]. aTempFile ensureCreateFile. aTempFile writeStreamDo: [ :aStream | aStream << anHtmlStream rawStream contents ]. WebBrowser openOn: aTempFile pathString. [ 5 second wait . aTempFile ensureDelete] asAsyncPromise
Since the code snippets in this article have syntax highlighting, I was able to re-use existing machinery for styling Pharo code in one scenario (inside GT) and rather easily be able to adapt it to a different scenario (html + css). Wheels weren't reinvented and I didn't need to depend on outside JavaScript dependencies to parse Smalltalk code being produced INSIDE a Smalltalk environment, which would be backwards. Doing the Right Thing ™ was much easier (and more FUN) than I expected.